Issues

• Request for raw data or script for Figure 3 (Throughput vs. Batch Size)

  LLMServe/DistServe#67 · Filter by author liwuan030-lgtm opened on Jun 16, 2026

• 'cudaMemcpy(ith_context_req_token_index.ptr, ith_context_req_token_index_cpu, sizeof(int32_t) * (batch_size+1), cudaMemcpyHostToDevice)'

  LLMServe/DistServe#66 · Filter by author Sabiha1225 opened on Jun 27, 2025

• 关于论文引用的小小问题

  LLMServe/DistServe#65 · Filter by author Qiu-Jianrong opened on Apr 28, 2025

• 初始化的时候， 报错

  LLMServe/DistServe#64 · Filter by author Haoyanlong opened on Apr 23, 2025

• the inference result sentence is null

  LLMServe/DistServe#63 · Filter by author Dreamer-HIT opened on Apr 8, 2025
  4

• How to guarantee that kv cache transmission finished

  LLMServe/DistServe#61 · Filter by author HarryWu99 opened on Mar 17, 2025

• How to handle preemption cases

  LLMServe/DistServe#59 · Filter by author Mrxiangli opened on Feb 26, 2025
  2

• How to use DistServe with ray?

  LLMServe/DistServe#58 · Filter by author Liaukx opened on Feb 25, 2025
  2

• Error: Peer-to-peer access is unsupported on this platform.

  LLMServe/DistServe#57 · Filter by author hh001-cyber opened on Feb 21, 2025
  1

• When using the cmake - B build command, the following message appears in a continuous loop

  LLMServe/DistServe#56 · Filter by author 67lc opened on Feb 7, 2025
  5

• 两个节点通过ray部署，每个节点一个GPU，运行不起来。

  LLMServe/DistServe#55 · Filter by author Liufeiran123 opened on Jan 17, 2025

• How to independently measure the performance of the Prefill phase and the Decode phase？

  LLMServe/DistServe#50 · Filter by author J1nLo opened on Nov 19, 2024
  1

• offline/online serving is stuck at fetching files in paraworkers

  LLMServe/DistServe#48 · Filter by author hyuenmin-choi opened on Oct 16, 2024
  2

• Can distserve change block_size ?

  LLMServe/DistServe#47 · Filter by author WANGHanshuo1220 opened on Oct 10, 2024

• Error in building SwiftTransformers (error: more than one conversion function from "half" to a built-in type applies)

  LLMServe/DistServe#46 · Filter by author lylcyl opened on Sep 19, 2024
  3

• How to use tensorrt-llm as the inference backend for DistServe

  LLMServe/DistServe#45 · Filter by author GGBond8488 opened on Sep 18, 2024

• Error in building SwiftTransformers with torch 2.2.4

  LLMServe/DistServe#44 · Filter by author gursimar opened on Aug 28, 2024
  1

• Adding a new model?

  LLMServe/DistServe#43 · Filter by author gursimar opened on Aug 22, 2024

• What does pp_cross mean in the simulator output?

  LLMServe/DistServe#41 · Filter by author xshqhua opened on Aug 14, 2024

• 模型推理结果混乱，怎么解决。

  LLMServe/DistServe#40 · Filter by author liweiqing1997 opened on Aug 13, 2024
  3

• 编译SwiftTransformer失败

  LLMServe/DistServe#37 · Filter by author FredHuang99 opened on Aug 9, 2024
  5

• fail to run examples/offline.py , unable to download the model to reproduce

  LLMServe/DistServe#35 · Filter by author William12github opened on Aug 6, 2024
  13

• Cmake build fail

  LLMServe/DistServe#31 · Filter by author hyuenmin-choi opened on Jul 31, 2024

• Generating max_num_tokens.csv for Different Hardware Environments

  LLMServe/DistServe#30 · Filter by author village-way opened on Jul 28, 2024
  1

• Why appear 0 unaccepted, 0 waiting, 0 processing？

  LLMServe/DistServe#26 · Filter by author LeSoleilGo opened on Jul 18, 2024
  5