Commits

Branch selector

 main

User selector

benchislett

Datepicker

All time

Commit History

Commits on Nov 12, 2025

• [Perf] Refactor cudagraph_support to enable full CUDA graphs for spec decoding with FlashInfer (#28479)

  Show description for 3044195
  benchislett

  authored
  3044195

  Copy full SHA for 3044195

Commits on Nov 8, 2025

• [Feat] Drop-in Torch CUDA Profiler (#27841)

  Show description for 975676d
  benchislett

  authored
  975676d

  Copy full SHA for 975676d

Commits on Nov 7, 2025

• [Bugfix] Fix and add tests for GptOss reasoning parser (#28000)

  Show description for 1890321
  benchislett

  authored
  1890321

  Copy full SHA for 1890321

Commits on Oct 21, 2025

• [Bugfix] skip cuda graph for drafter when running with eager (#26821)

  Show description for 1974880
  benchislett

  authored
  1974880

  Copy full SHA for 1974880

• [Bugfix] Fix broken MTP weight loading for FP8 KV Scales (#27227)

  Show description for f381cf2
  benchislett

  authored
  f381cf2

  Copy full SHA for f381cf2

Commits on Oct 9, 2025

• [Bugfix] Fix CUDA graph selection bug in FlashInfer at high concurrency (#26499)

  Show description for 6e783bc
  benchislett

  authored
  6e783bc

  Copy full SHA for 6e783bc

Commits on Oct 7, 2025

• [Bugfix] Fix MTP+FlashInfer crash when trtllm kernels are available but disabled (#26361)

  Show description for caf8b1c
  

  benchislett

  and

  gemini-code-assist[bot]

  authored
  caf8b1c

  Copy full SHA for caf8b1c

• [Spec Decode] Enable efficient speculative decoding with FlashInfer-MLA (#25984)

  Show description for 3d1f676
  benchislett

  authored
  3d1f676

  Copy full SHA for 3d1f676

Commits on Oct 6, 2025

• [Perf] Add decode full-graph support to FlashInfer-MLA backend (#26313)

  Show description for f77df94
  benchislett

  authored
  f77df94

  Copy full SHA for f77df94

• [Bugfix] Allow skipping MoE in NVFP4 (fix for MTP) (#25987)

  Show description for 2161efe
  benchislett

  authored
  2161efe

  Copy full SHA for 2161efe

Commits on Oct 3, 2025

• [Bug]: Limit num_reqs in dummy_run when max_num_seqs is small (#26144)

  Show description for 6a7796e
  benchislett

  authored
  6a7796e

  Copy full SHA for 6a7796e

Commits on Sep 24, 2025

• [Spec Decode] Enable FlashInfer Spec Decoding (#25196)

  Show description for c30b405
  

  benchislett

  and

  lhsjohn

  authored
  c30b405

  Copy full SHA for c30b405

• [Bugfix] Use a separate FlashInfer workspace buffer for trtllm-gen (#25520)

  benchislett

  authored
  1983609

  Copy full SHA for 1983609

Commits on Sep 18, 2025

• [Spec Decode] Efficient padded speculation (#24539)

  Show description for b7433ca
  benchislett

  authored
  b7433ca

  Copy full SHA for b7433ca

Commits on Sep 6, 2025

• Add @benchislett to codeowner for spec decode and structured outputs (#24362)

  Show description for 0077c86
  benchislett

  authored
  0077c86

  Copy full SHA for 0077c86

• [Perf][V1] Fully overlap model execution (#23569)

  Show description for cee182b
  benchislett

  authored
  cee182b

  Copy full SHA for cee182b

Commits on Aug 16, 2025

• [Bugfix] Fix DeepSeek MTP (#22934)

  Show description for fbd8872
  benchislett

  authored
  fbd8872

  Copy full SHA for fbd8872

Commits on Aug 6, 2025

• [Perf] Parallelize fill_bitmask to accelerate high-throughput guided decoding (#21862)

  Show description for 7e6544c
  benchislett

  authored
  7e6544c

  Copy full SHA for 7e6544c

Commits on Jul 31, 2025

• [Bugfix] Fix MTP weight loading (#21941)

  benchislett

  authored
  2dff2e2

  Copy full SHA for 2dff2e2

Commits on Jun 6, 2025

• [Bugfix] Fix EAGLE vocab embedding construction for Llama 70B (#19033)

  Show description for 3465b87
  benchislett

  authored
  3465b87

  Copy full SHA for 3465b87

Commits on Jun 1, 2025

• [Bugfix] Fix EAGLE3 broken logits (#18909)

  Show description for 1bc86a3
  benchislett

  authored
  1bc86a3

  Copy full SHA for 1bc86a3

Commits on May 23, 2025

• [Spec Decode] Make EAGLE3 draft token ID mapping optional (#18488)

  Show description for 583507d
  

  benchislett

  and

  WoosukKwon

  authored
  583507d

  Copy full SHA for 583507d

Commits on Apr 30, 2025

• [V1][Feature] Enable Speculative Decoding with Structured Outputs (#14702)

  Show description for 34120f5
  benchislett

  authored
  34120f5

  Copy full SHA for 34120f5

Commits on Apr 25, 2025

• [V1][Spec Decode] EAGLE-3 Support (#16937)

  Show description for a0e619e
  

  benchislett

  and

  luyuzhe111

  authored
  a0e619e

  Copy full SHA for a0e619e

Commits on Apr 7, 2025

• [Bugfix] Fix guidance backend for Qwen models (#16210)

  Show description for 3147586
  benchislett

  authored
  3147586

  Copy full SHA for 3147586

Commits on Mar 12, 2025

• [V1][Bugfix][Spec Decode] Fix incorrect outputs in V1 speculative decoding due to batch indexing (#14645)

  Show description for 5c538c3
  benchislett

  authored
  5c538c3

  Copy full SHA for 5c538c3

Commits on Mar 5, 2025

• [Frontend] Allow return_tokens_as_token_ids to be passed as a request param (#14066)

  Show description for 32985be
  benchislett

  authored
  32985be

  Copy full SHA for 32985be

Commits on Feb 27, 2025

• [Model][Speculative Decoding] Expand DeepSeek MTP code to support k > n_predict (#13626)

  Show description for 9804145
  benchislett

  authored
  9804145

  Copy full SHA for 9804145