Skip to content

[Inference] FP8 dual gemm auto-tune and support compile parallelization #9151

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
yuanlehome merged 16 commits into from
Sep 20, 2024
Merged

[Inference] FP8 dual gemm auto-tune and support compile parallelization #9151

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
62483d3
fp8
ckl117 Aug 16, 2024
3ebc594
check
ckl117 Aug 16, 2024
7957f84
check
ckl117 Aug 16, 2024
f79baaa
check
ckl117 Aug 19, 2024
3c63b7a
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
ckl117 Aug 19, 2024
c88f712
check
ckl117 Aug 19, 2024
3f41111
cutlass fp8
ckl117 Aug 20, 2024
6fe2613
fp8 chech
ckl117 Aug 21, 2024
bfc106d
check
ckl117 Aug 25, 2024
83fe653
ffn1 tune
ckl117 Sep 19, 2024
4836310
merge
ckl117 Sep 19, 2024
6feda1c
delete
ckl117 Sep 19, 2024
fa91dad
check
ckl117 Sep 19, 2024
9ebca00
change file path
ckl117 Sep 20, 2024
e312a75
merge develop
ckl117 Sep 20, 2024
5ebdb13
top_p_sampling_reject.cu
ckl117 Sep 20, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 4 additions & 2 deletions csrc/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,11 @@ pip install -r requirements.txt

## 编译 Cuda 算子

生成 FP8的 cutlass 算子(编译耗时较长)
生成 FP8的 cutlass 算子
```shell
python generate_code_gemm_fused_kernels.py
python utils/auto_gen_fp8_fp8_gemm_fused_kernels.py

python utils/auto_gen_fp8_fp8_dual_gemm_fused_kernels.py
```

编译
Expand Down
31 changes: 0 additions & 31 deletions csrc/gpu/cutlass_kernels/fp8_gemm_fused/dispatch_dual_gemm_scale_bias_swiglu.h
View file
Open in desktop

This file was deleted.

185 changes: 0 additions & 185 deletions csrc/gpu/cutlass_kernels/fp8_gemm_fused/dual_gemm_scale_bias_swiglu_16_32_64_stages3.h
View file
Open in desktop

This file was deleted.

Loading