Skip to content

[LLM INFER] Append attn #9244

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ZHUI merged 51 commits into from
Oct 23, 2024
Merged

[LLM INFER] Append attn #9244

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
b072465
append_attention 0914
yuanlehome Sep 14, 2024
b915f95
paddle::empty to phi::allocator
yuanlehome Sep 14, 2024
9b1e1d8
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
yuanlehome Sep 19, 2024
140a509
append_attn 0919
yuanlehome Sep 20, 2024
5272b6f
0920 fix split_kv_block
yuanlehome Sep 20, 2024
a42157d
my change for merge 4 to 1
yuanlehome Sep 23, 2024
bec8eef
fix prev
yuanlehome Sep 23, 2024
8dab056
merge zhenyun 0923
yuanlehome Sep 23, 2024
d5047b5
fix prev
yuanlehome Sep 23, 2024
006a467
fix var name
yuanlehome Sep 23, 2024
73e2c06
update
yuanlehome Sep 23, 2024
a8acb2b
fix config
yuanlehome Sep 24, 2024
ec46a89
fix
yuanlehome Sep 24, 2024
cb02ee5
fix append_attn
lizhenyun01 Sep 27, 2024
83a19a6
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
yuanlehome Sep 27, 2024
37fc7da
fix --use_fake_parameter
yuanlehome Sep 27, 2024
a3b265b
refine paddle::empty(), fix memory error, support multi_stream for at…
yuanlehome Sep 29, 2024
68a09b6
fix and rename attention as append_attention
yuanlehome Sep 29, 2024
2bcd939
rename file
yuanlehome Sep 29, 2024
74941a0
fix
yuanlehome Sep 29, 2024
19a0bdb
encoder GQANEOX rope support
lizhenyun01 Oct 8, 2024
a9078cb
decoder a8w8c8 GQANEOX rope support
lizhenyun01 Oct 8, 2024
f64f962
merge get_block_shape and split_kv_block
yuanlehome Oct 8, 2024
7ba73f8
bf16 neox rope support
lizhenyun01 Oct 9, 2024
6837c23
fix diff
lizhenyun01 Oct 9, 2024
0a5ae96
separate compilation
lizhenyun01 Oct 9, 2024
e9cfc55
manual destroy stream
lizhenyun01 Oct 9, 2024
478c517
fix multi stream
yuanlehome Oct 10, 2024
aa1e96a
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
yuanlehome Oct 10, 2024
e8ddfe8
qwen/llama support weightonly
yuanlehome Oct 10, 2024
8798938
fix multi stream
yuanlehome Oct 10, 2024
f6a64d0
qwen-moe and mixtral support append_attn
yuanlehome Oct 10, 2024
2292780
refine code
yuanlehome Oct 11, 2024
036fb73
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
yuanlehome Oct 11, 2024
b85782d
decoder neox_rope_c4 support
lizhenyun01 Oct 11, 2024
9814578
instantiation of append_attn with float16
lizhenyun01 Oct 11, 2024
7a1f591
Merge branch 'develop' of https://github.com/PaddlePaddle/PaddleNLP i…
yuanlehome Oct 12, 2024
5c126ad
optimize cpu performance
yuanlehome Oct 12, 2024
2ef7c11
format code
yuanlehome Oct 12, 2024