Skip to content

[Doc] Adding recurrent policies to export tutorial #2559

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Nov 13, 2024
Merged

[Doc] Adding recurrent policies to export tutorial #2559

merged 5 commits into from
Nov 13, 2024

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Nov 13, 2024 •
edited
Loading

Update
8ea65fb 
[ghstack-poisoned]
@vmoens vmoens mentioned this pull request Nov 13, 2024
Merged
@pytorch-bot
Copy link

pytorch-bot bot commented Nov 13, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2559

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

❌ 17 New Failures, 3 Unrelated Failures

As of commit 1f20580 with merge base 50a35f6 (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
a6a2317 
ghstack-source-id: e8c6aab
Pull Request resolved: #2559
@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 13, 2024
@github-actions
Copy link

github-actions bot commented Nov 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}27$. Worsened: $\large\color{#d91a1a}4$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4285s 0.4261s 2.3469 Ops/s 2.2410 Ops/s $\color{#35bf28}+4.73\%$
test_transformed 0.6045s 0.6010s 1.6639 Ops/s 1.6124 Ops/s $\color{#35bf28}+3.19\%$
test_serial 1.3307s 1.3301s 0.7518 Ops/s 0.7442 Ops/s $\color{#35bf28}+1.02\%$
test_parallel 1.2908s 1.2805s 0.7809 Ops/s 0.7622 Ops/s $\color{#35bf28}+2.45\%$
test_step_mdp_speed[True-True-True-True-True] 0.1870ms 27.1282μs 36.8620 KOps/s 36.9929 KOps/s $\color{#d91a1a}-0.35\%$
test_step_mdp_speed[True-True-True-True-False] 0.1133ms 16.0148μs 62.4424 KOps/s 62.1578 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[True-True-True-False-True] 84.2690μs 15.1965μs 65.8044 KOps/s 65.0506 KOps/s $\color{#35bf28}+1.16\%$
test_step_mdp_speed[True-True-True-False-False] 63.5190μs 8.7903μs 113.7623 KOps/s 111.6606 KOps/s $\color{#35bf28}+1.88\%$
test_step_mdp_speed[True-True-False-True-True] 0.6097ms 28.5683μs 35.0039 KOps/s 34.6818 KOps/s $\color{#35bf28}+0.93\%$
test_step_mdp_speed[True-True-False-True-False] 52.4990μs 17.0949μs 58.4971 KOps/s 56.9622 KOps/s $\color{#35bf28}+2.69\%$
test_step_mdp_speed[True-True-False-False-True] 60.2230μs 17.0136μs 58.7764 KOps/s 58.6743 KOps/s $\color{#35bf28}+0.17\%$
test_step_mdp_speed[True-True-False-False-False] 46.0960μs 10.4958μs 95.2759 KOps/s 94.2457 KOps/s $\color{#35bf28}+1.09\%$
test_step_mdp_speed[True-False-True-True-True] 0.1077ms 30.1489μs 33.1687 KOps/s 33.2404 KOps/s $\color{#d91a1a}-0.22\%$
test_step_mdp_speed[True-False-True-True-False] 55.2240μs 18.8655μs 53.0069 KOps/s 52.3943 KOps/s $\color{#35bf28}+1.17\%$
test_step_mdp_speed[True-False-True-False-True] 70.1020μs 16.7916μs 59.5536 KOps/s 59.1795 KOps/s $\color{#35bf28}+0.63\%$
test_step_mdp_speed[True-False-True-False-False] 52.5080μs 10.4918μs 95.3128 KOps/s 95.8179 KOps/s $\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-False-False-True-True] 67.0560μs 31.2053μs 32.0459 KOps/s 31.5184 KOps/s $\color{#35bf28}+1.67\%$
test_step_mdp_speed[True-False-False-True-False] 69.2000μs 20.2939μs 49.2758 KOps/s 48.4414 KOps/s $\color{#35bf28}+1.72\%$
test_step_mdp_speed[True-False-False-False-True] 87.6350μs 18.2159μs 54.8972 KOps/s 54.4726 KOps/s $\color{#35bf28}+0.78\%$
test_step_mdp_speed[True-False-False-False-False] 62.9390μs 11.9793μs 83.4771 KOps/s 82.2883 KOps/s $\color{#35bf28}+1.44\%$
test_step_mdp_speed[False-True-True-True-True] 78.8190μs 30.2152μs 33.0959 KOps/s 33.0822 KOps/s $\color{#35bf28}+0.04\%$
test_step_mdp_speed[False-True-True-True-False] 64.4710μs 18.7634μs 53.2953 KOps/s 52.5926 KOps/s $\color{#35bf28}+1.34\%$
test_step_mdp_speed[False-True-True-False-True] 89.6290μs 19.5828μs 51.0651 KOps/s 51.7438 KOps/s $\color{#d91a1a}-1.31\%$
test_step_mdp_speed[False-True-True-False-False] 40.3260μs 11.6835μs 85.5909 KOps/s 84.5911 KOps/s $\color{#35bf28}+1.18\%$
test_step_mdp_speed[False-True-False-True-True] 81.5530μs 31.7787μs 31.4677 KOps/s 30.9294 KOps/s $\color{#35bf28}+1.74\%$
test_step_mdp_speed[False-True-False-True-False] 56.9570μs 20.4975μs 48.7864 KOps/s 48.6485 KOps/s $\color{#35bf28}+0.28\%$
test_step_mdp_speed[False-True-False-False-True] 3.1108ms 20.9240μs 47.7920 KOps/s 47.1995 KOps/s $\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-True-False-False-False] 47.9300μs 13.2328μs 75.5700 KOps/s 74.5322 KOps/s $\color{#35bf28}+1.39\%$
test_step_mdp_speed[False-False-True-True-True] 91.8630μs 33.4075μs 29.9334 KOps/s 29.7820 KOps/s $\color{#35bf28}+0.51\%$
test_step_mdp_speed[False-False-True-True-False] 77.1350μs 22.1603μs 45.1258 KOps/s 45.4230 KOps/s $\color{#d91a1a}-0.65\%$
test_step_mdp_speed[False-False-True-False-True] 72.0650μs 21.2023μs 47.1646 KOps/s 47.4246 KOps/s $\color{#d91a1a}-0.55\%$
test_step_mdp_speed[False-False-True-False-False] 64.6710μs 13.4092μs 74.5757 KOps/s 73.9835 KOps/s $\color{#35bf28}+0.80\%$
test_step_mdp_speed[False-False-False-True-True] 67.3670μs 34.8562μs 28.6893 KOps/s 28.9210 KOps/s $\color{#d91a1a}-0.80\%$
test_step_mdp_speed[False-False-False-True-False] 90.2090μs 23.3453μs 42.8352 KOps/s 42.5792 KOps/s $\color{#35bf28}+0.60\%$
test_step_mdp_speed[False-False-False-False-True] 74.4200μs 22.2378μs 44.9685 KOps/s 45.2413 KOps/s $\color{#d91a1a}-0.60\%$
test_step_mdp_speed[False-False-False-False-False] 40.4360μs 14.8233μs 67.4613 KOps/s 67.8898 KOps/s $\color{#d91a1a}-0.63\%$
test_values[generalized_advantage_estimate-True-True] 13.0438ms 9.7184ms 102.8979 Ops/s 103.4772 Ops/s $\color{#d91a1a}-0.56\%$
test_values[vec_generalized_advantage_estimate-True-True] 36.0261ms 33.7269ms 29.6499 Ops/s 28.6558 Ops/s $\color{#35bf28}+3.47\%$
test_values[td0_return_estimate-False-False] 0.2441ms 0.1754ms 5.7007 KOps/s 5.7961 KOps/s $\color{#d91a1a}-1.65\%$
test_values[td1_return_estimate-False-False] 27.1025ms 25.1222ms 39.8054 Ops/s 41.0236 Ops/s $\color{#d91a1a}-2.97\%$
test_values[vec_td1_return_estimate-False-False] 35.8202ms 33.8255ms 29.5635 Ops/s 27.9061 Ops/s $\textbf{\color{#35bf28}+5.94\%}$
test_values[td_lambda_return_estimate-True-False] 37.2879ms 36.1627ms 27.6528 Ops/s 28.6134 Ops/s $\color{#d91a1a}-3.36\%$
test_values[vec_td_lambda_return_estimate-True-False] 35.8657ms 33.8307ms 29.5589 Ops/s 27.5189 Ops/s $\textbf{\color{#35bf28}+7.41\%}$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.6900ms 8.4673ms 118.1019 Ops/s 116.5692 Ops/s $\color{#35bf28}+1.31\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.1824ms 1.9835ms 504.1588 Ops/s 487.3825 Ops/s $\color{#35bf28}+3.44\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4960ms 0.3613ms 2.7675 KOps/s 2.7528 KOps/s $\color{#35bf28}+0.53\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 47.9081ms 44.7312ms 22.3557 Ops/s 20.8875 Ops/s $\textbf{\color{#35bf28}+7.03\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.8939ms 3.0460ms 328.3043 Ops/s 327.4578 Ops/s $\color{#35bf28}+0.26\%$
test_dqn_speed[False-None] 5.8587ms 1.3403ms 746.1114 Ops/s 751.1984 Ops/s $\color{#d91a1a}-0.68\%$
test_dqn_speed[False-backward] 1.8979ms 1.8337ms 545.3387 Ops/s 551.9312 Ops/s $\color{#d91a1a}-1.19\%$
test_dqn_speed[True-None] 1.1614ms 0.4636ms 2.1569 KOps/s 2.1481 KOps/s $\color{#35bf28}+0.41\%$
test_dqn_speed[True-backward] 0.9987ms 0.8963ms 1.1157 KOps/s 1.1237 KOps/s $\color{#d91a1a}-0.71\%$
test_dqn_speed[reduce-overhead-None] 0.5686ms 0.4652ms 2.1494 KOps/s 2.1137 KOps/s $\color{#35bf28}+1.69\%$
test_dqn_speed[reduce-overhead-backward] 0.9409ms 0.8850ms 1.1299 KOps/s 1.1214 KOps/s $\color{#35bf28}+0.76\%$
test_ddpg_speed[False-None] 3.4468ms 2.7757ms 360.2723 Ops/s 361.2417 Ops/s $\color{#d91a1a}-0.27\%$
test_ddpg_speed[False-backward] 4.0416ms 3.9216ms 254.9980 Ops/s 255.6319 Ops/s $\color{#d91a1a}-0.25\%$
test_ddpg_speed[True-None] 1.3320ms 1.0091ms 990.9792 Ops/s 991.9367 Ops/s $\color{#d91a1a}-0.10\%$
test_ddpg_speed[True-backward] 1.9427ms 1.8970ms 527.1460 Ops/s 517.6481 Ops/s $\color{#35bf28}+1.83\%$
test_ddpg_speed[reduce-overhead-None] 1.4608ms 1.0065ms 993.5717 Ops/s 994.5645 Ops/s $\color{#d91a1a}-0.10\%$
test_ddpg_speed[reduce-overhead-backward] 1.9678ms 1.9084ms 524.0065 Ops/s 523.2607 Ops/s $\color{#35bf28}+0.14\%$
test_sac_speed[False-None] 9.7909ms 7.8752ms 126.9809 Ops/s 126.7864 Ops/s $\color{#35bf28}+0.15\%$
test_sac_speed[False-backward] 11.0180ms 10.6547ms 93.8550 Ops/s 93.9570 Ops/s $\color{#d91a1a}-0.11\%$
test_sac_speed[True-None] 2.0342ms 1.8280ms 547.0451 Ops/s 546.0562 Ops/s $\color{#35bf28}+0.18\%$
test_sac_speed[True-backward] 3.5855ms 3.5031ms 285.4644 Ops/s 284.4049 Ops/s $\color{#35bf28}+0.37\%$
test_sac_speed[reduce-overhead-None] 2.2911ms 1.8185ms 549.9130 Ops/s 539.3931 Ops/s $\color{#35bf28}+1.95\%$
test_sac_speed[reduce-overhead-backward] 3.5518ms 3.5030ms 285.4688 Ops/s 280.5960 Ops/s $\color{#35bf28}+1.74\%$
test_redq_speed[False-None] 14.7617ms 12.9805ms 77.0386 Ops/s 75.8290 Ops/s $\color{#35bf28}+1.60\%$
test_redq_speed[False-backward] 23.0253ms 22.1207ms 45.2066 Ops/s 44.6481 Ops/s $\color{#35bf28}+1.25\%$
test_redq_speed[True-None] 5.3282ms 4.7641ms 209.9010 Ops/s 188.6111 Ops/s $\textbf{\color{#35bf28}+11.29\%}$
test_redq_speed[True-backward] 13.3426ms 12.2018ms 81.9551 Ops/s 78.1705 Ops/s $\color{#35bf28}+4.84\%$
test_redq_speed[reduce-overhead-None] 4.9603ms 4.4864ms 222.8939 Ops/s 197.2567 Ops/s $\textbf{\color{#35bf28}+13.00\%}$
test_redq_speed[reduce-overhead-backward] 12.7369ms 11.8729ms 84.2256 Ops/s 77.0089 Ops/s $\textbf{\color{#35bf28}+9.37\%}$
test_redq_deprec_speed[False-None] 14.2926ms 12.7373ms 78.5095 Ops/s 73.2166 Ops/s $\textbf{\color{#35bf28}+7.23\%}$
test_redq_deprec_speed[False-backward] 21.0075ms 18.9329ms 52.8182 Ops/s 49.4972 Ops/s $\textbf{\color{#35bf28}+6.71\%}$
test_redq_deprec_speed[True-None] 4.1782ms 3.5420ms 282.3253 Ops/s 276.5061 Ops/s $\color{#35bf28}+2.10\%$
test_redq_deprec_speed[True-backward] 8.4255ms 8.1783ms 122.2750 Ops/s 125.3714 Ops/s $\color{#d91a1a}-2.47\%$
test_redq_deprec_speed[reduce-overhead-None] 3.8700ms 3.5318ms 283.1454 Ops/s 277.0927 Ops/s $\color{#35bf28}+2.18\%$
test_redq_deprec_speed[reduce-overhead-backward] 8.8245ms 8.0091ms 124.8576 Ops/s 120.7797 Ops/s $\color{#35bf28}+3.38\%$
test_td3_speed[False-None] 8.9203ms 7.7468ms 129.0852 Ops/s 127.7791 Ops/s $\color{#35bf28}+1.02\%$
test_td3_speed[False-backward] 10.7606ms 10.2170ms 97.8758 Ops/s 96.6397 Ops/s $\color{#35bf28}+1.28\%$
test_td3_speed[True-None] 1.7867ms 1.6940ms 590.3140 Ops/s 575.8237 Ops/s $\color{#35bf28}+2.52\%$
test_td3_speed[True-backward] 3.4953ms 3.3460ms 298.8602 Ops/s 297.1542 Ops/s $\color{#35bf28}+0.57\%$
test_td3_speed[reduce-overhead-None] 1.8801ms 1.7009ms 587.9188 Ops/s 579.3316 Ops/s $\color{#35bf28}+1.48\%$
test_td3_speed[reduce-overhead-backward] 3.3580ms 3.2833ms 304.5709 Ops/s 301.2913 Ops/s $\color{#35bf28}+1.09\%$
test_cql_speed[False-None] 47.5614ms 37.5024ms 26.6650 Ops/s 28.0521 Ops/s $\color{#d91a1a}-4.94\%$
test_cql_speed[False-backward] 62.5617ms 48.2605ms 20.7209 Ops/s 21.4086 Ops/s $\color{#d91a1a}-3.21\%$
test_cql_speed[True-None] 17.0935ms 15.6429ms 63.9268 Ops/s 62.9628 Ops/s $\color{#35bf28}+1.53\%$
test_cql_speed[True-backward] 23.7810ms 22.4446ms 44.5541 Ops/s 41.9721 Ops/s $\textbf{\color{#35bf28}+6.15\%}$
test_cql_speed[reduce-overhead-None] 16.1320ms 15.7556ms 63.4697 Ops/s 58.1709 Ops/s $\textbf{\color{#35bf28}+9.11\%}$
test_cql_speed[reduce-overhead-backward] 23.2075ms 22.2857ms 44.8717 Ops/s 40.3023 Ops/s $\textbf{\color{#35bf28}+11.34\%}$
test_a2c_speed[False-None] 9.5531ms 7.0422ms 142.0003 Ops/s 128.7020 Ops/s $\textbf{\color{#35bf28}+10.33\%}$
test_a2c_speed[False-backward] 15.7884ms 14.1746ms 70.5487 Ops/s 66.1080 Ops/s $\textbf{\color{#35bf28}+6.72\%}$
test_a2c_speed[True-None] 3.5779ms 3.2893ms 304.0167 Ops/s 287.3338 Ops/s $\textbf{\color{#35bf28}+5.81\%}$
test_a2c_speed[True-backward] 10.3266ms 9.8635ms 101.3836 Ops/s 95.8489 Ops/s $\textbf{\color{#35bf28}+5.77\%}$
test_a2c_speed[reduce-overhead-None] 3.7457ms 3.3017ms 302.8755 Ops/s 290.3953 Ops/s $\color{#35bf28}+4.30\%$
test_a2c_speed[reduce-overhead-backward] 10.6446ms 9.8113ms 101.9237 Ops/s 100.2858 Ops/s $\color{#35bf28}+1.63\%$
test_ppo_speed[False-None] 7.9468ms 7.3143ms 136.7193 Ops/s 126.9027 Ops/s $\textbf{\color{#35bf28}+7.74\%}$
test_ppo_speed[False-backward] 15.4826ms 14.6981ms 68.0358 Ops/s 63.6057 Ops/s $\textbf{\color{#35bf28}+6.97\%}$
test_ppo_speed[True-None] 4.3215ms 3.6726ms 272.2887 Ops/s 264.6969 Ops/s $\color{#35bf28}+2.87\%$
test_ppo_speed[True-backward] 9.9594ms 9.5886ms 104.2904 Ops/s 101.7222 Ops/s $\color{#35bf28}+2.52\%$
test_ppo_speed[reduce-overhead-None] 4.2841ms 3.6718ms 272.3463 Ops/s 269.3441 Ops/s $\color{#35bf28}+1.11\%$
test_ppo_speed[reduce-overhead-backward] 9.8432ms 9.5024ms 105.2361 Ops/s 96.5327 Ops/s $\textbf{\color{#35bf28}+9.02\%}$
test_reinforce_speed[False-None] 7.4948ms 6.4011ms 156.2240 Ops/s 145.2748 Ops/s $\textbf{\color{#35bf28}+7.54\%}$
test_reinforce_speed[False-backward] 9.8637ms 9.5863ms 104.3158 Ops/s 97.3474 Ops/s $\textbf{\color{#35bf28}+7.16\%}$
test_reinforce_speed[True-None] 3.3198ms 2.6343ms 379.6018 Ops/s 348.2940 Ops/s $\textbf{\color{#35bf28}+8.99\%}$
test_reinforce_speed[True-backward] 9.3556ms 8.7017ms 114.9205 Ops/s 114.9835 Ops/s $\color{#d91a1a}-0.05\%$
test_reinforce_speed[reduce-overhead-None] 3.0480ms 2.6433ms 378.3090 Ops/s 307.4885 Ops/s $\textbf{\color{#35bf28}+23.03\%}$
test_reinforce_speed[reduce-overhead-backward] 9.3780ms 8.6677ms 115.3708 Ops/s 107.7419 Ops/s $\textbf{\color{#35bf28}+7.08\%}$
test_iql_speed[False-None] 33.9249ms 31.9517ms 31.2972 Ops/s 30.2330 Ops/s $\color{#35bf28}+3.52\%$
test_iql_speed[False-backward] 45.7870ms 44.5947ms 22.4242 Ops/s 21.2000 Ops/s $\textbf{\color{#35bf28}+5.77\%}$
test_iql_speed[True-None] 12.4723ms 10.9609ms 91.2332 Ops/s 91.4091 Ops/s $\color{#d91a1a}-0.19\%$
test_iql_speed[True-backward] 22.0462ms 21.4167ms 46.6926 Ops/s 45.6733 Ops/s $\color{#35bf28}+2.23\%$
test_iql_speed[reduce-overhead-None] 11.4875ms 10.7792ms 92.7709 Ops/s 91.0945 Ops/s $\color{#35bf28}+1.84\%$
test_iql_speed[reduce-overhead-backward] 23.8200ms 22.0617ms 45.3275 Ops/s 45.4541 Ops/s $\color{#d91a1a}-0.28\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 5.1380ms 4.8307ms 207.0099 Ops/s 189.4938 Ops/s $\textbf{\color{#35bf28}+9.24\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2810ms 0.5162ms 1.9372 KOps/s 1.9309 KOps/s $\color{#35bf28}+0.33\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.8405ms 0.4808ms 2.0798 KOps/s 2.0620 KOps/s $\color{#35bf28}+0.86\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.6413ms 4.5603ms 219.2831 Ops/s 209.4166 Ops/s $\color{#35bf28}+4.71\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 1.0069ms 0.4961ms 2.0156 KOps/s 1.9992 KOps/s $\color{#35bf28}+0.82\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6982ms 0.4680ms 2.1367 KOps/s 2.1039 KOps/s $\color{#35bf28}+1.56\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.5667ms 1.6425ms 608.8428 Ops/s 607.6904 Ops/s $\color{#35bf28}+0.19\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.3956ms 1.5952ms 626.8967 Ops/s 632.7482 Ops/s $\color{#d91a1a}-0.92\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.8833ms 4.9416ms 202.3645 Ops/s 204.7809 Ops/s $\color{#d91a1a}-1.18\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 3.1000ms 0.6465ms 1.5469 KOps/s 1.5437 KOps/s $\color{#35bf28}+0.21\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.8570ms 0.6184ms 1.6171 KOps/s 1.6106 KOps/s $\color{#35bf28}+0.40\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.5674ms 4.8975ms 204.1842 Ops/s 205.5275 Ops/s $\color{#d91a1a}-0.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 2.9707ms 0.5416ms 1.8463 KOps/s 1.9597 KOps/s $\textbf{\color{#d91a1a}-5.79\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7715ms 0.4968ms 2.0129 KOps/s 1.9621 KOps/s $\color{#35bf28}+2.59\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 5.0104ms 4.6610ms 214.5474 Ops/s 211.9295 Ops/s $\color{#35bf28}+1.24\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 3.1516ms 0.4957ms 2.0173 KOps/s 2.0245 KOps/s $\color{#d91a1a}-0.36\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.8377ms 0.4721ms 2.1180 KOps/s 2.1312 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.9368ms 5.0509ms 197.9863 Ops/s 206.3925 Ops/s $\color{#d91a1a}-4.07\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9578ms 0.6461ms 1.5477 KOps/s 1.5320 KOps/s $\color{#35bf28}+1.02\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.3283ms 0.6384ms 1.5664 KOps/s 1.6237 KOps/s $\color{#d91a1a}-3.53\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.4384ms 4.4118ms 226.6674 Ops/s 249.1308 Ops/s $\textbf{\color{#d91a1a}-9.02\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 8.0515ms 2.3778ms 420.5608 Ops/s 443.7628 Ops/s $\textbf{\color{#d91a1a}-5.23\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 4.9760ms 1.3128ms 761.7085 Ops/s 804.5073 Ops/s $\textbf{\color{#d91a1a}-5.32\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4976s 14.4336ms 69.2826 Ops/s 36.7958 Ops/s $\textbf{\color{#35bf28}+88.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 9.2002ms 2.4428ms 409.3653 Ops/s 424.1734 Ops/s $\color{#d91a1a}-3.49\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 2.7839ms 1.2810ms 780.6187 Ops/s 726.7360 Ops/s $\textbf{\color{#35bf28}+7.41\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 6.8697ms 4.6175ms 216.5685 Ops/s 216.2273 Ops/s $\color{#35bf28}+0.16\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 6.3978ms 2.4771ms 403.6997 Ops/s 407.3763 Ops/s $\color{#d91a1a}-0.90\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 7.3488ms 1.5217ms 657.1635 Ops/s 655.8201 Ops/s $\color{#35bf28}+0.20\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 11.9119ms 11.1717ms 89.5123 Ops/s 87.0642 Ops/s $\color{#35bf28}+2.81\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 17.1203ms 14.4722ms 69.0979 Ops/s 67.8685 Ops/s $\color{#35bf28}+1.81\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 21.9255ms 20.0315ms 49.9214 Ops/s 49.9576 Ops/s $\color{#d91a1a}-0.07\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 15.5723ms 14.6048ms 68.4707 Ops/s 67.7247 Ops/s $\color{#35bf28}+1.10\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 21.2664ms 19.9064ms 50.2351 Ops/s 50.4880 Ops/s $\color{#d91a1a}-0.50\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 16.3540ms 15.6559ms 63.8735 Ops/s 62.5174 Ops/s $\color{#35bf28}+2.17\%$

@github-actions
Copy link

github-actions bot commented Nov 13, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 149. Improved: $\large\color{#35bf28}15$. Worsened: $\large\color{#d91a1a}13$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7197s 0.7190s 1.3908 Ops/s 1.3332 Ops/s $\color{#35bf28}+4.32\%$
test_transformed 1.0543s 0.9778s 1.0227 Ops/s 1.0225 Ops/s $\color{#35bf28}+0.02\%$
test_serial 2.1582s 2.0752s 0.4819 Ops/s 0.4708 Ops/s $\color{#35bf28}+2.36\%$
test_parallel 1.9955s 1.9244s 0.5196 Ops/s 0.5199 Ops/s $\color{#d91a1a}-0.04\%$
test_step_mdp_speed[True-True-True-True-True] 0.2626ms 34.4749μs 29.0066 KOps/s 29.6442 KOps/s $\color{#d91a1a}-2.15\%$
test_step_mdp_speed[True-True-True-True-False] 48.8210μs 19.5515μs 51.1471 KOps/s 52.1230 KOps/s $\color{#d91a1a}-1.87\%$
test_step_mdp_speed[True-True-True-False-True] 55.4810μs 19.0493μs 52.4954 KOps/s 54.0637 KOps/s $\color{#d91a1a}-2.90\%$
test_step_mdp_speed[True-True-True-False-False] 66.8410μs 11.1014μs 90.0791 KOps/s 92.2948 KOps/s $\color{#d91a1a}-2.40\%$
test_step_mdp_speed[True-True-False-True-True] 85.3520μs 37.2930μs 26.8147 KOps/s 27.9398 KOps/s $\color{#d91a1a}-4.03\%$
test_step_mdp_speed[True-True-False-True-False] 49.4010μs 21.8859μs 45.6916 KOps/s 47.7174 KOps/s $\color{#d91a1a}-4.25\%$
test_step_mdp_speed[True-True-False-False-True] 56.0010μs 21.6739μs 46.1385 KOps/s 47.9270 KOps/s $\color{#d91a1a}-3.73\%$
test_step_mdp_speed[True-True-False-False-False] 40.0510μs 13.2188μs 75.6500 KOps/s 78.0437 KOps/s $\color{#d91a1a}-3.07\%$
test_step_mdp_speed[True-False-True-True-True] 67.3110μs 38.9762μs 25.6567 KOps/s 26.2269 KOps/s $\color{#d91a1a}-2.17\%$
test_step_mdp_speed[True-False-True-True-False] 67.4120μs 23.5892μs 42.3923 KOps/s 42.4004 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-False-True-False-True] 52.8310μs 21.2657μs 47.0240 KOps/s 48.2428 KOps/s $\color{#d91a1a}-2.53\%$
test_step_mdp_speed[True-False-True-False-False] 45.9810μs 13.0483μs 76.6386 KOps/s 77.7757 KOps/s $\color{#d91a1a}-1.46\%$
test_step_mdp_speed[True-False-False-True-True] 82.1220μs 40.6971μs 24.5718 KOps/s 25.0917 KOps/s $\color{#d91a1a}-2.07\%$
test_step_mdp_speed[True-False-False-True-False] 59.9820μs 25.3778μs 39.4045 KOps/s 39.7120 KOps/s $\color{#d91a1a}-0.77\%$
test_step_mdp_speed[True-False-False-False-True] 53.9510μs 23.2040μs 43.0959 KOps/s 44.8478 KOps/s $\color{#d91a1a}-3.91\%$
test_step_mdp_speed[True-False-False-False-False] 44.6310μs 14.6071μs 68.4599 KOps/s 68.4104 KOps/s $\color{#35bf28}+0.07\%$
test_step_mdp_speed[False-True-True-True-True] 81.1120μs 38.3295μs 26.0896 KOps/s 26.4590 KOps/s $\color{#d91a1a}-1.40\%$
test_step_mdp_speed[False-True-True-True-False] 50.9210μs 23.4370μs 42.6675 KOps/s 42.4717 KOps/s $\color{#35bf28}+0.46\%$
test_step_mdp_speed[False-True-True-False-True] 0.1850ms 24.3638μs 41.0446 KOps/s 41.4334 KOps/s $\color{#d91a1a}-0.94\%$
test_step_mdp_speed[False-True-True-False-False] 36.6410μs 14.7696μs 67.7065 KOps/s 69.4347 KOps/s $\color{#d91a1a}-2.49\%$
test_step_mdp_speed[False-True-False-True-True] 82.8120μs 41.3609μs 24.1775 KOps/s 25.5666 KOps/s $\textbf{\color{#d91a1a}-5.43\%}$
test_step_mdp_speed[False-True-False-True-False] 52.0510μs 25.4412μs 39.3063 KOps/s 40.0433 KOps/s $\color{#d91a1a}-1.84\%$
test_step_mdp_speed[False-True-False-False-True] 3.7227ms 26.3079μs 38.0113 KOps/s 39.0694 KOps/s $\color{#d91a1a}-2.71\%$
test_step_mdp_speed[False-True-False-False-False] 49.1510μs 16.4778μs 60.6878 KOps/s 61.1880 KOps/s $\color{#d91a1a}-0.82\%$
test_step_mdp_speed[False-False-True-True-True] 71.2020μs 42.4172μs 23.5753 KOps/s 24.2523 KOps/s $\color{#d91a1a}-2.79\%$
test_step_mdp_speed[False-False-True-True-False] 59.5420μs 27.1951μs 36.7714 KOps/s 37.2521 KOps/s $\color{#d91a1a}-1.29\%$
test_step_mdp_speed[False-False-True-False-True] 78.4620μs 26.3253μs 37.9863 KOps/s 39.3760 KOps/s $\color{#d91a1a}-3.53\%$
test_step_mdp_speed[False-False-True-False-False] 48.1410μs 16.3389μs 61.2036 KOps/s 61.3306 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-False-False-True-True] 76.7210μs 42.8167μs 23.3554 KOps/s 23.5539 KOps/s $\color{#d91a1a}-0.84\%$
test_step_mdp_speed[False-False-False-True-False] 68.5510μs 29.3320μs 34.0925 KOps/s 34.7072 KOps/s $\color{#d91a1a}-1.77\%$
test_step_mdp_speed[False-False-False-False-True] 53.2310μs 27.8689μs 35.8822 KOps/s 36.2332 KOps/s $\color{#d91a1a}-0.97\%$
test_step_mdp_speed[False-False-False-False-False] 49.5310μs 18.2592μs 54.7670 KOps/s 55.1829 KOps/s $\color{#d91a1a}-0.75\%$
test_values[generalized_advantage_estimate-True-True] 24.3619ms 23.8088ms 42.0013 Ops/s 41.4320 Ops/s $\color{#35bf28}+1.37\%$
test_values[vec_generalized_advantage_estimate-True-True] 0.1017s 2.9071ms 343.9818 Ops/s 359.7567 Ops/s $\color{#d91a1a}-4.38\%$
test_values[td0_return_estimate-False-False] 82.3420μs 63.2831μs 15.8020 KOps/s 14.9345 KOps/s $\textbf{\color{#35bf28}+5.81\%}$
test_values[td1_return_estimate-False-False] 53.4961ms 53.0749ms 18.8413 Ops/s 18.7279 Ops/s $\color{#35bf28}+0.61\%$
test_values[vec_td1_return_estimate-False-False] 1.2559ms 1.0523ms 950.3089 Ops/s 945.1112 Ops/s $\color{#35bf28}+0.55\%$
test_values[td_lambda_return_estimate-True-False] 85.8034ms 84.8500ms 11.7855 Ops/s 11.6315 Ops/s $\color{#35bf28}+1.32\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.1953ms 1.0502ms 952.1573 Ops/s 939.4380 Ops/s $\color{#35bf28}+1.35\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 24.1758ms 23.8187ms 41.9838 Ops/s 42.5333 Ops/s $\color{#d91a1a}-1.29\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0044ms 0.7304ms 1.3691 KOps/s 1.3755 KOps/s $\color{#d91a1a}-0.46\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7809ms 0.6379ms 1.5677 KOps/s 1.5594 KOps/s $\color{#35bf28}+0.54\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.5980ms 1.4509ms 689.2046 Ops/s 688.2203 Ops/s $\color{#35bf28}+0.14\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.7915ms 0.6509ms 1.5364 KOps/s 1.5025 KOps/s $\color{#35bf28}+2.25\%$
test_dqn_speed[False-None] 7.1296ms 1.2721ms 786.1104 Ops/s 752.8398 Ops/s $\color{#35bf28}+4.42\%$
test_dqn_speed[False-backward] 1.8799ms 1.7736ms 563.8347 Ops/s 533.6291 Ops/s $\textbf{\color{#35bf28}+5.66\%}$
test_dqn_speed[True-None] 0.7205ms 0.5523ms 1.8105 KOps/s 1.8069 KOps/s $\color{#35bf28}+0.20\%$
test_dqn_speed[True-backward] 1.0432ms 1.0007ms 999.2703 Ops/s 808.0280 Ops/s $\textbf{\color{#35bf28}+23.67\%}$
test_dqn_speed[reduce-overhead-None] 0.9194ms 0.5628ms 1.7770 KOps/s 1.7145 KOps/s $\color{#35bf28}+3.64\%$
test_dqn_speed[reduce-overhead-backward] 1.1556ms 1.0072ms 992.8826 Ops/s 971.8604 Ops/s $\color{#35bf28}+2.16\%$
test_ddpg_speed[False-None] 3.4246ms 2.6194ms 381.7640 Ops/s 373.2403 Ops/s $\color{#35bf28}+2.28\%$
test_ddpg_speed[False-backward] 4.1737ms 3.8467ms 259.9662 Ops/s 253.1531 Ops/s $\color{#35bf28}+2.69\%$
test_ddpg_speed[True-None] 1.5762ms 1.2335ms 810.7274 Ops/s 789.1173 Ops/s $\color{#35bf28}+2.74\%$
test_ddpg_speed[True-backward] 2.4343ms 2.2557ms 443.3288 Ops/s 430.6729 Ops/s $\color{#35bf28}+2.94\%$
test_ddpg_speed[reduce-overhead-None] 1.5976ms 1.2395ms 806.7582 Ops/s 793.9170 Ops/s $\color{#35bf28}+1.62\%$
test_ddpg_speed[reduce-overhead-backward] 2.3895ms 2.2262ms 449.2051 Ops/s 441.1422 Ops/s $\color{#35bf28}+1.83\%$
test_sac_speed[False-None] 8.4639ms 7.3578ms 135.9094 Ops/s 131.0183 Ops/s $\color{#35bf28}+3.73\%$
test_sac_speed[False-backward] 11.0258ms 10.6503ms 93.8936 Ops/s 91.3696 Ops/s $\color{#35bf28}+2.76\%$
test_sac_speed[True-None] 2.3843ms 2.0181ms 495.5069 Ops/s 486.8608 Ops/s $\color{#35bf28}+1.78\%$
test_sac_speed[True-backward] 4.2102ms 4.0143ms 249.1083 Ops/s 223.4482 Ops/s $\textbf{\color{#35bf28}+11.48\%}$
test_sac_speed[reduce-overhead-None] 2.4355ms 2.0229ms 494.3330 Ops/s 490.1987 Ops/s $\color{#35bf28}+0.84\%$
test_sac_speed[reduce-overhead-backward] 4.3223ms 3.9378ms 253.9466 Ops/s 251.6204 Ops/s $\color{#35bf28}+0.92\%$
test_redq_speed[False-None] 14.9068ms 10.0956ms 99.0528 Ops/s 89.7118 Ops/s $\textbf{\color{#35bf28}+10.41\%}$
test_redq_speed[False-backward] 18.4331ms 17.5509ms 56.9770 Ops/s 56.0086 Ops/s $\color{#35bf28}+1.73\%$
test_redq_speed[True-None] 3.8728ms 3.5576ms 281.0888 Ops/s 284.2551 Ops/s $\color{#d91a1a}-1.11\%$
test_redq_speed[True-backward] 9.4141ms 8.8450ms 113.0577 Ops/s 116.8653 Ops/s $\color{#d91a1a}-3.26\%$
test_redq_speed[reduce-overhead-None] 4.0287ms 3.6398ms 274.7397 Ops/s 283.4297 Ops/s $\color{#d91a1a}-3.07\%$
test_redq_speed[reduce-overhead-backward] 9.3643ms 8.8399ms 113.1232 Ops/s 114.2577 Ops/s $\color{#d91a1a}-0.99\%$
test_redq_deprec_speed[False-None] 12.4904ms 10.5843ms 94.4794 Ops/s 92.2596 Ops/s $\color{#35bf28}+2.41\%$
test_redq_deprec_speed[False-backward] 16.1222ms 15.5803ms 64.1837 Ops/s 63.4127 Ops/s $\color{#35bf28}+1.22\%$
test_redq_deprec_speed[True-None] 3.6711ms 3.3237ms 300.8669 Ops/s 307.5005 Ops/s $\color{#d91a1a}-2.16\%$
test_redq_deprec_speed[True-backward] 7.8545ms 7.2545ms 137.8449 Ops/s 133.2159 Ops/s $\color{#35bf28}+3.47\%$
test_redq_deprec_speed[reduce-overhead-None] 3.6422ms 3.2663ms 306.1575 Ops/s 316.0162 Ops/s $\color{#d91a1a}-3.12\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.5554ms 7.1851ms 139.1767 Ops/s 147.5450 Ops/s $\textbf{\color{#d91a1a}-5.67\%}$
test_td3_speed[False-None] 30.7008ms 7.6199ms 131.2347 Ops/s 124.7537 Ops/s $\textbf{\color{#35bf28}+5.20\%}$
test_td3_speed[False-backward] 10.7849ms 10.3840ms 96.3024 Ops/s 94.5011 Ops/s $\color{#35bf28}+1.91\%$
test_td3_speed[True-None] 1.9397ms 1.8983ms 526.7972 Ops/s 521.6566 Ops/s $\color{#35bf28}+0.99\%$
test_td3_speed[True-backward] 4.0230ms 3.7378ms 267.5362 Ops/s 256.0803 Ops/s $\color{#35bf28}+4.47\%$
test_td3_speed[reduce-overhead-None] 1.9487ms 1.8873ms 529.8707 Ops/s 522.1445 Ops/s $\color{#35bf28}+1.48\%$
test_td3_speed[reduce-overhead-backward] 3.9421ms 3.7282ms 268.2232 Ops/s 264.4173 Ops/s $\color{#35bf28}+1.44\%$
test_cql_speed[False-None] 27.2918ms 24.9484ms 40.0827 Ops/s 40.2856 Ops/s $\color{#d91a1a}-0.50\%$
test_cql_speed[False-backward] 39.1594ms 35.2739ms 28.3496 Ops/s 28.6506 Ops/s $\color{#d91a1a}-1.05\%$
test_cql_speed[True-None] 11.4865ms 11.0588ms 90.4259 Ops/s 92.3091 Ops/s $\color{#d91a1a}-2.04\%$
test_cql_speed[True-backward] 17.6246ms 16.9796ms 58.8943 Ops/s 60.1495 Ops/s $\color{#d91a1a}-2.09\%$
test_cql_speed[reduce-overhead-None] 11.6325ms 11.0508ms 90.4913 Ops/s 91.6252 Ops/s $\color{#d91a1a}-1.24\%$
test_cql_speed[reduce-overhead-backward] 17.7556ms 17.1132ms 58.4343 Ops/s 59.6196 Ops/s $\color{#d91a1a}-1.99\%$
test_a2c_speed[False-None] 5.5056ms 5.3002ms 188.6705 Ops/s 186.5780 Ops/s $\color{#35bf28}+1.12\%$
test_a2c_speed[False-backward] 12.1747ms 11.8295ms 84.5345 Ops/s 82.4837 Ops/s $\color{#35bf28}+2.49\%$
test_a2c_speed[True-None] 3.2930ms 3.0746ms 325.2490 Ops/s 345.8707 Ops/s $\textbf{\color{#d91a1a}-5.96\%}$
test_a2c_speed[True-backward] 8.7783ms 8.4902ms 117.7822 Ops/s 115.0100 Ops/s $\color{#35bf28}+2.41\%$
test_a2c_speed[reduce-overhead-None] 3.3125ms 3.0414ms 328.7987 Ops/s 321.3293 Ops/s $\color{#35bf28}+2.32\%$
test_a2c_speed[reduce-overhead-backward] 8.8480ms 8.4948ms 117.7194 Ops/s 118.6987 Ops/s $\color{#d91a1a}-0.82\%$
test_ppo_speed[False-None] 7.5786ms 5.6980ms 175.5011 Ops/s 174.7004 Ops/s $\color{#35bf28}+0.46\%$
test_ppo_speed[False-backward] 13.0397ms 12.5538ms 79.6570 Ops/s 79.9368 Ops/s $\color{#d91a1a}-0.35\%$
test_ppo_speed[True-None] 3.8623ms 3.4318ms 291.3895 Ops/s 306.8928 Ops/s $\textbf{\color{#d91a1a}-5.05\%}$
test_ppo_speed[True-backward] 8.5814ms 8.2522ms 121.1805 Ops/s 117.1270 Ops/s $\color{#35bf28}+3.46\%$
test_ppo_speed[reduce-overhead-None] 3.5993ms 3.4420ms 290.5328 Ops/s 291.9531 Ops/s $\color{#d91a1a}-0.49\%$
test_ppo_speed[reduce-overhead-backward] 8.5933ms 8.3233ms 120.1442 Ops/s 122.5764 Ops/s $\color{#d91a1a}-1.98\%$
test_reinforce_speed[False-None] 6.4524ms 4.5421ms 220.1620 Ops/s 226.7985 Ops/s $\color{#d91a1a}-2.93\%$
test_reinforce_speed[False-backward] 7.6413ms 7.3680ms 135.7223 Ops/s 136.4870 Ops/s $\color{#d91a1a}-0.56\%$
test_reinforce_speed[True-None] 2.7125ms 2.2464ms 445.1473 Ops/s 433.2324 Ops/s $\color{#35bf28}+2.75\%$
test_reinforce_speed[True-backward] 7.4597ms 7.1508ms 139.8437 Ops/s 139.8723 Ops/s $\color{#d91a1a}-0.02\%$
test_reinforce_speed[reduce-overhead-None] 2.4936ms 2.2387ms 446.6915 Ops/s 515.6837 Ops/s $\textbf{\color{#d91a1a}-13.38\%}$
test_reinforce_speed[reduce-overhead-backward] 7.4716ms 7.1921ms 139.0416 Ops/s 146.1880 Ops/s $\color{#d91a1a}-4.89\%$
test_iql_speed[False-None] 20.4499ms 19.5477ms 51.1570 Ops/s 53.3367 Ops/s $\color{#d91a1a}-4.09\%$
test_iql_speed[False-backward] 31.0480ms 30.1855ms 33.1285 Ops/s 34.1052 Ops/s $\color{#d91a1a}-2.86\%$
test_iql_speed[True-None] 7.3111ms 6.7481ms 148.1909 Ops/s 165.4352 Ops/s $\textbf{\color{#d91a1a}-10.42\%}$
test_iql_speed[True-backward] 15.8065ms 15.3725ms 65.0513 Ops/s 63.0825 Ops/s $\color{#35bf28}+3.12\%$
test_iql_speed[reduce-overhead-None] 7.8858ms 6.8713ms 145.5319 Ops/s 147.3239 Ops/s $\color{#d91a1a}-1.22\%$
test_iql_speed[reduce-overhead-backward] 17.6508ms 15.5384ms 64.3565 Ops/s 64.7263 Ops/s $\color{#d91a1a}-0.57\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.9520ms 6.1379ms 162.9226 Ops/s 160.2510 Ops/s $\color{#35bf28}+1.67\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.5265ms 0.3145ms 3.1797 KOps/s 3.6018 KOps/s $\textbf{\color{#d91a1a}-11.72\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5427ms 0.2838ms 3.5233 KOps/s 3.7220 KOps/s $\textbf{\color{#d91a1a}-5.34\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.1606ms 5.8353ms 171.3698 Ops/s 167.5404 Ops/s $\color{#35bf28}+2.29\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.3417s 0.5317ms 1.8807 KOps/s 3.7095 KOps/s $\textbf{\color{#d91a1a}-49.30\%}$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6871ms 0.3022ms 3.3091 KOps/s 3.6577 KOps/s $\textbf{\color{#d91a1a}-9.53\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.4862ms 1.2452ms 803.0897 Ops/s 801.3135 Ops/s $\color{#35bf28}+0.22\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.3972ms 1.1879ms 841.8226 Ops/s 825.5898 Ops/s $\color{#35bf28}+1.97\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.2583ms 6.0621ms 164.9583 Ops/s 163.6369 Ops/s $\color{#35bf28}+0.81\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0791ms 0.4376ms 2.2851 KOps/s 2.3946 KOps/s $\color{#d91a1a}-4.57\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6806ms 0.4074ms 2.4544 KOps/s 2.5382 KOps/s $\color{#d91a1a}-3.30\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.1410ms 5.9331ms 168.5455 Ops/s 168.2168 Ops/s $\color{#35bf28}+0.20\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.0384ms 0.3215ms 3.1109 KOps/s 2.5913 KOps/s $\textbf{\color{#35bf28}+20.05\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5081ms 0.2810ms 3.5589 KOps/s 2.9796 KOps/s $\textbf{\color{#35bf28}+19.44\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.2096ms 5.8927ms 169.7013 Ops/s 168.0965 Ops/s $\color{#35bf28}+0.95\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.6582ms 0.2632ms 3.8001 KOps/s 3.5921 KOps/s $\textbf{\color{#35bf28}+5.79\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.6561ms 0.2626ms 3.8077 KOps/s 3.2330 KOps/s $\textbf{\color{#35bf28}+17.78\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 9.3455ms 6.3402ms 157.7229 Ops/s 165.8704 Ops/s $\color{#d91a1a}-4.91\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 2.0093ms 0.4386ms 2.2800 KOps/s 2.1127 KOps/s $\textbf{\color{#35bf28}+7.92\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.7607ms 0.4231ms 2.3635 KOps/s 2.0377 KOps/s $\textbf{\color{#35bf28}+15.99\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 0.4343s 13.8568ms 72.1668 Ops/s 192.7693 Ops/s $\textbf{\color{#d91a1a}-62.56\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.2452ms 2.0967ms 476.9429 Ops/s 477.7646 Ops/s $\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 8.0108ms 1.3125ms 761.9018 Ops/s 935.4632 Ops/s $\textbf{\color{#d91a1a}-18.55\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 7.5153ms 5.2657ms 189.9092 Ops/s 193.2180 Ops/s $\color{#d91a1a}-1.71\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.7319ms 2.0328ms 491.9348 Ops/s 461.7841 Ops/s $\textbf{\color{#35bf28}+6.53\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 7.0854ms 1.1982ms 834.5852 Ops/s 848.6998 Ops/s $\color{#d91a1a}-1.66\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 0.3783s 12.9055ms 77.4866 Ops/s 35.3866 Ops/s $\textbf{\color{#35bf28}+118.97\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 12.4780ms 2.0831ms 480.0471 Ops/s 535.2661 Ops/s $\textbf{\color{#d91a1a}-10.32\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.4841ms 1.2572ms 795.4460 Ops/s 690.8140 Ops/s $\textbf{\color{#35bf28}+15.15\%}$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-True] 13.1223ms 12.4718ms 80.1806 Ops/s 77.5453 Ops/s $\color{#35bf28}+3.40\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-10000-10000-100-False] 17.9272ms 16.5563ms 60.3999 Ops/s 59.2933 Ops/s $\color{#35bf28}+1.87\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-True] 17.8809ms 17.1400ms 58.3431 Ops/s 56.2092 Ops/s $\color{#35bf28}+3.80\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-100000-10000-100-False] 17.5711ms 16.8861ms 59.2203 Ops/s 57.8631 Ops/s $\color{#35bf28}+2.35\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-True] 19.4185ms 17.1965ms 58.1512 Ops/s 56.8877 Ops/s $\color{#35bf28}+2.22\%$
test_rb_extend_sample[ReplayBuffer-LazyTensorStorage-RandomSampler-1000000-10000-100-False] 18.7418ms 17.9935ms 55.5756 Ops/s 55.0295 Ops/s $\color{#35bf28}+0.99\%$

Update
97ab2e1 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
8ea8e96 
ghstack-source-id: d253f1b
Pull Request resolved: #2559
@vmoens vmoens added the documentation Improvements or additions to documentation label Nov 13, 2024
Update
06af9d2 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
13b0a14 
ghstack-source-id: 0c37d01
Pull Request resolved: #2559
Update
8e3bb88 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
ac5b2d7 
ghstack-source-id: 60865f2
Pull Request resolved: #2559
Update
1f20580 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
29d825e 
ghstack-source-id: 1f1af39
Pull Request resolved: #2559
@vmoens vmoens merged commit 1f20580 into gh/vmoens/42/base Nov 13, 2024
40 of 46 checks passed
vmoens pushed a commit that referenced this pull request Nov 13, 2024
[Doc] Adding recurrent policies to export tutorial
7051238 
ghstack-source-id: 1f1af39
Pull Request resolved: #2559
@vmoens vmoens deleted the gh/vmoens/42/head branch November 13, 2024 18:47
vmoens pushed a commit that referenced this pull request Nov 14, 2024
[Doc] Adding recurrent policies to export tutorial
de224e3 
ghstack-source-id: 1f1af39
Pull Request resolved: #2559

(cherry picked from commit 7051238)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. documentation Improvements or additions to documentation

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vmoens @facebook-github-bot