Skip to content

[BugFix] Fix old deps tests #2500

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 5 commits into from
Oct 21, 2024
Merged

[BugFix] Fix old deps tests #2500

merged 5 commits into from
Oct 21, 2024

Conversation

@vmoens
Copy link
Collaborator

@vmoens vmoens commented Oct 17, 2024 •
edited
Loading

Stack from ghstack (oldest at bottom):

Update
a0e08c8 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Oct 17, 2024
[BugFix] Fix old deps tests
81e0ef6 
ghstack-source-id: 4d16887
Pull Request resolved: #2500
@pytorch-bot
Copy link

pytorch-bot bot commented Oct 17, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/2500

Note: Links to docs will display an error until the docs builds have been completed.

❌ 7 New Failures, 4 Unrelated Failures

As of commit ad40abb with merge base 9f6c21f (image):

NEW FAILURES - The following jobs have failed:

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Oct 17, 2024
@vmoens vmoens added the bug Something isn't working label Oct 17, 2024
@github-actions
Copy link

github-actions bot commented Oct 17, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 143. Improved: $\large\color{#35bf28}5$. Worsened: $\large\color{#d91a1a}12$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.4235s 0.4170s 2.3983 Ops/s 2.3424 Ops/s $\color{#35bf28}+2.38\%$
test_transformed 0.6906s 0.6130s 1.6312 Ops/s 1.7285 Ops/s $\textbf{\color{#d91a1a}-5.63\%}$
test_serial 1.4534s 1.3582s 0.7363 Ops/s 0.7574 Ops/s $\color{#d91a1a}-2.79\%$
test_parallel 1.4361s 1.3303s 0.7517 Ops/s 0.7311 Ops/s $\color{#35bf28}+2.82\%$
test_step_mdp_speed[True-True-True-True-True] 0.2353ms 28.4808μs 35.1114 KOps/s 35.1195 KOps/s $\color{#d91a1a}-0.02\%$
test_step_mdp_speed[True-True-True-True-False] 57.1060μs 17.0382μs 58.6918 KOps/s 59.2701 KOps/s $\color{#d91a1a}-0.98\%$
test_step_mdp_speed[True-True-True-False-True] 46.4570μs 16.2470μs 61.5499 KOps/s 62.9972 KOps/s $\color{#d91a1a}-2.30\%$
test_step_mdp_speed[True-True-True-False-False] 36.5380μs 9.5798μs 104.3860 KOps/s 108.4929 KOps/s $\color{#d91a1a}-3.79\%$
test_step_mdp_speed[True-True-False-True-True] 68.3380μs 31.1328μs 32.1205 KOps/s 32.5196 KOps/s $\color{#d91a1a}-1.23\%$
test_step_mdp_speed[True-True-False-True-False] 52.3480μs 19.1923μs 52.1041 KOps/s 52.4238 KOps/s $\color{#d91a1a}-0.61\%$
test_step_mdp_speed[True-True-False-False-True] 50.3840μs 18.3241μs 54.5729 KOps/s 55.6611 KOps/s $\color{#d91a1a}-1.96\%$
test_step_mdp_speed[True-True-False-False-False] 40.2050μs 11.6846μs 85.5829 KOps/s 87.2046 KOps/s $\color{#d91a1a}-1.86\%$
test_step_mdp_speed[True-False-True-True-True] 79.6890μs 33.6686μs 29.7012 KOps/s 30.6004 KOps/s $\color{#d91a1a}-2.94\%$
test_step_mdp_speed[True-False-True-True-False] 53.3100μs 21.5311μs 46.4445 KOps/s 48.2047 KOps/s $\color{#d91a1a}-3.65\%$
test_step_mdp_speed[True-False-True-False-True] 61.8160μs 18.4784μs 54.1173 KOps/s 56.0267 KOps/s $\color{#d91a1a}-3.41\%$
test_step_mdp_speed[True-False-True-False-False] 46.6470μs 11.6765μs 85.6421 KOps/s 87.3431 KOps/s $\color{#d91a1a}-1.95\%$
test_step_mdp_speed[True-False-False-True-True] 73.3270μs 35.8438μs 27.8989 KOps/s 29.0674 KOps/s $\color{#d91a1a}-4.02\%$
test_step_mdp_speed[True-False-False-True-False] 60.0230μs 23.4075μs 42.7213 KOps/s 43.1568 KOps/s $\color{#d91a1a}-1.01\%$
test_step_mdp_speed[True-False-False-False-True] 48.2100μs 20.4518μs 48.8955 KOps/s 50.8715 KOps/s $\color{#d91a1a}-3.88\%$
test_step_mdp_speed[True-False-False-False-False] 40.4550μs 13.5387μs 73.8624 KOps/s 75.0804 KOps/s $\color{#d91a1a}-1.62\%$
test_step_mdp_speed[False-True-True-True-True] 68.5080μs 33.3419μs 29.9923 KOps/s 30.7804 KOps/s $\color{#d91a1a}-2.56\%$
test_step_mdp_speed[False-True-True-True-False] 55.4240μs 21.2442μs 47.0717 KOps/s 47.3452 KOps/s $\color{#d91a1a}-0.58\%$
test_step_mdp_speed[False-True-True-False-True] 63.4080μs 21.8460μs 45.7750 KOps/s 47.3450 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[False-True-True-False-False] 43.1910μs 13.3105μs 75.1287 KOps/s 77.7090 KOps/s $\color{#d91a1a}-3.32\%$
test_step_mdp_speed[False-True-False-True-True] 72.9360μs 35.3730μs 28.2701 KOps/s 29.2870 KOps/s $\color{#d91a1a}-3.47\%$
test_step_mdp_speed[False-True-False-True-False] 61.8850μs 23.4681μs 42.6111 KOps/s 44.4169 KOps/s $\color{#d91a1a}-4.07\%$
test_step_mdp_speed[False-True-False-False-True] 2.6293ms 23.8245μs 41.9735 KOps/s 43.5291 KOps/s $\color{#d91a1a}-3.57\%$
test_step_mdp_speed[False-True-False-False-False] 45.5550μs 15.3925μs 64.9668 KOps/s 67.3015 KOps/s $\color{#d91a1a}-3.47\%$
test_step_mdp_speed[False-False-True-True-True] 96.5780μs 37.3832μs 26.7500 KOps/s 27.2079 KOps/s $\color{#d91a1a}-1.68\%$
test_step_mdp_speed[False-False-True-True-False] 71.1730μs 25.8299μs 38.7148 KOps/s 40.1622 KOps/s $\color{#d91a1a}-3.60\%$
test_step_mdp_speed[False-False-True-False-True] 59.5510μs 24.0757μs 41.5356 KOps/s 44.0405 KOps/s $\textbf{\color{#d91a1a}-5.69\%}$
test_step_mdp_speed[False-False-True-False-False] 56.4150μs 15.3677μs 65.0715 KOps/s 67.0996 KOps/s $\color{#d91a1a}-3.02\%$
test_step_mdp_speed[False-False-False-True-True] 82.8950μs 38.8628μs 25.7315 KOps/s 26.1502 KOps/s $\color{#d91a1a}-1.60\%$
test_step_mdp_speed[False-False-False-True-False] 54.1210μs 27.3892μs 36.5108 KOps/s 37.3232 KOps/s $\color{#d91a1a}-2.18\%$
test_step_mdp_speed[False-False-False-False-True] 59.3700μs 25.3158μs 39.5011 KOps/s 41.0557 KOps/s $\color{#d91a1a}-3.79\%$
test_step_mdp_speed[False-False-False-False-False] 70.5320μs 16.8633μs 59.3003 KOps/s 59.1648 KOps/s $\color{#35bf28}+0.23\%$
test_values[generalized_advantage_estimate-True-True] 13.5525ms 9.6164ms 103.9888 Ops/s 105.0062 Ops/s $\color{#d91a1a}-0.97\%$
test_values[vec_generalized_advantage_estimate-True-True] 45.0669ms 35.7829ms 27.9463 Ops/s 28.1525 Ops/s $\color{#d91a1a}-0.73\%$
test_values[td0_return_estimate-False-False] 0.2445ms 0.1708ms 5.8548 KOps/s 5.6620 KOps/s $\color{#35bf28}+3.41\%$
test_values[td1_return_estimate-False-False] 27.8120ms 23.8938ms 41.8518 Ops/s 42.2957 Ops/s $\color{#d91a1a}-1.05\%$
test_values[vec_td1_return_estimate-False-False] 38.3054ms 35.8175ms 27.9193 Ops/s 27.8884 Ops/s $\color{#35bf28}+0.11\%$
test_values[td_lambda_return_estimate-True-False] 38.0536ms 34.3076ms 29.1481 Ops/s 29.0980 Ops/s $\color{#35bf28}+0.17\%$
test_values[vec_td_lambda_return_estimate-True-False] 37.7123ms 35.6950ms 28.0152 Ops/s 27.9729 Ops/s $\color{#35bf28}+0.15\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 8.5883ms 8.4432ms 118.4381 Ops/s 119.8155 Ops/s $\color{#d91a1a}-1.15\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 2.4429ms 1.9406ms 515.3125 Ops/s 502.4368 Ops/s $\color{#35bf28}+2.56\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.4489ms 0.3627ms 2.7574 KOps/s 2.7344 KOps/s $\color{#35bf28}+0.84\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 43.0964ms 41.1493ms 24.3018 Ops/s 21.2662 Ops/s $\textbf{\color{#35bf28}+14.27\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 3.7734ms 3.0520ms 327.6487 Ops/s 323.2949 Ops/s $\color{#35bf28}+1.35\%$
test_dqn_speed[False-None] 5.6024ms 1.3650ms 732.6160 Ops/s 729.8487 Ops/s $\color{#35bf28}+0.38\%$
test_dqn_speed[False-backward] 1.9568ms 1.8482ms 541.0739 Ops/s 538.5735 Ops/s $\color{#35bf28}+0.46\%$
test_dqn_speed[True-None] 0.7447ms 0.4688ms 2.1330 KOps/s 2.1555 KOps/s $\color{#d91a1a}-1.05\%$
test_dqn_speed[True-backward] 0.9569ms 0.8944ms 1.1180 KOps/s 1.1060 KOps/s $\color{#35bf28}+1.08\%$
test_dqn_speed[reduce-overhead-None] 0.6058ms 0.4763ms 2.0996 KOps/s 2.0582 KOps/s $\color{#35bf28}+2.01\%$
test_dqn_speed[reduce-overhead-backward] 1.0996ms 0.9216ms 1.0850 KOps/s 1.1074 KOps/s $\color{#d91a1a}-2.02\%$
test_ddpg_speed[False-None] 3.6052ms 2.8509ms 350.7708 Ops/s 342.4904 Ops/s $\color{#35bf28}+2.42\%$
test_ddpg_speed[False-backward] 4.0998ms 3.9850ms 250.9427 Ops/s 253.4670 Ops/s $\color{#d91a1a}-1.00\%$
test_ddpg_speed[True-None] 1.5632ms 1.0239ms 976.6165 Ops/s 973.2399 Ops/s $\color{#35bf28}+0.35\%$
test_ddpg_speed[True-backward] 1.9741ms 1.9096ms 523.6761 Ops/s 496.2783 Ops/s $\textbf{\color{#35bf28}+5.52\%}$
test_ddpg_speed[reduce-overhead-None] 1.7387ms 1.0118ms 988.3751 Ops/s 973.1946 Ops/s $\color{#35bf28}+1.56\%$
test_ddpg_speed[reduce-overhead-backward] 1.9528ms 1.9017ms 525.8536 Ops/s 516.6452 Ops/s $\color{#35bf28}+1.78\%$
test_sac_speed[False-None] 9.1212ms 8.0640ms 124.0087 Ops/s 123.9168 Ops/s $\color{#35bf28}+0.07\%$
test_sac_speed[False-backward] 11.1680ms 10.7178ms 93.3027 Ops/s 92.4988 Ops/s $\color{#35bf28}+0.87\%$
test_sac_speed[True-None] 2.1558ms 1.8607ms 537.4178 Ops/s 528.2138 Ops/s $\color{#35bf28}+1.74\%$
test_sac_speed[True-backward] 4.4693ms 3.5879ms 278.7110 Ops/s 272.3135 Ops/s $\color{#35bf28}+2.35\%$
test_sac_speed[reduce-overhead-None] 2.3135ms 1.8804ms 531.7909 Ops/s 529.8768 Ops/s $\color{#35bf28}+0.36\%$
test_sac_speed[reduce-overhead-backward] 4.5545ms 3.7154ms 269.1475 Ops/s 278.0338 Ops/s $\color{#d91a1a}-3.20\%$
test_redq_speed[False-None] 14.3109ms 13.0110ms 76.8583 Ops/s 76.5681 Ops/s $\color{#35bf28}+0.38\%$
test_redq_speed[False-backward] 23.8624ms 22.5552ms 44.3356 Ops/s 44.4358 Ops/s $\color{#d91a1a}-0.23\%$
test_redq_speed[True-None] 6.4459ms 4.8559ms 205.9349 Ops/s 199.1576 Ops/s $\color{#35bf28}+3.40\%$
test_redq_speed[True-backward] 13.4187ms 12.5879ms 79.4413 Ops/s 81.5342 Ops/s $\color{#d91a1a}-2.57\%$
test_redq_speed[reduce-overhead-None] 6.0163ms 4.8863ms 204.6537 Ops/s 204.4830 Ops/s $\color{#35bf28}+0.08\%$
test_redq_speed[reduce-overhead-backward] 13.3900ms 12.5339ms 79.7839 Ops/s 79.9961 Ops/s $\color{#d91a1a}-0.27\%$
test_redq_deprec_speed[False-None] 15.1326ms 13.0511ms 76.6219 Ops/s 76.7329 Ops/s $\color{#d91a1a}-0.14\%$
test_redq_deprec_speed[False-backward] 20.7204ms 18.9387ms 52.8018 Ops/s 52.9729 Ops/s $\color{#d91a1a}-0.32\%$
test_redq_deprec_speed[True-None] 5.2745ms 3.7475ms 266.8477 Ops/s 273.2259 Ops/s $\color{#d91a1a}-2.33\%$
test_redq_deprec_speed[True-backward] 9.8211ms 8.4466ms 118.3901 Ops/s 114.2860 Ops/s $\color{#35bf28}+3.59\%$
test_redq_deprec_speed[reduce-overhead-None] 4.1428ms 3.6675ms 272.6676 Ops/s 250.7437 Ops/s $\textbf{\color{#35bf28}+8.74\%}$
test_redq_deprec_speed[reduce-overhead-backward] 9.0146ms 8.4307ms 118.6139 Ops/s 117.3756 Ops/s $\color{#35bf28}+1.05\%$
test_td3_speed[False-None] 8.3849ms 8.0605ms 124.0611 Ops/s 124.9719 Ops/s $\color{#d91a1a}-0.73\%$
test_td3_speed[False-backward] 11.2274ms 10.5704ms 94.6042 Ops/s 95.6681 Ops/s $\color{#d91a1a}-1.11\%$
test_td3_speed[True-None] 1.8713ms 1.7719ms 564.3755 Ops/s 540.1798 Ops/s $\color{#35bf28}+4.48\%$
test_td3_speed[True-backward] 4.0155ms 3.4413ms 290.5881 Ops/s 224.2202 Ops/s $\textbf{\color{#35bf28}+29.60\%}$
test_td3_speed[reduce-overhead-None] 1.9712ms 1.7731ms 563.9749 Ops/s 563.4631 Ops/s $\color{#35bf28}+0.09\%$
test_td3_speed[reduce-overhead-backward] 7.7311ms 3.5720ms 279.9579 Ops/s 285.2878 Ops/s $\color{#d91a1a}-1.87\%$
test_cql_speed[False-None] 38.4260ms 35.9994ms 27.7783 Ops/s 27.2509 Ops/s $\color{#35bf28}+1.94\%$
test_cql_speed[False-backward] 0.3357s 51.8700ms 19.2790 Ops/s 21.4729 Ops/s $\textbf{\color{#d91a1a}-10.22\%}$
test_cql_speed[True-None] 17.5872ms 15.8192ms 63.2141 Ops/s 63.2291 Ops/s $\color{#d91a1a}-0.02\%$
test_cql_speed[True-backward] 23.7558ms 22.7718ms 43.9139 Ops/s 44.7767 Ops/s $\color{#d91a1a}-1.93\%$
test_cql_speed[reduce-overhead-None] 16.6046ms 15.9023ms 62.8839 Ops/s 63.2947 Ops/s $\color{#d91a1a}-0.65\%$
test_cql_speed[reduce-overhead-backward] 24.2400ms 22.6687ms 44.1137 Ops/s 44.4315 Ops/s $\color{#d91a1a}-0.72\%$
test_a2c_speed[False-None] 8.0495ms 7.2542ms 137.8511 Ops/s 139.2517 Ops/s $\color{#d91a1a}-1.01\%$
test_a2c_speed[False-backward] 15.1166ms 14.5848ms 68.5643 Ops/s 70.5094 Ops/s $\color{#d91a1a}-2.76\%$
test_a2c_speed[True-None] 3.6759ms 3.3247ms 300.7797 Ops/s 298.7333 Ops/s $\color{#35bf28}+0.69\%$
test_a2c_speed[True-backward] 10.6483ms 10.0170ms 99.8304 Ops/s 101.2806 Ops/s $\color{#d91a1a}-1.43\%$
test_a2c_speed[reduce-overhead-None] 3.6992ms 3.3387ms 299.5140 Ops/s 298.7195 Ops/s $\color{#35bf28}+0.27\%$
test_a2c_speed[reduce-overhead-backward] 10.5258ms 9.9433ms 100.5701 Ops/s 100.0822 Ops/s $\color{#35bf28}+0.49\%$
test_ppo_speed[False-None] 8.5009ms 7.5021ms 133.2966 Ops/s 134.6761 Ops/s $\color{#d91a1a}-1.02\%$
test_ppo_speed[False-backward] 15.3459ms 14.8732ms 67.2352 Ops/s 66.7960 Ops/s $\color{#35bf28}+0.66\%$
test_ppo_speed[True-None] 4.1077ms 3.7701ms 265.2440 Ops/s 265.2051 Ops/s $\color{#35bf28}+0.01\%$
test_ppo_speed[True-backward] 10.3255ms 9.8749ms 101.2665 Ops/s 101.9605 Ops/s $\color{#d91a1a}-0.68\%$
test_ppo_speed[reduce-overhead-None] 4.2678ms 3.7908ms 263.7962 Ops/s 265.7104 Ops/s $\color{#d91a1a}-0.72\%$
test_ppo_speed[reduce-overhead-backward] 11.5994ms 11.1848ms 89.4067 Ops/s 102.9698 Ops/s $\textbf{\color{#d91a1a}-13.17\%}$
test_reinforce_speed[False-None] 8.5688ms 6.5687ms 152.2368 Ops/s 153.7259 Ops/s $\color{#d91a1a}-0.97\%$
test_reinforce_speed[False-backward] 11.1334ms 10.2834ms 97.2438 Ops/s 101.8822 Ops/s $\color{#d91a1a}-4.55\%$
test_reinforce_speed[True-None] 3.1744ms 2.6698ms 374.5556 Ops/s 376.5699 Ops/s $\color{#d91a1a}-0.53\%$
test_reinforce_speed[True-backward] 10.8999ms 10.2719ms 97.3533 Ops/s 112.9283 Ops/s $\textbf{\color{#d91a1a}-13.79\%}$
test_reinforce_speed[reduce-overhead-None] 3.9196ms 3.1336ms 319.1237 Ops/s 362.6227 Ops/s $\textbf{\color{#d91a1a}-12.00\%}$
test_reinforce_speed[reduce-overhead-backward] 9.7319ms 9.0964ms 109.9339 Ops/s 112.4356 Ops/s $\color{#d91a1a}-2.22\%$
test_iql_speed[False-None] 34.5837ms 32.3769ms 30.8862 Ops/s 29.8826 Ops/s $\color{#35bf28}+3.36\%$
test_iql_speed[False-backward] 47.6038ms 45.6573ms 21.9023 Ops/s 21.6996 Ops/s $\color{#35bf28}+0.93\%$
test_iql_speed[True-None] 12.2553ms 11.0384ms 90.5927 Ops/s 90.9823 Ops/s $\color{#d91a1a}-0.43\%$
test_iql_speed[True-backward] 24.6777ms 22.9861ms 43.5045 Ops/s 44.6419 Ops/s $\color{#d91a1a}-2.55\%$
test_iql_speed[reduce-overhead-None] 12.2869ms 11.0532ms 90.4715 Ops/s 90.8459 Ops/s $\color{#d91a1a}-0.41\%$
test_iql_speed[reduce-overhead-backward] 23.3203ms 22.3026ms 44.8379 Ops/s 44.9848 Ops/s $\color{#d91a1a}-0.33\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.7591ms 5.2290ms 191.2412 Ops/s 207.0038 Ops/s $\textbf{\color{#d91a1a}-7.61\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.7462ms 0.4906ms 2.0384 KOps/s 2.0819 KOps/s $\color{#d91a1a}-2.09\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7365ms 0.4778ms 2.0931 KOps/s 2.1831 KOps/s $\color{#d91a1a}-4.12\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 7.9352ms 5.0152ms 199.3921 Ops/s 205.8911 Ops/s $\color{#d91a1a}-3.16\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.9602ms 0.4987ms 2.0052 KOps/s 2.0897 KOps/s $\color{#d91a1a}-4.04\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7612ms 0.4707ms 2.1245 KOps/s 2.2095 KOps/s $\color{#d91a1a}-3.85\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 2.4955ms 1.6205ms 617.0810 Ops/s 625.1799 Ops/s $\color{#d91a1a}-1.30\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 2.0770ms 1.5610ms 640.6089 Ops/s 641.7968 Ops/s $\color{#d91a1a}-0.19\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 5.4250ms 5.1140ms 195.5418 Ops/s 202.2808 Ops/s $\color{#d91a1a}-3.33\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.0776ms 0.6313ms 1.5840 KOps/s 1.5967 KOps/s $\color{#d91a1a}-0.80\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.9391ms 0.6112ms 1.6360 KOps/s 1.6462 KOps/s $\color{#d91a1a}-0.62\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 7.8381ms 5.0870ms 196.5790 Ops/s 211.4425 Ops/s $\textbf{\color{#d91a1a}-7.03\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 1.2794ms 0.4904ms 2.0391 KOps/s 2.0534 KOps/s $\color{#d91a1a}-0.70\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.7476ms 0.4867ms 2.0545 KOps/s 2.1642 KOps/s $\textbf{\color{#d91a1a}-5.07\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.0448ms 5.3902ms 185.5226 Ops/s 196.5179 Ops/s $\textbf{\color{#d91a1a}-5.60\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 2.6051ms 0.4862ms 2.0568 KOps/s 2.0339 KOps/s $\color{#35bf28}+1.13\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.7628ms 0.4734ms 2.1124 KOps/s 2.1862 KOps/s $\color{#d91a1a}-3.37\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 7.8386ms 5.1827ms 192.9494 Ops/s 200.0176 Ops/s $\color{#d91a1a}-3.53\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 0.9936ms 0.6360ms 1.5723 KOps/s 1.6211 KOps/s $\color{#d91a1a}-3.01\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 8.4106ms 0.6221ms 1.6074 KOps/s 1.6663 KOps/s $\color{#d91a1a}-3.54\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 5.6929ms 4.3212ms 231.4197 Ops/s 238.0520 Ops/s $\color{#d91a1a}-2.79\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 7.2136ms 2.4133ms 414.3665 Ops/s 419.9735 Ops/s $\color{#d91a1a}-1.34\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 5.9906ms 1.3825ms 723.3298 Ops/s 792.5134 Ops/s $\textbf{\color{#d91a1a}-8.73\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4632s 13.6572ms 73.2216 Ops/s 227.6215 Ops/s $\textbf{\color{#d91a1a}-67.83\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 8.9190ms 2.3342ms 428.4194 Ops/s 445.1047 Ops/s $\color{#d91a1a}-3.75\%$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 4.0927ms 1.2472ms 801.8119 Ops/s 767.9582 Ops/s $\color{#35bf28}+4.41\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 5.7209ms 4.4156ms 226.4683 Ops/s 231.1980 Ops/s $\color{#d91a1a}-2.05\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 8.8223ms 2.4377ms 410.2204 Ops/s 421.3733 Ops/s $\color{#d91a1a}-2.65\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 2.7535ms 1.3692ms 730.3526 Ops/s 691.8802 Ops/s $\textbf{\color{#35bf28}+5.56\%}$

@github-actions
Copy link

github-actions bot commented Oct 17, 2024 •
edited
Loading

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 143. Improved: $\large\color{#35bf28}16$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results
Name Max Mean Ops Ops on Repo HEAD Change
test_simple 0.7350s 0.7259s 1.3776 Ops/s 1.3761 Ops/s $\color{#35bf28}+0.11\%$
test_transformed 1.0556s 0.9793s 1.0211 Ops/s 1.0300 Ops/s $\color{#d91a1a}-0.86\%$
test_serial 2.1802s 2.1150s 0.4728 Ops/s 0.4708 Ops/s $\color{#35bf28}+0.43\%$
test_parallel 2.1139s 2.0005s 0.4999 Ops/s 0.5085 Ops/s $\color{#d91a1a}-1.69\%$
test_step_mdp_speed[True-True-True-True-True] 0.1753ms 38.5231μs 25.9585 KOps/s 25.4333 KOps/s $\color{#35bf28}+2.06\%$
test_step_mdp_speed[True-True-True-True-False] 0.2064ms 22.7975μs 43.8645 KOps/s 43.3878 KOps/s $\color{#35bf28}+1.10\%$
test_step_mdp_speed[True-True-True-False-True] 65.7410μs 20.7972μs 48.0833 KOps/s 47.4898 KOps/s $\color{#35bf28}+1.25\%$
test_step_mdp_speed[True-True-True-False-False] 0.1379ms 11.9536μs 83.6568 KOps/s 80.6402 KOps/s $\color{#35bf28}+3.74\%$
test_step_mdp_speed[True-True-False-True-True] 76.4120μs 41.6143μs 24.0302 KOps/s 24.0074 KOps/s $\color{#35bf28}+0.10\%$
test_step_mdp_speed[True-True-False-True-False] 0.2190ms 25.8957μs 38.6165 KOps/s 39.1585 KOps/s $\color{#d91a1a}-1.38\%$
test_step_mdp_speed[True-True-False-False-True] 81.4810μs 23.9365μs 41.7772 KOps/s 41.7768 KOps/s $+0.00\%$
test_step_mdp_speed[True-True-False-False-False] 0.1235ms 15.0340μs 66.5158 KOps/s 65.4740 KOps/s $\color{#35bf28}+1.59\%$
test_step_mdp_speed[True-False-True-True-True] 81.3320μs 44.1056μs 22.6728 KOps/s 22.4560 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-True-True-False] 80.3310μs 28.1093μs 35.5754 KOps/s 35.5563 KOps/s $\color{#35bf28}+0.05\%$
test_step_mdp_speed[True-False-True-False-True] 63.3910μs 23.7671μs 42.0749 KOps/s 41.9480 KOps/s $\color{#35bf28}+0.30\%$
test_step_mdp_speed[True-False-True-False-False] 44.1200μs 14.9868μs 66.7254 KOps/s 66.7878 KOps/s $\color{#d91a1a}-0.09\%$
test_step_mdp_speed[True-False-False-True-True] 0.1239ms 46.2711μs 21.6118 KOps/s 21.4038 KOps/s $\color{#35bf28}+0.97\%$
test_step_mdp_speed[True-False-False-True-False] 0.1878ms 29.7050μs 33.6644 KOps/s 32.2511 KOps/s $\color{#35bf28}+4.38\%$
test_step_mdp_speed[True-False-False-False-True] 61.5610μs 26.0996μs 38.3148 KOps/s 37.4637 KOps/s $\color{#35bf28}+2.27\%$
test_step_mdp_speed[True-False-False-False-False] 57.4710μs 17.3633μs 57.5926 KOps/s 56.3692 KOps/s $\color{#35bf28}+2.17\%$
test_step_mdp_speed[False-True-True-True-True] 0.1533ms 43.4503μs 23.0148 KOps/s 22.6641 KOps/s $\color{#35bf28}+1.55\%$
test_step_mdp_speed[False-True-True-True-False] 92.1810μs 27.8836μs 35.8634 KOps/s 35.4811 KOps/s $\color{#35bf28}+1.08\%$
test_step_mdp_speed[False-True-True-False-True] 67.1120μs 28.2076μs 35.4515 KOps/s 35.7899 KOps/s $\color{#d91a1a}-0.95\%$
test_step_mdp_speed[False-True-True-False-False] 55.6810μs 17.5814μs 56.8782 KOps/s 57.1056 KOps/s $\color{#d91a1a}-0.40\%$
test_step_mdp_speed[False-True-False-True-True] 0.1370ms 46.6732μs 21.4256 KOps/s 21.4717 KOps/s $\color{#d91a1a}-0.21\%$
test_step_mdp_speed[False-True-False-True-False] 69.5320μs 30.7818μs 32.4867 KOps/s 32.3924 KOps/s $\color{#35bf28}+0.29\%$
test_step_mdp_speed[False-True-False-False-True] 3.3285ms 31.4953μs 31.7508 KOps/s 32.6438 KOps/s $\color{#d91a1a}-2.74\%$
test_step_mdp_speed[False-True-False-False-False] 66.5320μs 20.6087μs 48.5232 KOps/s 50.2511 KOps/s $\color{#d91a1a}-3.44\%$
test_step_mdp_speed[False-False-True-True-True] 0.1456ms 49.8781μs 20.0489 KOps/s 20.2546 KOps/s $\color{#d91a1a}-1.02\%$
test_step_mdp_speed[False-False-True-True-False] 70.0720μs 33.7408μs 29.6377 KOps/s 29.6760 KOps/s $\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-False-True-False-True] 69.2620μs 31.4316μs 31.8151 KOps/s 32.8253 KOps/s $\color{#d91a1a}-3.08\%$
test_step_mdp_speed[False-False-True-False-False] 50.9010μs 19.9956μs 50.0110 KOps/s 50.1717 KOps/s $\color{#d91a1a}-0.32\%$
test_step_mdp_speed[False-False-False-True-True] 90.3620μs 51.9635μs 19.2443 KOps/s 19.4120 KOps/s $\color{#d91a1a}-0.86\%$
test_step_mdp_speed[False-False-False-True-False] 76.9810μs 36.0851μs 27.7122 KOps/s 27.7422 KOps/s $\color{#d91a1a}-0.11\%$
test_step_mdp_speed[False-False-False-False-True] 77.9820μs 33.3301μs 30.0029 KOps/s 30.0873 KOps/s $\color{#d91a1a}-0.28\%$
test_step_mdp_speed[False-False-False-False-False] 66.1410μs 22.6649μs 44.1211 KOps/s 44.1921 KOps/s $\color{#d91a1a}-0.16\%$
test_values[generalized_advantage_estimate-True-True] 23.8835ms 23.3658ms 42.7976 Ops/s 42.6646 Ops/s $\color{#35bf28}+0.31\%$
test_values[vec_generalized_advantage_estimate-True-True] 99.7457ms 2.8684ms 348.6319 Ops/s 317.3227 Ops/s $\textbf{\color{#35bf28}+9.87\%}$
test_values[td0_return_estimate-False-False] 0.2087ms 67.0018μs 14.9250 KOps/s 15.5830 KOps/s $\color{#d91a1a}-4.22\%$
test_values[td1_return_estimate-False-False] 52.4674ms 52.0914ms 19.1970 Ops/s 19.1744 Ops/s $\color{#35bf28}+0.12\%$
test_values[vec_td1_return_estimate-False-False] 1.3367ms 1.0532ms 949.4808 Ops/s 950.5757 Ops/s $\color{#d91a1a}-0.12\%$
test_values[td_lambda_return_estimate-True-False] 83.6279ms 83.0768ms 12.0371 Ops/s 12.0593 Ops/s $\color{#d91a1a}-0.18\%$
test_values[vec_td_lambda_return_estimate-True-False] 1.3731ms 1.0515ms 951.0006 Ops/s 954.9630 Ops/s $\color{#d91a1a}-0.41\%$
test_gae_speed[generalized_advantage_estimate-False-1-512] 23.4700ms 23.1143ms 43.2633 Ops/s 43.3581 Ops/s $\color{#d91a1a}-0.22\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512] 1.0380ms 0.7220ms 1.3851 KOps/s 1.3900 KOps/s $\color{#d91a1a}-0.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512] 0.7776ms 0.6371ms 1.5696 KOps/s 1.5649 KOps/s $\color{#35bf28}+0.30\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512] 1.6210ms 1.4524ms 688.5099 Ops/s 689.5763 Ops/s $\color{#d91a1a}-0.15\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512] 0.8500ms 0.6527ms 1.5322 KOps/s 1.5318 KOps/s $\color{#35bf28}+0.02\%$
test_dqn_speed[False-None] 6.9401ms 1.3123ms 762.0494 Ops/s 754.4455 Ops/s $\color{#35bf28}+1.01\%$
test_dqn_speed[False-backward] 2.1011ms 1.8263ms 547.5575 Ops/s 550.2076 Ops/s $\color{#d91a1a}-0.48\%$
test_dqn_speed[True-None] 0.9258ms 0.5539ms 1.8055 KOps/s 1.7860 KOps/s $\color{#35bf28}+1.09\%$
test_dqn_speed[True-backward] 1.1150ms 1.0019ms 998.1371 Ops/s 972.7270 Ops/s $\color{#35bf28}+2.61\%$
test_dqn_speed[reduce-overhead-None] 0.9375ms 0.5622ms 1.7787 KOps/s 1.7835 KOps/s $\color{#d91a1a}-0.27\%$
test_dqn_speed[reduce-overhead-backward] 1.0445ms 1.0020ms 998.0064 Ops/s 996.8344 Ops/s $\color{#35bf28}+0.12\%$
test_ddpg_speed[False-None] 3.1076ms 2.6958ms 370.9433 Ops/s 371.9960 Ops/s $\color{#d91a1a}-0.28\%$
test_ddpg_speed[False-backward] 4.0726ms 3.9045ms 256.1180 Ops/s 255.7204 Ops/s $\color{#35bf28}+0.16\%$
test_ddpg_speed[True-None] 1.4384ms 1.2419ms 805.2316 Ops/s 801.8335 Ops/s $\color{#35bf28}+0.42\%$
test_ddpg_speed[True-backward] 2.4646ms 2.2292ms 448.5977 Ops/s 451.4712 Ops/s $\color{#d91a1a}-0.64\%$
test_ddpg_speed[reduce-overhead-None] 1.5642ms 1.2413ms 805.6095 Ops/s 793.7445 Ops/s $\color{#35bf28}+1.49\%$
test_ddpg_speed[reduce-overhead-backward] 2.3837ms 2.2326ms 447.9015 Ops/s 449.1755 Ops/s $\color{#d91a1a}-0.28\%$
test_sac_speed[False-None] 8.5488ms 7.5177ms 133.0187 Ops/s 130.6271 Ops/s $\color{#35bf28}+1.83\%$
test_sac_speed[False-backward] 11.2306ms 10.6541ms 93.8603 Ops/s 92.7958 Ops/s $\color{#35bf28}+1.15\%$
test_sac_speed[True-None] 2.3500ms 2.0550ms 486.6275 Ops/s 478.9452 Ops/s $\color{#35bf28}+1.60\%$
test_sac_speed[True-backward] 4.1362ms 4.0059ms 249.6343 Ops/s 217.6340 Ops/s $\textbf{\color{#35bf28}+14.70\%}$
test_sac_speed[reduce-overhead-None] 2.3390ms 2.0352ms 491.3467 Ops/s 485.0874 Ops/s $\color{#35bf28}+1.29\%$
test_sac_speed[reduce-overhead-backward] 4.3816ms 4.0248ms 248.4590 Ops/s 250.5699 Ops/s $\color{#d91a1a}-0.84\%$
test_redq_speed[False-None] 14.3082ms 10.2048ms 97.9928 Ops/s 90.4120 Ops/s $\textbf{\color{#35bf28}+8.38\%}$
test_redq_speed[False-backward] 23.0191ms 17.8312ms 56.0815 Ops/s 55.5923 Ops/s $\color{#35bf28}+0.88\%$
test_redq_speed[True-None] 4.0103ms 3.6523ms 273.7973 Ops/s 271.5631 Ops/s $\color{#35bf28}+0.82\%$
test_redq_speed[True-backward] 9.1798ms 8.7521ms 114.2580 Ops/s 106.8660 Ops/s $\textbf{\color{#35bf28}+6.92\%}$
test_redq_speed[reduce-overhead-None] 3.9376ms 3.5593ms 280.9555 Ops/s 282.0900 Ops/s $\color{#d91a1a}-0.40\%$
test_redq_speed[reduce-overhead-backward] 9.1722ms 8.7578ms 114.1836 Ops/s 115.1984 Ops/s $\color{#d91a1a}-0.88\%$
test_redq_deprec_speed[False-None] 11.0923ms 10.6069ms 94.2781 Ops/s 94.7033 Ops/s $\color{#d91a1a}-0.45\%$
test_redq_deprec_speed[False-backward] 16.0561ms 15.4480ms 64.7333 Ops/s 65.8853 Ops/s $\color{#d91a1a}-1.75\%$
test_redq_deprec_speed[True-None] 3.5214ms 3.2855ms 304.3698 Ops/s 307.1782 Ops/s $\color{#d91a1a}-0.91\%$
test_redq_deprec_speed[True-backward] 7.6087ms 7.2719ms 137.5159 Ops/s 144.1580 Ops/s $\color{#d91a1a}-4.61\%$
test_redq_deprec_speed[reduce-overhead-None] 3.5836ms 3.2618ms 306.5748 Ops/s 311.2977 Ops/s $\color{#d91a1a}-1.52\%$
test_redq_deprec_speed[reduce-overhead-backward] 7.4854ms 7.2266ms 138.3780 Ops/s 144.0667 Ops/s $\color{#d91a1a}-3.95\%$
test_td3_speed[False-None] 7.6395ms 7.4812ms 133.6691 Ops/s 132.7315 Ops/s $\color{#35bf28}+0.71\%$
test_td3_speed[False-backward] 10.5808ms 10.2284ms 97.7666 Ops/s 96.6761 Ops/s $\color{#35bf28}+1.13\%$
test_td3_speed[True-None] 2.0966ms 1.9449ms 514.1637 Ops/s 516.2900 Ops/s $\color{#d91a1a}-0.41\%$
test_td3_speed[True-backward] 3.9367ms 3.7739ms 264.9809 Ops/s 266.3980 Ops/s $\color{#d91a1a}-0.53\%$
test_td3_speed[reduce-overhead-None] 1.9935ms 1.9225ms 520.1585 Ops/s 519.5411 Ops/s $\color{#35bf28}+0.12\%$
test_td3_speed[reduce-overhead-backward] 3.9641ms 3.7796ms 264.5810 Ops/s 260.0931 Ops/s $\color{#35bf28}+1.73\%$
test_cql_speed[False-None] 28.6428ms 25.4429ms 39.3037 Ops/s 39.3985 Ops/s $\color{#d91a1a}-0.24\%$
test_cql_speed[False-backward] 39.2271ms 35.2430ms 28.3744 Ops/s 28.7292 Ops/s $\color{#d91a1a}-1.23\%$
test_cql_speed[True-None] 11.7042ms 11.1878ms 89.3831 Ops/s 90.5207 Ops/s $\color{#d91a1a}-1.26\%$
test_cql_speed[True-backward] 17.8804ms 17.1246ms 58.3956 Ops/s 58.9575 Ops/s $\color{#d91a1a}-0.95\%$
test_cql_speed[reduce-overhead-None] 11.5300ms 11.1332ms 89.8217 Ops/s 90.1729 Ops/s $\color{#d91a1a}-0.39\%$
test_cql_speed[reduce-overhead-backward] 17.7268ms 17.0305ms 58.7183 Ops/s 59.3611 Ops/s $\color{#d91a1a}-1.08\%$
test_a2c_speed[False-None] 5.5201ms 5.2560ms 190.2583 Ops/s 185.3446 Ops/s $\color{#35bf28}+2.65\%$
test_a2c_speed[False-backward] 13.5081ms 11.7924ms 84.8007 Ops/s 84.2036 Ops/s $\color{#35bf28}+0.71\%$
test_a2c_speed[True-None] 3.4291ms 3.0558ms 327.2418 Ops/s 318.9335 Ops/s $\color{#35bf28}+2.61\%$
test_a2c_speed[True-backward] 9.3892ms 8.8719ms 112.7154 Ops/s 113.8660 Ops/s $\color{#d91a1a}-1.01\%$
test_a2c_speed[reduce-overhead-None] 3.3472ms 3.1165ms 320.8725 Ops/s 323.3592 Ops/s $\color{#d91a1a}-0.77\%$
test_a2c_speed[reduce-overhead-backward] 8.9372ms 8.6171ms 116.0477 Ops/s 115.7620 Ops/s $\color{#35bf28}+0.25\%$
test_ppo_speed[False-None] 7.5839ms 5.7180ms 174.8864 Ops/s 174.8215 Ops/s $\color{#35bf28}+0.04\%$
test_ppo_speed[False-backward] 12.6484ms 12.3737ms 80.8164 Ops/s 80.5779 Ops/s $\color{#35bf28}+0.30\%$
test_ppo_speed[True-None] 3.8127ms 3.4805ms 287.3124 Ops/s 279.8715 Ops/s $\color{#35bf28}+2.66\%$
test_ppo_speed[True-backward] 8.8660ms 8.4817ms 117.9014 Ops/s 109.4250 Ops/s $\textbf{\color{#35bf28}+7.75\%}$
test_ppo_speed[reduce-overhead-None] 3.8173ms 3.5033ms 285.4470 Ops/s 279.0611 Ops/s $\color{#35bf28}+2.29\%$
test_ppo_speed[reduce-overhead-backward] 8.8304ms 8.4274ms 118.6599 Ops/s 117.8737 Ops/s $\color{#35bf28}+0.67\%$
test_reinforce_speed[False-None] 5.0199ms 4.4445ms 224.9962 Ops/s 215.6515 Ops/s $\color{#35bf28}+4.33\%$
test_reinforce_speed[False-backward] 7.6674ms 7.3757ms 135.5808 Ops/s 130.2460 Ops/s $\color{#35bf28}+4.10\%$
test_reinforce_speed[True-None] 2.6610ms 2.2479ms 444.8577 Ops/s 431.3634 Ops/s $\color{#35bf28}+3.13\%$
test_reinforce_speed[True-backward] 7.7110ms 7.2731ms 137.4930 Ops/s 129.4419 Ops/s $\textbf{\color{#35bf28}+6.22\%}$
test_reinforce_speed[reduce-overhead-None] 2.6053ms 2.2625ms 441.9959 Ops/s 439.5058 Ops/s $\color{#35bf28}+0.57\%$
test_reinforce_speed[reduce-overhead-backward] 7.5289ms 7.2276ms 138.3593 Ops/s 130.6128 Ops/s $\textbf{\color{#35bf28}+5.93\%}$
test_iql_speed[False-None] 25.3414ms 19.8626ms 50.3458 Ops/s 50.4720 Ops/s $\color{#d91a1a}-0.25\%$
test_iql_speed[False-backward] 36.1014ms 30.6185ms 32.6600 Ops/s 33.0310 Ops/s $\color{#d91a1a}-1.12\%$
test_iql_speed[True-None] 7.3612ms 6.8803ms 145.3432 Ops/s 143.6867 Ops/s $\color{#35bf28}+1.15\%$
test_iql_speed[True-backward] 16.4742ms 15.8309ms 63.1676 Ops/s 61.9638 Ops/s $\color{#35bf28}+1.94\%$
test_iql_speed[reduce-overhead-None] 7.3270ms 6.9299ms 144.3024 Ops/s 145.0611 Ops/s $\color{#d91a1a}-0.52\%$
test_iql_speed[reduce-overhead-backward] 16.4205ms 15.8164ms 63.2255 Ops/s 62.6263 Ops/s $\color{#35bf28}+0.96\%$
test_rb_sample[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.6559ms 6.3942ms 156.3928 Ops/s 157.6340 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.8572ms 0.3448ms 2.9002 KOps/s 2.8546 KOps/s $\color{#35bf28}+1.59\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5763ms 0.2697ms 3.7078 KOps/s 3.0340 KOps/s $\textbf{\color{#35bf28}+22.21\%}$
test_rb_sample[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 6.6284ms 6.1706ms 162.0578 Ops/s 163.3505 Ops/s $\color{#d91a1a}-0.79\%$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.9379ms 0.3325ms 3.0073 KOps/s 2.9303 KOps/s $\color{#35bf28}+2.63\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 0.5622ms 0.2768ms 3.6129 KOps/s 3.0875 KOps/s $\textbf{\color{#35bf28}+17.02\%}$
test_rb_sample[TensorDictReplayBuffer-LazyMemmapStorage-sampler6-10000] 1.7939ms 1.3573ms 736.7588 Ops/s 734.9587 Ops/s $\color{#35bf28}+0.24\%$
test_rb_sample[TensorDictReplayBuffer-LazyTensorStorage-sampler7-10000] 1.6432ms 1.3065ms 765.3895 Ops/s 761.8741 Ops/s $\color{#35bf28}+0.46\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.6119ms 6.3376ms 157.7878 Ops/s 159.4839 Ops/s $\color{#d91a1a}-1.06\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.1994ms 0.4636ms 2.1572 KOps/s 2.1098 KOps/s $\color{#35bf28}+2.25\%$
test_rb_sample[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6925ms 0.4442ms 2.2514 KOps/s 2.1981 KOps/s $\color{#35bf28}+2.42\%$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-RandomSampler-4000] 6.5032ms 6.1813ms 161.7772 Ops/s 163.5545 Ops/s $\color{#d91a1a}-1.09\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000] 0.9244ms 0.3432ms 2.9133 KOps/s 3.8549 KOps/s $\textbf{\color{#d91a1a}-24.43\%}$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000] 0.5489ms 0.3188ms 3.1371 KOps/s 4.5465 KOps/s $\textbf{\color{#d91a1a}-31.00\%}$
test_rb_iterate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000] 9.6622ms 6.1108ms 163.6450 Ops/s 161.3823 Ops/s $\color{#35bf28}+1.40\%$
test_rb_iterate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000] 0.4891ms 0.3226ms 3.0997 KOps/s 2.9904 KOps/s $\color{#35bf28}+3.65\%$
test_rb_iterate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000] 7.1186ms 0.3042ms 3.2872 KOps/s 3.0652 KOps/s $\textbf{\color{#35bf28}+7.24\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000] 6.4861ms 6.2962ms 158.8256 Ops/s 156.8320 Ops/s $\color{#35bf28}+1.27\%$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000] 1.9254ms 0.4405ms 2.2702 KOps/s 2.0982 KOps/s $\textbf{\color{#35bf28}+8.20\%}$
test_rb_iterate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000] 0.6510ms 0.4350ms 2.2991 KOps/s 2.1737 KOps/s $\textbf{\color{#35bf28}+5.77\%}$
test_rb_populate[TensorDictReplayBuffer-ListStorage-RandomSampler-400] 6.7493ms 5.2009ms 192.2755 Ops/s 188.4025 Ops/s $\color{#35bf28}+2.06\%$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400] 9.9292ms 2.0315ms 492.2440 Ops/s 437.4243 Ops/s $\textbf{\color{#35bf28}+12.53\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400] 6.9205ms 1.1885ms 841.3875 Ops/s 883.8153 Ops/s $\color{#d91a1a}-4.80\%$
test_rb_populate[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400] 0.4359s 13.8834ms 72.0286 Ops/s 186.0771 Ops/s $\textbf{\color{#d91a1a}-61.29\%}$
test_rb_populate[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400] 10.3976ms 2.0579ms 485.9353 Ops/s 451.1415 Ops/s $\textbf{\color{#35bf28}+7.71\%}$
test_rb_populate[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400] 3.3185ms 1.1064ms 903.8731 Ops/s 788.6680 Ops/s $\textbf{\color{#35bf28}+14.61\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-ListStorage-None-400] 9.4917ms 5.4886ms 182.1947 Ops/s 182.5025 Ops/s $\color{#d91a1a}-0.17\%$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400] 7.7376ms 2.1797ms 458.7709 Ops/s 415.0748 Ops/s $\textbf{\color{#35bf28}+10.53\%}$
test_rb_populate[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400] 9.1916ms 1.3829ms 723.1101 Ops/s 703.8230 Ops/s $\color{#35bf28}+2.74\%$

Update
4b688cb 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Oct 18, 2024
[BugFix] Fix old deps tests
2a9d456 
ghstack-source-id: a8265bb
Pull Request resolved: #2500
Update
d0d1e37 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Oct 21, 2024
[BugFix] Fix old deps tests
c2d7a06 
ghstack-source-id: bf175d8
Pull Request resolved: #2500
Update
02f5bfc 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Oct 21, 2024
[BugFix] Fix old deps tests
4fd7408 
ghstack-source-id: b1f4902
Pull Request resolved: #2500
Update
ad40abb 
[ghstack-poisoned]
vmoens pushed a commit that referenced this pull request Oct 21, 2024
[BugFix] Fix old deps tests
2aad624 
ghstack-source-id: 134d129
Pull Request resolved: #2500
@vmoens vmoens merged commit ad40abb into gh/vmoens/33/base Oct 21, 2024
53 of 60 checks passed
@vmoens vmoens deleted the gh/vmoens/33/head branch October 21, 2024 12:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

bug Something isn't working CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@vmoens @facebook-github-bot