Skip to content

Conversation

@candrewlee14
Copy link
Contributor

@candrewlee14 candrewlee14 commented Jun 14, 2023

image
An important missing feature is a confidence interval on the delta, since insignificant run-to-run changes might wrongly show up as meaningful.

This PR adds that. Notice the cache_misses - 3.3% (which would've been considered significant before) is no longer significant since the standard deviation was high.

Related to #7 and #2

Resources:

@candrewlee14 candrewlee14 force-pushed the confidence-interval branch from ee8bccf to 2a5af42 Compare June 14, 2023 23:06
@candrewlee14 candrewlee14 force-pushed the confidence-interval branch from 4618d90 to 578baa9 Compare June 14, 2023 23:13
@andrewrk
Copy link
Owner

Thanks!

@andrewrk andrewrk merged commit d759d10 into andrewrk:main Jun 15, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants