Don't store large binary files in git repo. Every time you make a commit every file in your repo is copyed. That's why it is 184 MiB now. Use Github Releases to store downloads and use git-lfs to store large files and blobs if they are strictly needed.