a solution that works with file chunks would work better here instead of loading the entire buffer into memory at one time, I tried a few different approaches with mmap, save having to write the files to the hard disk before loading I haven't been able to solve it using that approach(chunks + mmap)

Codecov Report

Merging #5394 into master will increase coverage by 2.15%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #5394      +/-   ##
=========================================
+ Coverage   29.85%     32%   +2.15%     
=========================================
  Files         241     242       +1     
  Lines       20406   23728    +3322     
  Branches     2918    3939    +1021     
=========================================
+ Hits         6092    7594    +1502     
- Misses      14063   15777    +1714     
- Partials      251     357     +106

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6b95c8f...1c309a1. Read the comment docs.

Oh this is sweet :D

I have updated the approach, to leverage chunks and to include a forbidden content array that can be populated/managed to ensure that content that would violate terms of service or otherwise cause harm can be scrubbed and denied, note this doesn't use mmap as the uploadedfiles are InMemory objects anyways, we can just handle the raw binary and iterate and search.