This PR contains unrelated commit after I pushed force it (I cannot change the base branch). The relevant commit is 63af7fb.

To packetize the H.264 raw stream, av_parser_parse2() (called by av_read_frame()) knows that it has received a full frame only after it has received some data for the next frame. As a consequence, the client always waited until the next frame before sending the current frame to the decoder!

On the device side, we know packets boundaries. To reduce latency, make the device always transmit the "frame meta" to packetize the stream manually (it was already implemented to send PTS, but only enabled on recording).

On the client side, replace av_read_frame() by manual packetizing and parsing.

https://stackoverflow.com/questions/50682518/replacing-av-read-frame-to-reduce-delay
https://trac.ffmpeg.org/ticket/3354

I'm opening a pull request to get more tests before merging (the changes are quite sensitive), to avoid regressions.

To confirm that it actually reduces the end-to-end latency by 1 frame, I created a video which increments a number at 30fps (every 33.3ms):

# requires ffmpeg and imagemagick
for i in {0..999}; do convert -size 400x400 xc:White -gravity Center -pointsize 200 -font DejaVu-Sans-Mono -annotate 0 $i $(printf '%03d' $i).png; done
ffmpeg -framerate 30 -pattern_type glob -i '*.png' -c:v libx264 -vf format=yuv420p 30fps.mp4

I run this video in VLC on Android, and start scrcpy.

I compared the current master with this PR for both 1080x1920 (just running scrcpy) and 448x800 (running scrcpy -m800).

1080x1920

on master

End-to-end latency is about 3 frames (~100ms).

on lowlatency

End-to-end latency is about 2 frames (~67ms).

Once a packet is received by the client, on average, the frame is rendered 12ms later.
(with "skip frames" disabled, i.e. --render-expired-frames)

448x800

on master

End-to-end latency is about 2 frames (~67ms).

on lowlatency

End-to-end latency is about 1 frame (~33ms).

Once a packet is received by the client, on average, the frame is rendered 8ms later.
(with "skip frames" disabled, i.e. -m800 --render-expired-frames)

For now, I preserved the compatibility with prebuilt server v1.9, so you can just:

git fetch origin && git checkout lowlatency

then build as usual.

For windows users, I built a binary (replace the one from v1.9): scrcpy.exe.
SHA-256: 6b8eae4eb4df811b6aa194320f00b51102cf3a254e432f353963863c82f362ea

Thank you for your feedbacks 😉