webrtcsink: Add AV1 support #77
Conversation
plugins/src/webrtcsink/imp.rs
Outdated
| .field("format", &gst::List::new(&[&"NV12", &"YV12", &"I420"])); | ||
| } | ||
| "rav1enc" => { | ||
| // Unfortunately other formats don't yet work. |
There was a problem hiding this comment.
Yes, please provide more information here, the existing restriction is a workaround, which is why I tried to document it as precisely as possible :)
There was a problem hiding this comment.
Presumably, due to buggy encoder elements.
By default rav1enc would negotiate Y444_12LE and segfault.
av1enc (and rav1enc for other less verbose formats) seemed to work fine but Chromium fails to decode any of the frames sent over.
I could only get GRAY8, I420 and YV12 to actually render.
There was a problem hiding this comment.
Interesting, that should probably be reported. If I understand you correctly, av1enc with Y444_12LE was working fine with chrome?
Can you create issues against rav1e (not the GStreamer plugin) for that and CC me there?
There was a problem hiding this comment.
If I understand you correctly, av1enc with Y444_12LE was working fine with chrome?
No, the av1enc element doesn't have caps for Y444_12LE. For Y444, its same as other formats where chrome receives the frames but fails to decode them.
Can you create issues against rav1e (not the GStreamer plugin) for that and CC me there?
The segfault was from sometime ago. I rebuilt it now and that's fixed already. Y444_12LE just fails to render in chrome like the others now.
Interestingly gst-launch-1.0 videotestsrc ! video/x-raw,height=480,width=640,framerate=30/1,format="Y42B" ! av1enc ! rsdav1ddec ! videoconvert ! autovideosink fails for me. But it works if I use av1dec instead of dav1d.
There was a problem hiding this comment.
In any case, please create issues about all this and link them from the code here with some more explanation :)
There was a problem hiding this comment.
Chrome only supports the main profile for AV1 at the moment.
The issue here is that av1enc doesn't set its sink caps based on downstream profile [1] (even when its present). Also, we need to interpret a missing profile parameter in the SDP to mean to the main profile. We would probably need to special case this optional/missing param flow even after the av1enc bug is fixed.
A similar problem crashes Firefox for vp9enc [2]. Its decoder only supports the main profile for VP9.
For now, I updated the code to set the appropriate raw_filter based on downstream profile[-id] and added comments for the respective issues.
[1] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1404
[2] https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1403
|
looks good to me otherwise |
|
What's the status here? :) |
|
I think this is blocked by https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs/-/merge_requests/884#note_1569937 at least |
OK thanks, would be nice to see this go forward :) |
|
Closing @yatinmaan please re-open against https://gitlab.freedesktop.org/gstreamer/gst-plugins-rs when ready :) |
|
I think it would be ready now |
|
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/merge_requests/2743 is still listed as a dependency at the top though? |
This depends on rtpav1 and some changes to av1enc so those would need to be merged first.