v4l2 overlay on mjpeg stream

[Baltazár Radics]

I'm trying to overlay a v4l2 input on the mjpeg stream of an android IP
camera. The first few seconds are alright, but the overlay slowly lags
behind more and more, and starts dropping frames.
I also get the following warning:
>
> Thread message queue blocking; consider raising the thread_queue_size
> option

I have tried raising it but it doesn't seem to help.
The command I'm using (the output is a v4l2loopback device):
ffmpeg -i https://192.168.11.139:8080/video -i /dev/video3 -filter_complex
'[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
-vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2
Overlaying a v4l2 source on another v4l2 source works perfectly with the
same filter.
What could be causing the issue?

Thanks
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

[Carl Eugen Hoyos-2]

Am So., 24. Jan. 2021 um 23:34 Uhr schrieb Baltazár Radics
<[hidden email]>:

>
> I'm trying to overlay a v4l2 input on the mjpeg stream of an android IP
> camera. The first few seconds are alright, but the overlay slowly lags
> behind more and more, and starts dropping frames.
> I also get the following warning:
> >
> > Thread message queue blocking; consider raising the
> > thread_queue_size option
>
> I have tried raising it but it doesn't seem to help.
> The command I'm using (the output is a v4l2loopback device):
> ffmpeg -i https://192.168.11.139:8080/video -i /dev/video3 -filter_complex
> '[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
> -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2

Please test current FFmpeg git head and provide the command line you tested
together with the complete, uncut console output.

> Overlaying a v4l2 source on another v4l2 source works perfectly with the
> same filter.

Feel free to show this one as well.

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

[Baltazár Radics]

output of `ffmpeg -i https://192.168.11.139:8080/video -thread_queue_size
2048 -i /dev/video3 -filter_complex
'[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam]
[calc]overlay' -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2`:
http://ix.io/2NeC
output of `ffmpeg -i /dev/video0 -i /dev/video3 -filter_complex
'[0:v]scale=iw*4/3:-1[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
-vcodec rawvideo -pix_fmt yuv420p -f v4l2 /de
v/video2`: http://ix.io/2NeD

In both cases, I pressed `q` after ten seconds or so.
I have built ffmpeg version N-100777-g2bcec40cce, but that should be
visible in the output.

On Mon, Jan 25, 2021 at 12:03 AM Carl Eugen Hoyos <[hidden email]>
wrote:

> Am So., 24. Jan. 2021 um 23:34 Uhr schrieb Baltazár Radics
> <[hidden email]>:
> >
> > I'm trying to overlay a v4l2 input on the mjpeg stream of an android IP
> > camera. The first few seconds are alright, but the overlay slowly lags
> > behind more and more, and starts dropping frames.
> > I also get the following warning:
> > >
> > > Thread message queue blocking; consider raising the
> > > thread_queue_size option
> >
> > I have tried raising it but it doesn't seem to help.
> > The command I'm using (the output is a v4l2loopback device):
> > ffmpeg -i https://192.168.11.139:8080/video -i /dev/video3
> -filter_complex
> >
> '[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
> > -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2
>
> Please test current FFmpeg git head and provide the command line you tested
> together with the complete, uncut console output.
>
> > Overlaying a v4l2 source on another v4l2 source works perfectly with the
> > same filter.
>
> Feel free to show this one as well.
>
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

[Carl Eugen Hoyos-2]

Am Mo., 25. Jan. 2021 um 00:42 Uhr schrieb Baltazár Radics
<[hidden email]>:

>
> output of `ffmpeg -i https://192.168.11.139:8080/video -thread_queue_size
> 2048 -i /dev/video3 -filter_complex
> '[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam]
> [calc]overlay' -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2`:
> http://ix.io/2NeC
> output of `ffmpeg -i /dev/video0 -i /dev/video3 -filter_complex
> '[0:v]scale=iw*4/3:-1[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
> -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /de
> v/video2`: http://ix.io/2NeD

For future questions:
Please always post all information here on the mailing list, do not
use external resources
except for input samples.
And please find out what top-posting means and avoid it here.

> Input #0, mpjpeg, from 'https://192.168.11.139:8080/video':
>   Duration: N/A, bitrate: N/A
>     Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc, bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25 tbc
> Input #1, video4linux2,v4l2, from '/dev/video3':
>   Duration: N/A, start: 36730.385449, bitrate: 1985 kb/s
>     Stream #1:0: Video: rawvideo (Y800 / 0x30303859), gray, 128x64, 1985 kb/s, 30.30 fps, 30.30 tbr, 1000k tbn, 1000k tbc

You cannot synchronize these streams because they have different start times,
there should be a filter incantation that resets the timestamps of the
second input,
something like "setpts=PTS-STARTPTS"

Carl Eugen
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

[Baltazár Radics]

On Mon, Jan 25, 2021 at 12:59 AM Carl Eugen Hoyos <[hidden email]>
wrote:

> For future questions:
> Please always post all information here on the mailing list, do not
> use external resources
> except for input samples.
>

Sorry, I thought I'd read to use pastesites but that must have been another
forum.

You cannot synchronize these streams because they have different start
> times,
> there should be a filter incantation that resets the timestamps of the
> second input,
> something like "setpts=PTS-STARTPTS"
>

That didn't help, but it made me read what setpts did, and apparently the
problem was that the mjpeg stream didn't have timestamps at all.
This command worked in the end: `ffmpeg -i https://192.168.11.139:8080/video
-i /dev/video3 -filter_complex
'[0:v]setpts=(RTCTIME-RTCSTART)/(TB*1000000),scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
-vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2`

Thanks again

On Mon, Jan 25, 2021 at 12:59 AM Carl Eugen Hoyos <[hidden email]>
wrote:

> Am Mo., 25. Jan. 2021 um 00:42 Uhr schrieb Baltazár Radics
> <[hidden email]>:
> >
> > output of `ffmpeg -i https://192.168.11.139:8080/video
> -thread_queue_size
> > 2048 -i /dev/video3 -filter_complex
> >
> '[0:v]scale=iw*4/3:-1,crop=1280:720[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam]
> > [calc]overlay' -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /dev/video2`:
> > http://ix.io/2NeC
> > output of `ffmpeg -i /dev/video0 -i /dev/video3 -filter_complex
> >
> '[0:v]scale=iw*4/3:-1[cam];[1:v]scale=4*iw:-1:flags=neighbor[calc];[cam][calc]overlay'
> > -vcodec rawvideo -pix_fmt yuv420p -f v4l2 /de
> > v/video2`: http://ix.io/2NeD
>
> For future questions:
> Please always post all information here on the mailing list, do not
> use external resources
> except for input samples.
> And please find out what top-posting means and avoid it here.
>
> > Input #0, mpjpeg, from 'https://192.168.11.139:8080/video':
> >   Duration: N/A, bitrate: N/A
> >     Stream #0:0: Video: mjpeg (Baseline), yuvj420p(pc,
> bt470bg/unknown/unknown), 1280x720 [SAR 1:1 DAR 16:9], 25 tbr, 25 tbn, 25
> tbc
> > Input #1, video4linux2,v4l2, from '/dev/video3':
> >   Duration: N/A, start: 36730.385449, bitrate: 1985 kb/s
> >     Stream #1:0: Video: rawvideo (Y800 / 0x30303859), gray, 128x64, 1985
> kb/s, 30.30 fps, 30.30 tbr, 1000k tbn, 1000k tbc
>
> You cannot synchronize these streams because they have different start
> times,
> there should be a filter incantation that resets the timestamps of the
> second input,
> something like "setpts=PTS-STARTPTS"
>
> Carl Eugen
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://ffmpeg.org/mailman/listinfo/ffmpeg-user
>
> To unsubscribe, visit link above, or email
> [hidden email] with subject "unsubscribe".

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".