'mix' filter questions

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

'mix' filter questions

Mark Filipak (ffmpeg)
Synopsis: I present the 'mix' filter (in compact format) followed by criticism followed by my
rewrite followed by remaining questions.

Tip: Before responding, I suggest you save time and effort by reading below the divider (=====). If
I have it right, then kindly say so; otherwise, let me know what I have wrong.

If our email clients mangle the message formatting, I apologize. I've attached an unmangled version.

Thanks a lot!
Mark.

PS: I'm asking for help, not because I'm lazy, but because I'm overloaded and you folks are probably
very familiar with the filter and can probably confirm/refute what I write from the top of your
heads without taking much time. -- M.


mix   ...Mix several video input streams into one video stream.
mix=nb_inputs=2:weights=Ø:scale=Ø:duration=longest   ...Defaults (Ø means no default).
               :         :       :          :
               :         :       :          Specify how end of stream is determined.
               :         :       :          longest    ...The duration of the longest input.
               :         :       :          shortest   ...The duration of the shortest input.
               :         :       :          first      ...The duration of the first input.
               :         :       :
               :         :       Specify scale, if it is set it will be multiplied with sum of each
weight multiplied with pixel values to give final destination pixel value. By default scale is auto
scaled to sum of weights.
               :         :
               :         Specify weight of each input video stream as sequence. Each weight is
separated by space. If number of weights is smaller than number of frames last specified weight will
be used for all remaining unset weights.
               :
               The number of inputs.

Criticism (re, "Specify scale"):
   "if it is set[,] it will be multiplied with sum of each weight multiplied with pixel values to
give final destination pixel value"
   That vaguely describes how the filter uses weights but not what scale is.
   The more I read that sentence, and the more I try to form a mental picture of the process, the
less sense it makes.
Criticism (re, scale):
   Is there one scale for all inputs or one scale per input?
Criticism (re, "Specify scale"):
   "sum of each weight" -- Really? What does "sum" of "each" mean?
   Assuming that what's intended is "sum of [all/the] weights", why would you multiply pixel values
by the sum of weights? -- all pixel values would be multiplied by the same total weight. That
wouldn't make sense, would it?
Criticism (re, "Specify scale"): "By default[,] scale is auto scaled to sum of weights." "scale is
auto scaled" -- Really? What does that really mean?
Criticism (re, "Specify weight"): "If number of weights is smaller than number of frames" -- Huh?
"frames"? Is there one weight per frame? ...That would make for a truly gigantic command line, eh?

=====

mix   ...Combine multiple streams via weighted, scaled, pixel mixing.
Changes frame rate? No, Frames? No, Fields? No, Pixels? Yes.

Process: for each input: '[A]' '[B]' etc.,
           for each frame,
            for each pixel: 'pixel_A' 'pixel_B' etc.,
             input pixels --> [A] pixel_A x weight_A x scale_A --------.
             input pixels --> [B] pixel_B x weight_B x scale_B ---.    ¦
                               :                                  ¦    ¦
             input pixels --> [Z] pixel_Z x weight_Z x scale_Z --(+)--(+)--> output pixel

               The number of inputs.
               :          first      ...End the output at the end of the first input.
               :          shortest   ...End the output at the end of the shortest input.
               :          longest    ...End the output at the end of the longest input.
               :          :
mix=nb_inputs=2:duration=longest:weights=?:scale=[note 1]   ...Defaults (? means no default).
                                          :       :
                                          :       Space-delimited number list [note 1].
                                          Space-delimited number list [note 2].

[note 1] One scale per input. If 'scale' is missing, a per-input (default) scale is applied: scale =
weight/(sum of all assigned weights).
[note 2] One weight per input. At least one weight must be listed, but not all inputs require that a
weight be listed. If there are more inputs than weights, the last weight in the list is assigned to
the remaining inputs.

Remaining questions:
What is 'scale' datatype?
What is 'weight' datatype?
Can inputs have differing frame rates? Differing frame sizes? Differing pixel formats?

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".

mix.filter.txt (5K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: 'mix' filter questions

Michael Koch
Am 20.03.2021 um 04:31 schrieb Mark Filipak (ffmpeg):
> :         :       :
>               :         :       Specify scale, if it is set it will be
> multiplied with sum of each weight multiplied with pixel values to
> give final destination pixel value. By default scale is auto scaled to
> sum of weights.

Correct would be "By default scale is set to (1 / sum_of weights)".
The same error is also in the documentation of the "tmix" filter.

>   Is there one scale for all inputs or one scale per input?

One for all inputs.

> Criticism (re, "Specify weight"): "If number of weights is smaller
> than number of frames" -- Huh?

"number of inputs"
The same error is also in the documentation of the "tmix" filter.

Michael
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://ffmpeg.org/mailman/listinfo/ffmpeg-user

To unsubscribe, visit link above, or email
[hidden email] with subject "unsubscribe".