I am trying to understand how the SSIM and VMAF filters work, with an
eye to finding the "best" compression settings for a video which will be composed from a series of TIFF images. Unfortunately, I'm stuck at the beginning, as I can't get the SSIM filter to behave as expected. ./source contains the original sequence of images. $ tiffinfo source/000000.tif TIFF Directory at offset 0x473108 (4665608) Image Width: 1440 Image Length: 1080 Bits/Sample: 8 Sample Format: unsigned integer Compression Scheme: None Photometric Interpretation: RGB color Samples/Pixel: 3 Rows/Strip: 1 Planar Configuration: single image plane I attempt to create a lossless video of the first minute. $ ffmpeg -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t 00:01:00 -c:v huffyuv lossless.mkv The result appears reasonable. $ mediainfo lossless.mkv General Unique ID : 235140899628261703308032414639716345340 (0xB0E67D6EF6B78362D9BCF9EA3080A5FC) Complete name : lossless.mkv Format : Matroska Format version : Version 4 File size : 8.54 GiB Duration : 59 s 994 ms Overall bit rate : 1 223 Mb/s Writing application : Lavf58.45.100 Writing library : Lavf58.45.100 ErrorDetectionType : Per level 1 Video ID : 1 Format : HuffYUV Format version : Version 2 Codec ID : V_MS/VFW/FOURCC / HFYU Duration : 59 s 994 ms Bit rate : 1 199 Mb/s Width : 1 440 pixels Height : 1 080 pixels Display aspect ratio : 4:3 Frame rate mode : Constant Frame rate : 59.940 FPS Color space : RGB Bit depth : 8 bits Scan type : Progressive Bits/(Pixel*Frame) : 12.860 Stream size : 8.37 GiB (98%) Writing library : Lavc58.91.100 huffyuv Default : Yes Forced : No Now let's see what the SSIM filter says. $ ffmpeg -i lossless.mkv -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t 00:01:00 -filter_complex ssim -f null - ... Input #0, matroska,webm, from 'lossless.mkv': Metadata: ENCODER : Lavf58.45.100 Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080, 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default) Metadata: ENCODER : Lavc58.91.100 huffyuv DURATION : 00:00:59.994000000 Input #1, image2, from 'source/%06d.tif': Duration: 01:47:16.00, start: 0.000000, bitrate: N/A Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, 59.94 tbc ... [Parsed_ssim_0 @ 0x55fb09c738c0] not matching timebases found between first input: 1/1000 and second input 1001/60000, results may be incorrect! ... [Parsed_ssim_0 @ 0x55fb09c738c0] SSIM R:0.833774 (7.793009) G:0.835401 (7.835723) B:0.831058 (7.722615) All:0.833411 (7.783532) That's not what I expected. My understanding is that the R, G, B, and All values should all be "1.000000 (inf)". The "not matching timebases" warning is the obvious thing to look at. After much searching, I came upon the -video_track_timescale option, but it seems to only take an integer, and 60 is not the same as 59.94, so it seems that I simply can't directly compare a video stream with a non- integer framerate to an image sequence. As a workaround, I tried extracting the lossless video frames as a separate image sequence. $ ffmpeg -i lossless.mkv -start_number 0 lossless/%06d.png This created the expected sequence of image files (000000.png - 003595.png). Since I have both the "source" and the "lossless" streams as images sequences, I can use ImageMagick to compare them. $ for I in `seq -w 0 3595` ; do compare -metric AE source/00${I}.tif lossless/00${I}.png /tmp/diff.png 2>/dev/null || echo $I ; done This produces no output, indicating that ImageMagick thinks that the TIFF files in ./source and the PNG files in ./lossless contain completely identical image data. What does the SSIM filter say? $ ffmpeg -framerate 60000/1001 -start_number 0 -i lossless/%06d.png -framerate 60000/1001 -start_number 0 -i source/%06d.tif -t 00:01:00 -filter_complex ssim -f null - ... Input #0, image2, from 'lossless/%06d.png': Duration: 00:00:59.99, start: 0.000000, bitrate: N/A Stream #0:0: Video: png, rgb24(pc), 1440x1080, 59.94 fps, 59.94 tbr, 59.94 tbn, 59.94 tbc Input #1, image2, from 'source/%06d.tif': Duration: 01:47:16.00, start: 0.000000, bitrate: N/A Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, 59.94 tbc ... [Parsed_ssim_0 @ 0x556c7e948980] SSIM R:0.999775 (36.484195) G:0.999777 (36.508857) B:0.999774 (36.451744) All:0.999775 (36.481536) Close, but not the "1.000000 (inf)" that I would expect for identical streams. I suppose that it may be related to the "rgb24(pc)" vs. rgb24 pixel formats; it's hard to be sure, as Google isn't turning up anything that describes what "rgb24(pc)" actually means. Does anyone know what the heck is going on here? Am I doing something wrong? -- ======================================================================== In Soviet Russia, Google searches you! ======================================================================== _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
Ian Pilcher wrote
> I am trying to understand how the SSIM and VMAF filters work, with an > eye to finding the "best" compression settings for a video which will > be composed from a series of TIFF images. Unfortunately, I'm stuck at > the beginning, as I can't get the SSIM filter to behave as expected. > > ./source contains the original sequence of images. > > $ tiffinfo source/000000.tif > TIFF Directory at offset 0x473108 (4665608) > Image Width: 1440 Image Length: 1080 > Bits/Sample: 8 > Sample Format: unsigned integer > Compression Scheme: None > Photometric Interpretation: RGB color > Samples/Pixel: 3 > Rows/Strip: 1 > Planar Configuration: single image plane > > I attempt to create a lossless video of the first minute. > > $ ffmpeg -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t > 00:01:00 -c:v huffyuv lossless.mkv > > The result appears reasonable. > > $ mediainfo lossless.mkv > General > Unique ID : > 235140899628261703308032414639716345340 > (0xB0E67D6EF6B78362D9BCF9EA3080A5FC) > Complete name : lossless.mkv > Format : Matroska > Format version : Version 4 > File size : 8.54 GiB > Duration : 59 s 994 ms > Overall bit rate : 1 223 Mb/s > Writing application : Lavf58.45.100 > Writing library : Lavf58.45.100 > ErrorDetectionType : Per level 1 > > Video > ID : 1 > Format : HuffYUV > Format version : Version 2 > Codec ID : V_MS/VFW/FOURCC / HFYU > Duration : 59 s 994 ms > Bit rate : 1 199 Mb/s > Width : 1 440 pixels > Height : 1 080 pixels > Display aspect ratio : 4:3 > Frame rate mode : Constant > Frame rate : 59.940 FPS > Color space : RGB > Bit depth : 8 bits > Scan type : Progressive > Bits/(Pixel*Frame) : 12.860 > Stream size : 8.37 GiB (98%) > Writing library : Lavc58.91.100 huffyuv > Default : Yes > Forced : No > > Now let's see what the SSIM filter says. > > $ ffmpeg -i lossless.mkv -start_number 0 -framerate 60000/1001 -i > source/%06d.tif -t 00:01:00 -filter_complex ssim -f null - > ... > Input #0, matroska,webm, from 'lossless.mkv': > Metadata: > ENCODER : Lavf58.45.100 > Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s > Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080, > 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default) > Metadata: > ENCODER : Lavc58.91.100 huffyuv > DURATION : 00:00:59.994000000 > Input #1, image2, from 'source/%06d.tif': > Duration: 01:47:16.00, start: 0.000000, bitrate: N/A > Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, > 59.94 tbc > ... > [Parsed_ssim_0 @ 0x55fb09c738c0] not matching timebases found between > first input: 1/1000 and second input 1001/60000, results may be incorrect! > ... > [Parsed_ssim_0 @ 0x55fb09c738c0] SSIM R:0.833774 (7.793009) G:0.835401 > (7.835723) B:0.831058 (7.722615) All:0.833411 (7.783532) > > That's not what I expected. My understanding is that the R, G, B, and > All values should all be "1.000000 (inf)". > > The "not matching timebases" warning is the obvious thing to look at. > After much searching, I came upon the -video_track_timescale option, but > it seems to only take an integer, and 60 is not the same as 59.94, so it > seems that I simply can't directly compare a video stream with a non- > integer framerate to an image sequence. > > As a workaround, I tried extracting the lossless video frames as a > separate image sequence. > > $ ffmpeg -i lossless.mkv -start_number 0 lossless/%06d.png > > This created the expected sequence of image files (000000.png - > 003595.png). Since I have both the "source" and the "lossless" streams > as images sequences, I can use ImageMagick to compare them. > > $ for I in `seq -w 0 3595` ; do compare -metric AE source/00${I}.tif > lossless/00${I}.png /tmp/diff.png 2>/dev/null || echo $I ; done > > This produces no output, indicating that ImageMagick thinks that the > TIFF files in ./source and the PNG files in ./lossless contain > completely identical image data. What does the SSIM filter say? > > $ ffmpeg -framerate 60000/1001 -start_number 0 -i lossless/%06d.png > -framerate 60000/1001 -start_number 0 -i source/%06d.tif -t 00:01:00 > -filter_complex ssim -f null - > ... > Input #0, image2, from 'lossless/%06d.png': > Duration: 00:00:59.99, start: 0.000000, bitrate: N/A > Stream #0:0: Video: png, rgb24(pc), 1440x1080, 59.94 fps, 59.94 > tbr, 59.94 tbn, 59.94 tbc > Input #1, image2, from 'source/%06d.tif': > Duration: 01:47:16.00, start: 0.000000, bitrate: N/A > Stream #1:0: Video: tiff, rgb24, 1440x1080, 59.94 tbr, 59.94 tbn, > 59.94 tbc > ... > [Parsed_ssim_0 @ 0x556c7e948980] SSIM R:0.999775 (36.484195) > G:0.999777 (36.508857) B:0.999774 (36.451744) All:0.999775 (36.481536) > > Close, but not the "1.000000 (inf)" that I would expect for identical > streams. I suppose that it may be related to the "rgb24(pc)" vs. > rgb24 pixel formats; it's hard to be sure, as Google isn't turning up > anything that describes what "rgb24(pc)" actually means. > > Does anyone know what the heck is going on here? Am I doing something > wrong? Different timebase means there is potential to compare different frames. Use settb ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 -framerate 60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" -f null - For the 2nd image sequence png vs. tiff, one is rgb24(pc) , one is rgb24 . There might be some metadata that is skewing the results. Need more info -- Sent from: http://ffmpeg-users.933282.n4.nabble.com/ _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
Am Sa., 27. Feb. 2021 um 04:53 Uhr schrieb pdr0 <[hidden email]>:
> ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 > -framerate 60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > -f null - I suspect you have to remove the first "framerate", the mkv demuxer doesn't know this option. > For the 2nd image sequence png vs. tiff, one is rgb24(pc) , one is rgb24 . Iirc, the ssim filter ignores this metadata difference. Carl Eugen _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
On Sat, Feb 27, 2021 at 9:37 AM Carl Eugen Hoyos <[hidden email]> wrote:
> Am Sa., 27. Feb. 2021 um 04:53 Uhr schrieb pdr0 <[hidden email]>: > > > ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 > > -framerate 60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi > > > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > > -f null - > > I suspect you have to remove the first "framerate", > the mkv demuxer doesn't know this option. > > > For the 2nd image sequence png vs. tiff, one is rgb24(pc) , one is rgb24 > . > > Iirc, the ssim filter ignores this metadata difference. > but not the swscale filter Full log would show this. > Carl Eugen > _______________________________________________ > ffmpeg-user mailing list > [hidden email] > https://ffmpeg.org/mailman/listinfo/ffmpeg-user > > To unsubscribe, visit link above, or email > [hidden email] with subject "unsubscribe". _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
In reply to this post by pdr0
On 2/26/21 9:52 PM, pdr0 wrote:
> Different timebase means there is potential to compare different frames. > > Use settb > > ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 -framerate > 60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > -f null - That seems to work. It gets rid of the "not matching timebases" warning. It does cause the duration option (-t 00:01:00) to not work, but I was able to use the "-frames:v" option instead. (In fact, I don't need specify the framerate of the image sequence at all.) $ ffmpeg -i lossless.mkv -start_number 0 -i source/%06d.tif -frames:v 3596 -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" -f null - ... Input #0, matroska,webm, from 'lossless.mkv': Metadata: ENCODER : Lavf58.45.100 Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080, 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default) Metadata: ENCODER : Lavc58.91.100 huffyuv DURATION : 00:00:59.994000000 Input #1, image2, from 'source/%06d.tif': Duration: 04:17:10.96, start: 0.000000, bitrate: N/A Stream #1:0: Video: tiff, rgb24, 1440x1080, 25 tbr, 25 tbn, 25 tbc ... [Parsed_ssim_4 @ 0x562a600ac780] SSIM R:0.999888 (39.506142) G:0.999889 (39.531716) B:0.999887 (39.466715) All:0.999888 (39.501442) So this gets me to the same result that I got when I compared the derived PNG image sequence to the original TIFF sequence. $ ffmpeg -start_number 0 -i lossless/%06d.png -start_number 0 -i source/%06d.tif -frames:v 3596 -filter_complex ssim -f null - ... [Parsed_ssim_0 @ 0x5630c77ad440] SSIM R:0.999888 (39.506142) G:0.999889 (39.531716) B:0.999887 (39.466715) All:0.999888 (39.501442) Now that I know about the -frames:v option, I decided to try omitting the last frame from the comparison. $ ffmpeg -start_number 0 -i lossless/%06d.png -start_number 0 -i source/%06d.tif -frames:v 3595 -filter_complex ssim -f null - ... [Parsed_ssim_0 @ 0x560797482440] SSIM R:1.000000 (inf) G:1.000000 (inf) B:1.000000 (inf) All:1.000000 (inf) $ ffmpeg -i lossless.mkv -start_number 0 -i source/%06d.tif -frames:v 3595 -lavfi "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" -f null - ... [Parsed_ssim_4 @ 0x5571f0a97780] SSIM R:1.000000 (inf) G:1.000000 (inf) B:1.000000 (inf) All:1.000000 (inf) That gets me the expected score. I don't understand why I have to omit the final frame. Both ImageMagick and ffmpeg show the images as identical. $ compare -metric AE lossless/003595.png source/003595.tif /tmp/diff.png 0 $ ffmpeg -i lossless/003595.png -i source/003595.tif -filter_complex ssim -f null - ... [Parsed_ssim_0 @ 0x561a6aae1540] SSIM R:1.000000 (inf) G:1.000000 (inf) B:1.000000 (inf) All:1.000000 (inf) This is definitely good enough for my purposes, however. I'm going to declare victory and move on. Thanks for your help! -- ======================================================================== In Soviet Russia, Google searches you! ======================================================================== _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
On Sun, Feb 28, 2021 at 5:31 PM Ian Pilcher <[hidden email]> wrote:
> On 2/26/21 9:52 PM, pdr0 wrote: > > Different timebase means there is potential to compare different frames. > > > > Use settb > > > > ffmpeg -framerate 60000/1001 -i lossless.mkv -start_number 0 -framerate > > 60000/1001 -i source/%06d.tif -t 00:01:00 -lavfi > > > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > > -f null - > > That seems to work. It gets rid of the "not matching timebases" > warning. It does cause the duration option (-t 00:01:00) to not work, > but I was able to use the "-frames:v" option instead. (In fact, I don't > need specify the framerate of the image sequence at all.) > > $ ffmpeg -i lossless.mkv -start_number 0 -i source/%06d.tif -frames:v > 3596 -lavfi > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > > -f null - > ... > Input #0, matroska,webm, from 'lossless.mkv': > Metadata: > ENCODER : Lavf58.45.100 > Duration: 00:00:59.99, start: 0.000000, bitrate: 1223104 kb/s > Stream #0:0: Video: huffyuv (HFYU / 0x55594648), bgr0, 1440x1080, > 59.94 fps, 59.94 tbr, 1k tbn, 1k tbc (default) > Metadata: > ENCODER : Lavc58.91.100 huffyuv > DURATION : 00:00:59.994000000 > Input #1, image2, from 'source/%06d.tif': > Duration: 04:17:10.96, start: 0.000000, bitrate: N/A > Stream #1:0: Video: tiff, rgb24, 1440x1080, 25 tbr, 25 tbn, 25 tbc > ... > [Parsed_ssim_4 @ 0x562a600ac780] SSIM R:0.999888 (39.506142) > G:0.999889 (39.531716) B:0.999887 (39.466715) All:0.999888 (39.501442) > > So this gets me to the same result that I got when I compared the > derived PNG image sequence to the original TIFF sequence. > > $ ffmpeg -start_number 0 -i lossless/%06d.png -start_number 0 -i > source/%06d.tif -frames:v 3596 -filter_complex ssim -f null - > ... > [Parsed_ssim_0 @ 0x5630c77ad440] SSIM R:0.999888 (39.506142) > G:0.999889 (39.531716) B:0.999887 (39.466715) All:0.999888 (39.501442) > > Now that I know about the -frames:v option, I decided to try omitting > the last frame from the comparison. > > $ ffmpeg -start_number 0 -i lossless/%06d.png -start_number 0 -i > source/%06d.tif -frames:v 3595 -filter_complex ssim -f null - > ... > [Parsed_ssim_0 @ 0x560797482440] SSIM R:1.000000 (inf) G:1.000000 > (inf) B:1.000000 (inf) All:1.000000 (inf) > > $ ffmpeg -i lossless.mkv -start_number 0 -i source/%06d.tif -frames:v > 3595 -lavfi > "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" > > -f null - > ... > [Parsed_ssim_4 @ 0x5571f0a97780] SSIM R:1.000000 (inf) G:1.000000 > (inf) B:1.000000 (inf) All:1.000000 (inf) > > That gets me the expected score. > > I don't understand why I have to omit the final frame. Both ImageMagick > and ffmpeg show the images as identical. > > $ compare -metric AE lossless/003595.png source/003595.tif /tmp/diff.png > 0 > > $ ffmpeg -i lossless/003595.png -i source/003595.tif -filter_complex > ssim -f null - > ... > [Parsed_ssim_0 @ 0x561a6aae1540] SSIM R:1.000000 (inf) G:1.000000 > (inf) B:1.000000 (inf) All:1.000000 (inf) > > This is definitely good enough for my purposes, however. I'm going to > declare victory and move on. > Try ssim=shortest=1 Most likely inputs have not same number of frames. > > Thanks for your help! > > -- > ======================================================================== > In Soviet Russia, Google searches you! > ======================================================================== > > _______________________________________________ > ffmpeg-user mailing list > [hidden email] > https://ffmpeg.org/mailman/listinfo/ffmpeg-user > > To unsubscribe, visit link above, or email > [hidden email] with subject "unsubscribe". ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
On 2/28/21 12:57 PM, Paul B Mahol wrote:
> On Sun, Feb 28, 2021 at 5:31 PM Ian Pilcher <[hidden email]> wrote: >> $ ffmpeg -i lossless.mkv -start_number 0 -i source/%06d.tif -frames:v >> 3596 -lavfi >> "[0:v]settb=1/AVTB,setpts=PTS-STARTPTS[main];[1:v]settb=1/AVTB,setpts=PTS-STARTPTS[ref];[main][ref]ssim" >> >> -f null - > > Try ssim=shortest=1 > > Most likely inputs have not same number of frames. They don't have the same number of frames. That's why I need to specify the duration, either with "-t 00:01:00" or with "-frames:v 3596". The correct number of frames *is* 3596, however, so I don't understand why I have to limit the comparison to only 3595 frames to get the correct result. -- ======================================================================== In Soviet Russia, Google searches you! ======================================================================== _______________________________________________ ffmpeg-user mailing list [hidden email] https://ffmpeg.org/mailman/listinfo/ffmpeg-user To unsubscribe, visit link above, or email [hidden email] with subject "unsubscribe". |
Free forum by Nabble | Edit this page |