FFmpeg-users

Scene detection

Classic

List

Threaded

34 messages Options

rogerdpack2

Re: Scene detection

>> Or is there any way to feed some sort of "edit list" to ffmpeg?

you can specify start and stop times. EDL support would be a nice
feature at some point :)

> Scene-detection looks like a much requested feature, please file a
> feature request on the issue tracker. I'll hope we'll be able to find
> some mechanisms for allowing users to (micro?)fund and vote on the
> issues/features they want to be fixed.

I've been interested in creating something like this recently.
Contact me offline to discuss it and requirements :)

re: scene detection

You could split your video to jpeg's then use imagemagick's "diff"
tool metric, and detect frames that have large diff's (sorry if this
was already mentioned previously).

Cheers!
-roger-
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

rogerdpack2

Re: Scene detection

In reply to this post by Stefano Sabatini

> Scene-detection looks like a much requested feature, please file a
> feature request on the issue tracker. I'll hope we'll be able to find
> some mechanisms for allowing users to (micro?)fund and vote on the
> issues/features they want to be fixed.

A bit unrelated, but I'd be willing to donate say $100 toward a
"motion detection" filter (you specify rectangles and intensity
thresholds, if any is triggered it allows the frame through, otherwise
it drops it).
Thanks!
-roger-
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Peter B.

Re: Scene detection

In reply to this post by lovelove

lovelove wrote:
> Thank you Peter. I tested shotdetect today and it really has some nice
> concepts. The http://shotdetect.nonutc.fr/ graphs are really cool. Problem
> is, almost all input videos failed. They were either outright refused or
> failed to produce any meaningful output. For example, just try any video
> from youtube, they all fail. I got only a single video to work.
>
Sorry, sorry, sorry for my late reply!
I was on vacation and didn't overlooked your feedback. :(

Here's the patch for improving the current version of shotdetect
(1.0.86) [1]

If you've been using the official shotdetect binary, I presume the
problems with the videos are, because it's compiled against a very old
version of ffmpeg (pre v.0.5).
My patch updates the code base to compile against the current version of
ffmpeg, which should fix the issues you're having with not being able to
open videos.

Additionally, I've added time-markers in the graphs, so they're more
useful to read :)

The patched version has now been in production use on Debian Lenny and
Squeeze systems at the Austrian Mediathek [2] for over a year now.

Regards,
Pb

== References:
[1]
http://download.das-werkstatt.com/pb/contribs/patches/shotdetect-1.0.86_to_1.0.86dva1.patch
[2] http://dva-profession.mediathek.at/
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

rogerdpack2

Re: Scene detection

> Here's the patch for improving the current version of shotdetect
> (1.0.86) [1]

Have you submitted them upstream I presume? :)

> Additionally, I've added time-markers in the graphs, so they're more
> useful to read :)

Also I assume libx264 uses something like this to decide on its
i-frame's (without having to do 2-pass)... ?
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Peter B.

Re: Scene detection

Roger Pack wrote:
>> Here's the patch for improving the current version of shotdetect
>> (1.0.86) [1]
>>
>
> Have you submitted them upstream I presume? :)
>
Yes, I did :)

I'm directly collaborating with the original author, Johan Mathe.
I'm always committing my changes back upstream, but the new version
hasn't been released yet - therefore the patch.

Pb
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

rogerdpack2

Re: Scene detection

In reply to this post by rogerdpack2

> Also I assume libx264 uses something like this to decide on its
> i-frame's (without having to do 2-pass)... ?

It apperas that libx264 does some scene cut detection internally, to
look for i-frames,
viz vlc's man page:

--sout-x264-scenecut=<integer [-1 .. 100]>
Extra I-frames aggressivity
Scene-cut detection. Controls how aggressively to insert extra
I-frames. With small values of scenecut, the codec often has to force
an I-frame when it would exceed keyint. Good values of scenecut may
find a better location for the I-frame. Large values use more
I-frames than necessary, thus wasting bits. -1 disables scene-cut
detection, so I-frames are inserted only every other keyint frames,
which probably leads to ugly encoding artifacts. Range 1 to 100.

I wonder why 2 pass is so much better then...
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Peter B.

Re: Scene detection

Roger Pack wrote:
> It apperas that libx264 does some scene cut detection internally, to
> look for i-frames,
>
> I wonder why 2 pass is so much better then...
>
I think because 2-pass is not only about when to put i-frames, but
rather how to distribute the available encoding bandwidth among the
video in an optimized way.

Example:
2 scenes in a video:
1) several minutes of almost no motion: low bandwidth requirement
2) a short scene with lots of motion: high bandwidth requirement

With 2-pass encoding, the codec is able to decide to save some bits in
the low-motion sequences in order to use them later on in high-motion
scenes.

At least this is how I understood its purpose...

Pb
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

gxr

Re: Scene detection

Hi,

Does anybody had any success in compiling shotdetect with the latest ffmpeg version.
I am trying to compile in mac osx. I have used the patches posted in this thread as well as another one found on the net but with no success. Then tweaked the code to solve some error cpp issues related to some changes in ffmpeg and at the end managed to succeed but I get the following error when using the executable:

shotdetect -i example.avi -o ./shots -s 60
Shot log :: Processing movie.
./shots/motion.csv
Using network protocols without global network initialization. Please use avformat_network_init(), this will become mandatory later.
Shot log :: 1568
Shot log :: 1968
Shot log :: 2168
...
Shot log :: 74407
Shot log :: 74607
Shot log :: 74808
Abort trap: 6

Can somebody help?
If you have succeeded in compiling shotdetect what version of ffmpeg did you use?

-George

Peter B.

Re: Scene detection

Zitat von gxr <[hidden email]>:

> Does anybody had any success in compiling shotdetect with the latest ffmpeg
> version.

I'm using a patched, self-compiled shotdetect version for our video
archiving tool. Daily.
However, it's been compiled with FFmpeg v0.5.2 (so far away from recent)

> I have used the patches posted in this
> thread as well as another one found on the net but with no success.

I am not sure which thread you're referring to, as I could only find
your current message in the mailing list archive [1]

I've written and released a patch [2] which makes it possible to
compile to currently available "latest" shotdetect version (1.0.86)
[3]. My patch has already been applied upstream, but the new version
hasn't been released yet.

As I do require shotdetect to build against FFmpeg, I'll have to make
this work anyway - unfortunately, my schedule is quite full, so I
can't tell when I'll find the time to do so.

After applying the patch (1.0.86-1.0.86dva1), could you please post
which compiler errors you're getting?

Yours,
Pb

== References:
[1] https://lists.ffmpeg.org/pipermail/ffmpeg-user/2012-February/005085.html
[2]
http://dva-profession.svn.sourceforge.net/viewvc/dva-profession/tags/REL-1.0/misc/
[3] http://shotdetect.nonutc.fr/releases/latest.tar.gz

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

Peter B.

Re: Scene detection

In reply to this post by gxr

Clément Bœsch

Re: Scene detection

In reply to this post by Peter B.

On Mon, Feb 20, 2012 at 11:34:54AM +0100, Peter B. wrote:
> Zitat von gxr <[hidden email]>:
>
> >Does anybody had any success in compiling shotdetect with the latest ffmpeg
> >version.
>
> I'm using a patched, self-compiled shotdetect version for our video
> archiving tool. Daily.
> However, it's been compiled with FFmpeg v0.5.2 (so far away from recent)
>

As a side note, if someone is motivated, maybe
https://ffmpeg.org/trac/ffmpeg/ticket/442 could be solved by porting
shotdetect in libavfilter; this will ensure the feature will always be
available.

I don't think it will require that much work.

[...]

--
Clément B.

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

attachment0 (501 bytes) Download Attachment

Peter B.

Re: Scene detection

Zitat von Clément B?sch <[hidden email]>:
> As a side note, if someone is motivated, maybe
> https://ffmpeg.org/trac/ffmpeg/ticket/442 could be solved by porting
> shotdetect in libavfilter; this will ensure the feature will always be
> available.

I absolutely agree.
I've generally been thinking of porting shotdetect's functionality to
libavfilter in the long term to ensure its availability.

Shotdetect does not only provide a scene-detection feature, but also
returns graph images which are incredibly useful for quality-checks of
video material.

We (the Video-archive team at the Austrian Mediathek) will definitely
need a way of sustaining and possibly improving the features currently
provided by shotdetect.

However, this is currently non high-priority, so it's not scheduled
yet - but it will. Sooner or later...

Regards,
Pb

_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

atigian

Re: Scene detection

In reply to this post by rogerdpack2

Roger Pack wrote

EDL support would be a nice
feature at some point :)

Using Powershell I've been able to create a batch file that generates thumbnails for every shot change in the EDL file:
ffmpeg -i videos\your_video_file.mov -f image2 -ss 00:00:00.000 -f image2 -s 146x82 -vframes 1 images\00000000.jpg
ffmpeg -i videos\your_video_file.mov -f image2 -ss 00:00:42.000 -f image2 -s 146x82 -vframes 1 images\00004200.jpg

I explain the whole conversion process and how to sync the thumbnails with a video player in this link:
http://www.videoproductionslondon.com/blog/edl-to-html-with-thumbnails

When I don't have an EDL, my approach is to do the scene detection during the encoding using keyint, min-keyint and scenecut, thus inserting a key frame in every scene change. Once I've encoded the file I just extract all key frames:
ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i myvideo.mp4 -vsync 2 -s 73x41 -f image2 thumbnails-%02d.jpeg

I also create a text file with the timecodes of the key frames by adding:
-loglevel debug 2>&1 | grep "pict_type:I -> select:1" | cut -d " " -f 6 - > keyframe-timecodes.txt

And I use this file to sync the thumbnails of the scene changes with a video player:
http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

Boštjan Strojan

Re: Scene detection

On Wed, Feb 29, 2012 at 4:56 PM, atigian <[hidden email]> wrote:

> When I don't have an EDL, my approach is to do the scene detection during
> the encoding using keyint, min-keyint and scenecut, thus inserting a key
> frame in every scene change. Once I've encoded the file I just extract all
> key frames:
> ffmpeg -vf select="eq(pict_type\,PICT_TYPE_I)" -i myvideo.mp4 -vsync 2 -s
> 73x41 -f image2 thumbnails-%02d.jpeg
>
> I also create a text file with the timecodes of the key frames by adding:
> -loglevel debug 2>&1 | grep "pict_type:I -> select:1" | cut -d " " -f 6 - >
> keyframe-timecodes.txt
>
> And I use this file to sync the thumbnails of the scene changes with a video
> player:
> http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code
> http://www.videoproductionslondon.com/blog/scene-change-detection-during-encoding-key-frame-extraction-code

Thanks, much appreciated.

b.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user