Scene detection

[Fori Pepe]

Hi,

I would like to generate images from videos at every scene changes.
My present solution is:
- ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
- lav2yuv -S list.el destination_video.avi
It generates a scene list text file with time codes that I use later to creating images.

I was wondering maybe there is an easier solution.
I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.

Is it possible to use ffmpeg to create images on scene changes?
Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?

If you have any idea, I would be very happy.
Thanks in advance.



     
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Bouke-3]

---- Original Message -----
From: "Fori Pepe" <[hidden email]>
To: <[hidden email]>
Sent: Thursday, October 01, 2009 5:25 PM
Subject: [FFmpeg-user] Scene detection

> Hi,
>
> I would like to generate images from videos at every scene changes.
> My present solution is:
> - ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
> - lav2yuv -S list.el destination_video.avi
> It generates a scene list text file with time codes that I use later to
> creating images.
>
> I was wondering maybe there is an easier solution.
> I have browsed the net to get a better solution (ffmpeg, sc_threshold
> etc.), but I couldn't find any other just my present lav2yus solution.
>
> Is it possible to use ffmpeg to create images on scene changes?
> Or is it possible to use ffmpeg to create a text file with the time codes
> of scene changes?
>
> If you have any idea, I would be very happy.

No idea with FFmpeg, but a couple of years ago i had a similar need, and
i've created this:
http://www.videotoolshed.com/?page=products&pID=13

Bouke

> Thanks in advance.
>
>
>
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
>

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Stefano Sabatini]

On date Thursday 2009-10-01 08:25:39 -0700, Fori Pepe encoded:

> Hi,
>
> I would like to generate images from videos at every scene changes.
> My present solution is:
> - ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
> - lav2yuv -S list.el destination_video.avi
> It generates a scene list text file with time codes that I use later to creating images.
>
> I was wondering maybe there is an easier solution.
> I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
>
> Is it possible to use ffmpeg to create images on scene changes?
> Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
>
> If you have any idea, I would be very happy.

It should be not too hard to implement a scene detection filter in
libavfilter.

Regards.
--
ffmpeg-user random tip #27
Use the source Luke! If RTFM fails to help you, consider RTFS.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[H. Vidal, Jr.]

Stefano Sabatini wrote:

> On date Thursday 2009-10-01 08:25:39 -0700, Fori Pepe encoded:
>> Hi,
>>
>> I would like to generate images from videos at every scene changes.
>> My present solution is:
>> - ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
>> - lav2yuv -S list.el destination_video.avi
>> It generates a scene list text file with time codes that I use later to creating images.
>>
>> I was wondering maybe there is an easier solution.
>> I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
>>
>> Is it possible to use ffmpeg to create images on scene changes?
>> Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
>>
>> If you have any idea, I would be very happy.
>
> It should be not too hard to implement a scene detection filter in
> libavfilter.

Perhaps a clarification is in order: when FP says
'at every scene change,' is the idea to generate a
time code or an event or an automatic cut when the
video 'content' changes? Or is this somehow encoded in
the video stream as control/time data?

Pardon the dumb question, but what is the goal? Is this
an artificial vision kind of problem or a 'see-the-marker-
and-cut/edit' problem?

h
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Stefano Sabatini]

On date Friday 2009-10-02 20:03:46 -0400, hvidal encoded:

> Stefano Sabatini wrote:
>> On date Thursday 2009-10-01 08:25:39 -0700, Fori Pepe encoded:
>>> Hi,
>>>
>>> I would like to generate images from videos at every scene changes.
>>> My present solution is:
>>> - ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
>>> - lav2yuv -S list.el destination_video.avi
>>> It generates a scene list text file with time codes that I use later to creating images.
>>>
>>> I was wondering maybe there is an easier solution.
>>> I have browsed the net to get a better solution (ffmpeg, sc_threshold etc.), but I couldn't find any other just my present lav2yus solution.
>>>
>>> Is it possible to use ffmpeg to create images on scene changes?
>>> Or is it possible to use ffmpeg to create a text file with the time codes of scene changes?
>>>
>>> If you have any idea, I would be very happy.
>>
>> It should be not too hard to implement a scene detection filter in
>> libavfilter.
>
> Perhaps a clarification is in order: when FP says
> 'at every scene change,' is the idea to generate a
> time code or an event or an automatic cut when the
> video 'content' changes? Or is this somehow encoded in
> the video stream as control/time data?

Cut-scenes are usually not hardcoded in the stream at least not
directly, but some encoders may decide for example to use an I-frame
when they detect a scene change, for improve the overall compression
ratio.

> Pardon the dumb question, but what is the goal? Is this
> an artificial vision kind of problem or a 'see-the-marker-
> and-cut/edit' problem?

You could store the various first frames of each scene in order to get
a sort of photographic storyline of a movie, or you could store the
timestamps somewhere in order to use that to automatically cut every
movie 'shot' with a video editor.

Both options should be possible with FFmpeg+libavfilter.

Regards.
--
ffmpeg-user random tip #23
FFmpeg FAQs (and answers):
http://ffmpeg.mplayerhq.hu/faq.html
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Leonardo Boselli]

Il 2 Oct 2009 alle 18:05 Bouke immise in rete
> > Is it possible to use ffmpeg to create images on scene changes? Or
> > is it possible to use ffmpeg to create a text file with the time
> > codes of scene changes?
> > If you have any idea, I would be very happy.
> No idea with FFmpeg, but a couple of years ago i had a similar need,
> and i've created this:
> http://www.videotoolshed.com/?page=products&pID=13

If you have a DV file whith timecode could joust have a piece of
software that split the file on each timecode jump ??? --
Leonardo Boselli
Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
tel [phone] cell [phone] fax [phone]
http://www.dicea.unifi.it/~leo

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[NokNok Developer]

I would think outputting an ".idx" file, with the TS location for each
scene change as well as a specific >out message line would be a better
solution?

this would allow multiple scenarios for use, as everyone has different
needs and could apply the solution to their need easily with just having
this information.

(1) grab a still 5 seconds > TS value
(2) splice video at exact spot
(3) tag as chapter in MP4

One could monitor the output of ffmpeg for SCENECHANGE event with the
Timestamp, or one could look for the idx file as it being written.  
Allows for both inline and out of process manipulation then?

Just 2 thoughts

Shawn



Leonardo Boselli wrote:

> Il 2 Oct 2009 alle 18:05 Bouke immise in rete
>  
>>> Is it possible to use ffmpeg to create images on scene changes? Or
>>> is it possible to use ffmpeg to create a text file with the time
>>> codes of scene changes?
>>> If you have any idea, I would be very happy.
>>>      
>> No idea with FFmpeg, but a couple of years ago i had a similar need,
>> and i've created this:
>> http://www.videotoolshed.com/?page=products&pID=13
>>    
>
> If you have a DV file whith timecode could joust have a piece of
> software that split the file on each timecode jump ??? --
> Leonardo Boselli
> Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
> Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
> tel [phone] cell [phone] fax [phone]
> http://www.dicea.unifi.it/~leo
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user
>  

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Bouke-3]

----- Original Message -----
From: "Leonardo Boselli" <[hidden email]>
To: "Bouke" <[hidden email]>; "FFmpeg user questions and RTFMs"
<[hidden email]>
Sent: Saturday, October 03, 2009 4:27 PM
Subject: Re: [FFmpeg-user] Scene detection

> Il 2 Oct 2009 alle 18:05 Bouke immise in rete
>> > Is it possible to use ffmpeg to create images on scene changes? Or
>> > is it possible to use ffmpeg to create a text file with the time
>> > codes of scene changes?
>> > If you have any idea, I would be very happy.
>> No idea with FFmpeg, but a couple of years ago i had a similar need,
>> and i've created this:
>> http://www.videotoolshed.com/?page=products&pID=13
>
> If you have a DV file whith timecode could joust have a piece of
> software that split the file on each timecode jump ??? --

Not sure if this is possible with FFmpeg, but i could make something that
splits the file on each shot break.
Not sure what you mean with Timecode. QT timecode cannot have breaks. I'm
not sure how the timecode is stored in a DV file, nor if it is possible with
that.
Or are you talking about LTC on an audio channel?

Bouke

> Leonardo Boselli
> Nucleo Informatico e Telematico del Dipartimento Ingegneria Civile
> Universita` di Firenze , V. S. Marta 3 - I-50139 Firenze
> tel [phone] cell [phone] fax [phone]
> http://www.dicea.unifi.it/~leo
>
> _______________________________________________
> ffmpeg-user mailing list
> [hidden email]
> https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user 

_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Fori Pepe]

Hi,

Thanks the responses for everybody.
What I really want is ffmpeg detect the scene changes (using I-frame?), and ffmpeg create an image on these points, or just gives me a list about these time codes.

It would be the most comfortable to use it like this:
ffmpeg -i test.mpg --detect_scene_changes scenes.txt

Do you think is it possible?

> Cut-scenes are usually not hardcoded in the stream at least not
> directly, but some encoders may decide for example to use an I-frame
> when they detect a scene change, for improve the overall compression
> ratio.
>
> You could store the various first frames of each scene in order to get
> a sort of photographic storyline of a movie, or you could store the
> timestamps somewhere in order to use that to automatically cut every
> movie 'shot' with a video editor.
>
> Both options should be possible with FFmpeg+libavfilter.

> Hi,
>
> I would like to generate images from videos at every scene changes.
> My present solution is:
> - ffmpeg -i source_video -r 25 -an -vcodec mjpeg destination_video.avi
> - lav2yuv -S list.el destination_video.avi
> It generates a scene list text file with time codes that I use later to
> creating images.
>
> I was wondering maybe there is an easier solution.
> I have browsed the net to get a better solution (ffmpeg, sc_threshold
> etc.), but I couldn't find any other just my present lav2yus solution.
>
> Is it possible to use ffmpeg to create images on scene changes?
> Or is it possible to use ffmpeg to create a text file with the time codes
> of scene changes?
>
> If you have any idea, I would be very happy.

     
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[Stefano Sabatini]

On date Monday 2009-10-05 01:07:05 -0700, Fori Pepe encoded:
> Hi,
>
> Thanks the responses for everybody.
> What I really want is ffmpeg detect the scene changes (using
> I-frame?), and ffmpeg create an image on these points, or just gives
> me a list about these time codes.

Using I-frames+sc_threshold is not a reliable way to detect scene
changes, and it only works with codec supporting I-frames (for example
it wouldn't work with rawvideo).

Anyway you can use ffprobe -show_frames to detect the I-frames and
their timestamps in a multimedia stream.
 
> It would be the most comfortable to use it like this:
> ffmpeg -i test.mpg --detect_scene_changes scenes.txt
>
> Do you think is it possible?

The fake syntax for a lavfi filter (BIG WARNING, libavfilter is not
still properly integrated into FFmpeg, check the soc repo) would be
something of the kind:

ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT] sc_detect=PARAMS, process_sc_frames=PARAMS"  /dev/null

As you see you would need two filters, one detecting scene changes and
outputting the detected scene change frames to the output, and a
filter to furtherly processes the information of the frames, for
example it could print to a file the timestamp of the file or to
render as an image file the processed frames.

The sc_detect filter implementation is the tricky part, but
a naive implementation shouldn't be too hard to accomplish.

Regards.
--
ffmpeg-user random tip #6
Please follow netiquette rules while posting to ffmpeg-user:
http://linux.sgms-centre.com/misc/netiquette.php
_______________________________________________
ffmpeg-user mailing list
[hidden email]
https://lists.mplayerhq.hu/mailman/listinfo/ffmpeg-user

[lovelove]

Hi, can someone please help a desperate soul like me?

I am in the same situation as Fori Pepe

Fori Pepe wrote

I would like to export/extract images from videos at every scene changes.
Is it possible to use ffmpeg to create images on scene changes?
It would be the most comfortable to use it like this:
ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png

... i.e. export first frame of each new scene as an image. This effectively generates a storyboard.
It also would be an enormous help in identifying added and deleted scenes when comparing 2 videos.

this was the last answer in that thread:

Stefano Sabatini wrote

The fake syntax for a lavfi filter [...] would be something of the kind:

ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT] sc_detect=PARAMS, process_sc_frames=PARAMS"  /dev/null

As you see you would need two filters, one detecting scene changes and
outputting the detected scene change frames to the output, and a
filter to furtherly processes the information of the frames, for
example it could print to a file the timestamp of the file or to
render as an image file the processed frames.

The sc_detect filter implementation is the tricky part, but
a naive implementation shouldn't be too hard to accomplish.

I am really a beginner, so I don't quite know what to do with that. All I know is I have been looking for >1 year on the www, from top to bottom and could not find anything.

I would be so thankful to anybody of you here, if you could tell me what to do.
with my best regards, Christian Davideck

[Mark Himsley]

On 07/08/2011 15:16, lovelove wrote:

> Hi, can someone please help a desperate soul like me?
>
> I am in the same situation as Fori Pepe
>
>
> Fori Pepe wrote:
>>
>> I would like to export/extract images from videos at every scene changes.
>> Is it possible to use ffmpeg to create images on scene changes?
>> It would be the most comfortable to use it like this:
>> ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png
>>
>
> ... i.e. export first frame of each new scene as an image. This effectively
> generates a storyboard.
> It also would be an enormous help in identifying added and deleted scenes
> when comparing 2 videos.
>
> this was the last answer in that thread:
>
>
> Stefano Sabatini wrote:
>>
>> The fake syntax for a lavfi filter [...] would be something of the kind:
>>
>> ffmpeg -i in.avi -vfilters "[IN] split [SC_DETECT] [OUT], [SC_DETECT]
>> sc_detect=PARAMS, process_sc_frames=PARAMS"  /dev/null
>>
>> As you see you would need two filters, one detecting scene changes and
>> outputting the detected scene change frames to the output, and a
>> filter to furtherly processes the information of the frames, for
>> example it could print to a file the timestamp of the file or to
>> render as an image file the processed frames.
>>
>> The sc_detect filter implementation is the tricky part, but
>> a naive implementation shouldn't be too hard to accomplish.

I wonder if any code from the BBC R&D's open source (LGPL) Video Shot
Change Detector could be used?

http://www.bbc.co.uk/opensource/projects/shot_change/

--
Mark
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[lovelove]

Mark Himsley wrote

I wonder if any code from the BBC R&D's open source (LGPL) Video Shot
Change Detector could be used?

http://www.bbc.co.uk/opensource/projects/shot_change/

The netiquette says no smalltalk, still I want to say how grateful I am for your reply. Hope that's ok.

Here is other code I found for scene change detection:
(being coding illiterate I don't know how userful it is)

http://www.mathworks.com/products/computer-vision/demos.html?file=/products/demos/shipping/vision/videoscenechange.html
http://www.catenary.com/howto/motion.html
http://forum.doom9.org/showthread.php?p=268534#post268534

(of course I did NOT expect anyone to do coding work)

proof-of-concept:
http://www.youtube.com/watch?v=4988BhKUa9Q&NR=1
http://www.youtube.com/watch?v=4esP2nO1_kY

[Dave Rice]

On Aug 7, 2011, at 10:16 AM, lovelove wrote:

> Hi, can someone please help a desperate soul like me?
>
> I am in the same situation as Fori Pepe
>
>
> Fori Pepe wrote:
>>
>> I would like to export/extract images from videos at every scene changes.
>> Is it possible to use ffmpeg to create images on scene changes?
>> It would be the most comfortable to use it like this:
>> ffmpeg -i test.mpg --detect_scene_changes newscene[00-99].png
>>
>
> ... i.e. export first frame of each new scene as an image. This effectively
> generates a storyboard.
> It also would be an enormous help in identifying added and deleted scenes
> when comparing 2 videos.

I don't know how to do this in ffmpeg along, but you could pipe yuv4mpegpipe output from ffmpeg to yuvdiff. See: http://www.silicontrip.net/~mark/view.php/lavtools/20100906-yuvdiff

For instance:
ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - | yuvdiff -g > frame_difference_values.txt

You'll get a two column output of frame number and a number representing the difference between the current frame and the prior. Then determine a threshold for what is a scene cut, parse the output to get the frame numbers of cuts, then use those to cut the desired frames out.

Dave Rice
avpreserve.com
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[Peter B.]

Quoting Dave Rice <[hidden email]>:
> For instance:
> ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - |  
> yuvdiff -g > frame_difference_values.txt
>
> You'll get a two column output of frame number and a number  
> representing the difference between the current frame and the prior.  
> Then determine a threshold for what is a scene cut, parse the output  
> to get the frame numbers of cuts, then use those to cut the desired  
> frames out.

For getting a list of detected scene cut offsets, I can also suggest  
using "shotdetect" by Johan Mathe:
http://shotdetect.nonutc.fr/

(NOTE: The current release is quite old and contains some issues, but  
I've fixed some of them and Johan agreed to merge them back upstream -  
there's no official release containing those changes, but if you need  
them, let me know)

...you could take its output XML with the frame offset numbers, and  
use a XSLT stylesheet to generate the necessary ffmpeg commands which  
actually perform the cuts. However, that's just some implementation  
idea from the top of my head :)

Or is there any way to feed some sort of "edit list" to ffmpeg?

Regards,
Peter B.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[Stefano Sabatini]

On date Monday 2011-08-08 16:12:54 +0200, Peter B. encoded:

> Quoting Dave Rice <[hidden email]>:
> >For instance:
> >ffmpeg -i input_file.mov -pix_fmt yuv420p -f yuv4mpegpipe - |
> >yuvdiff -g > frame_difference_values.txt
> >
> >You'll get a two column output of frame number and a number
> >representing the difference between the current frame and the
> >prior. Then determine a threshold for what is a scene cut, parse
> >the output to get the frame numbers of cuts, then use those to cut
> >the desired frames out.
>
> For getting a list of detected scene cut offsets, I can also suggest
> using "shotdetect" by Johan Mathe:
> http://shotdetect.nonutc.fr/
>
> (NOTE: The current release is quite old and contains some issues,
> but I've fixed some of them and Johan agreed to merge them back
> upstream - there's no official release containing those changes, but
> if you need them, let me know)
>
> ...you could take its output XML with the frame offset numbers, and
> use a XSLT stylesheet to generate the necessary ffmpeg commands
> which actually perform the cuts. However, that's just some
> implementation idea from the top of my head :)
>
> Or is there any way to feed some sort of "edit list" to ffmpeg?

Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker. I'll hope we'll be able to find
some mechanisms for allowing users to (micro?)fund and vote on the
issues/features they want to be fixed.

On the technical side: this has been already discussed on
ffmpeg-devel, an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

For images with moving objects that's not a proper solution, and you
need motion estimation code, which is already integrated in
libavcodec, so the ideal solution would be to refactor the ME code in
libavcodec and move it to libavutil. By doing this you may also
improve the code itself, and thus improve the codecs using it at the
same time.
--
ffmpeg-user random tip #19
X11 session recording with ffmpeg:
ffmpeg -f oss -i /dev/audio -f x11grab -s $WIDTHx$HEIGHT -r 5 -i :0.0 \
    x11-session.avi
Use xdpyinfo to get WIDTH and HEIGHT values.
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[Peter B.]

Quoting Stefano Sabatini <[hidden email]>:

> On the technical side: this has been already discussed on
> ffmpeg-devel, an underkill solution may be as simple as tweaking the
> select filter and adopt a simple pixel-per-pixel average difference as
> metric.
>
> For images with moving objects that's not a proper solution, and you
> need motion estimation code, which is already integrated in
> libavcodec, so the ideal solution would be to refactor the ME code in
> libavcodec and move it to libavutil. By doing this you may also
> improve the code itself, and thus improve the codecs using it at the
> same time.

For ideas/details about how to implement a scene-cut detection, I  
would suggest looking at the code of shotdetect (Johan Mathe is really  
nice, and the license is LGPL by the way).
His approach is actually really simple and straightforward, and we've  
now been using it in our long-term archive solution for quite a while  
and could provide information based on hands-on experience about what  
to expect from that straight-forwards algorithm.

I'm mentioning this, because I think Johan's algorithm is merely a  
variation of a "pixel-per-pixel average difference" and it works  
surprisingly well - on really different kinds of content.

Pb
_______________________________________________
ffmpeg-user mailing list
[hidden email]
http://ffmpeg.org/mailman/listinfo/ffmpeg-user

[lovelove]

Mark Himsley, Dave Rice, Peter B. and Stefano Sabatini: Thank you all for your replies and useful suggestions. Sorry for the delay, I got a bit side-tracked, but here is my follow-up post in the scene change detection thread.

Stefano Sabatini wrote

Scene-detection looks like a much requested feature, please file a
feature request on the issue tracker.

I have written ticket#442: http://ffmpeg.org/trac/ffmpeg/ticket/442 and also included your ideas and links. Sorry I can't do more, my coding skills aren't even enough for "hello world" greetings :-(

Stefano Sabatini wrote

On the technical side: this has been already discussed on
ffmpeg-devel, an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

I think this would work only with motionless scenes, but could fail on e.g. a continuous shot of a crowded street. Or spinning elements ( http://www.wheeloffortuneinfo.com/WheelAnimated-White.gif ). But I think you did address this issue later on:

Stefano Sabatini wrote

For images with moving objects that's not a proper solution, and you need motion estimation code, which is already integrated in libavcodec, so the ideal solution would be to refactor the ME code in libavcodec and move it to libavutil. By doing this you may also improve the code itself, and thus improve the codecs using it at the same time.

What about the -sc_threshold parameter in FFmpeg? Isn't this actually the implementation of what we are talking here?

Peter B. wrote

For ideas/details about how to implement a scene-cut detection, I  
would suggest looking at the code of shotdetect (Johan Mathe is really  
nice, and the license is LGPL by the way).
His approach is actually really simple and straightforward, and we've  
now been using it in our long-term archive solution for quite a while  
and could provide information based on hands-on experience about what  
to expect from that straight-forwards algorithm.

good call!
In ticket#442 I also listed a couple of other open source examples which can possibly be re-used.

[lovelove]

lovelove wrote

Stefano Sabatini wrote

an underkill solution may be as simple as tweaking the
select filter and adopt a simple pixel-per-pixel average difference as
metric.

I think this would work only with motionless scenes, but could fail on e.g. a continuous shot of a crowded street. Or spinning elements ( http://www.wheeloffortuneinfo.com/WheelAnimated-White.gif ).

An alternative to the pixel-per-pixel average difference would be to detect sudden histogram changes (if that's any easier to implement). This shouldn't be fooled by the 2 examples I gave above, I think.

[lovelove]

Peter B. wrote

For getting a list of detected scene cut offsets, I can also suggest  
using "shotdetect" by Johan Mathe:
http://shotdetect.nonutc.fr/

(NOTE: The current release is quite old and contains some issues, but  
I've fixed some of them and Johan agreed to merge them back upstream -  
there's no official release containing those changes, but if you need  
them, let me know)

Thank you Peter. I tested shotdetect today and it really has some nice concepts. The graphs are really cool. Problem is, almost all input videos failed. They were either outright refused or failed to produce any meaningful output. For example, just try any video from youtube, they all fail. I got only a single video to work.

It's really a pity because I that's the only command line tool with scene change detection I found and I really quite badly wanted this to work ...

If you want to send me your improved release, I would of course be happy to test it and give feedback. I'd have some comments/suggestions, but I think they are secondary to the input issue.