Author Topic: Movie Layer  (Read 4179 times)

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Movie Layer
« on: August 06, 2015, 05:50:12 PM »
I was bound to get stumped with no information and by guessing way through.

Code: [Select]
woof = procgame.dmd.Movie().load("assets/dmd/test.mp4")
woof.width=192
woof.height=64
self.layer = procgame.dmd.MovieLayer(False,True,True,1,woof.frames)

Instantiating it incorrectly?

Code: [Select]
2015-08-06 22:45:23,808 - vpcom - INFO -     if self.movie.vc == None:
2015-08-06 22:45:23,808 - vpcom - INFO - AttributeError: 'NoneType' object has no attribute 'vc'

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #1 on: August 06, 2015, 06:57:42 PM »
 8) I just cannot help myself and had to revisit this issue.

When I had first tried I was doing it correctly passing in the movie object and not movie.frames but I had an error with an mp4. I think I really should just try and re-encode this file.
Code: [Select]
error: ..\..\..\opencv-2.4.12\modules\imgproc\src\color.cpp:3650: error: (-215) scn == 3 || scn == 4 in function cv::cvtColor
I tried with an AVI and it's working.

About the audio....it's best to split movie/audio or you can load direct from the video file?

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #2 on: August 06, 2015, 08:03:58 PM »
Nice one Mikey, grinning like Cheshire cat, LOVE IT.

The 'not' hacking has gone out of the window, really needed them in the attract. Tempted to add audio to this if it's not already there?

And scrolling text as before (Markdown), but I suppose if it's just video they could be pre-made in other soft beforehand.

dmdhelper.py genLayerFromYAML
Code: [Select]
elif ('Movie' in yamlStruct):
v = yamlStruct['Movie']
movie = dmd.Movie().load('assets\\movie\\' + value_for_key(v,'Name'))
lyrTmp = dmd.MovieLayer(movie=movie)
duration = [value_for_key(v,'Duration')]

attactYaml
Code: [Select]
    - Movie:
        Name: test.avi
        duration: 44

 :-*

MOcean

  • Moderator
  • ****
  • Posts: 822
  • Michael Ocean
    • View Profile
Re: Movie Layer
« Reply #3 on: August 06, 2015, 10:32:20 PM »
Hehe. Hack away! Especially if you can take notes so I can push the changes back into the github repo.

Triggering sound and lampshows are both supported for the attract.yaml, unless I'm confusing that with something else.

Fwiw, instead of supporting "movie" like that in the attract mode, you might be able to just refer to it as an animation, since specifying a n MP4 in the asset_list.yaml does work (though it's preloaded not streamed/decoded on the fly).

MOcean

  • Moderator
  • ****
  • Posts: 822
  • Michael Ocean
    • View Profile
Re: Movie Layer
« Reply #4 on: August 06, 2015, 10:39:05 PM »

Code: [Select]
error: ..\..\..\opencv-2.4.12\modules\imgproc\src\color.cpp:3650: error: (-215) scn == 3 || scn == 4 in function cv::cvtColor
I tried with an AVI and it's working.

About the audio....it's best to split movie/audio or you can load direct from the video file?

I'm suspicious that the libffmpeg codec might be missing, and perhaps that's what's causing the MP4 not to decode properly?  Not totally sure.

As for audio you need to split them as opencv doesn't provide the wrapper to decode the audio. You have to play the sound and video separately and I haven't done anything smart yet to sync them up. In my experiments decoding on the fly and playing back an audio file, they do gal out of sync over a long enough time. A three minute music video tends to end close to on time, but if there were lips to keep in sync it would totally be a problem the longer the video is.

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #5 on: August 06, 2015, 10:57:10 PM »
Quote
Triggering sound and lampshows are both supported for the attract.yaml, unless I'm confusing that with something else.

Couldn't find any sound calls in there, the attract module or the scriptless layer.

Quote
since specifying a n MP4 in the asset_list.yaml does work (though it's preloaded not streamed/decoded on the fly

Oh ok. Will give that a try in a min. Cheers.

Quote
I'm suspicious that the libffmpeg codec might be missing, and perhaps that's what's causing the MP4 not to decode properly?  Not totally sure.

I think I had a different error when I didn't have the codec and if that error is googled I think it's layers (rgb?) missing from the video. This is why I need to re-encode but I'll do that and try in the attract.yaml.

Quote
As for audio ....

The 45 second file seemed to be great but that's a perfect way to check on the lip sync, I'm not sure I'd use THAT long clips of anything but you never know! And all the time I've synced to .dmd on other games it's decent.

Quote
Hehe. Hack away! Especially if you can take notes so I can push the changes back into the github repo.

Will do, but my plan was not to but just build a game as quick as poss and failed. :)
Cheers

MOcean

  • Moderator
  • ****
  • Posts: 822
  • Michael Ocean
    • View Profile
Re: Movie Layer
« Reply #6 on: August 06, 2015, 11:28:14 PM »
Looks like I lied about sounds in the attract mode that's yaml-based.

Quickest (and maybe not the dirtiest) is to add a check to see if the callback value is a list, here:
https://github.com/mjocean/PyProcGameHD-SkeletonGame/blob/master/procgame/dmd/layers.py#L62

Then if so call each individual callback. Then over in attract.py add a callback for next sound if there is one in the yaml (just like is done for lampshows) so if there's both the callback would be a list of next_show and next_sound.

Alternatively a little dirtier is to not change layers.py and just define next_sound and next_both and register one of the three as the callback depending on the values of the attract.yaml.

If you don't get to it before I do, I'll add it for you.

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #7 on: August 06, 2015, 11:30:38 PM »
Ok a re-encode and adding to the assetlist.yaml worked.

Hah classic, so impatient, but at least I've been through lots of modules tonight.

 Need to try and figure out where these sound calls are though, can't get my head around how I would use the scriptless layer callback whilst running from the attract.yaml.

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #8 on: August 06, 2015, 11:33:33 PM »
Damn you beat me to it.

Quote
Looks like I lied about sounds in the attract mode that's yaml-based.

Not so much of a lie, probably dreamt :)

You would be alot quicker than me doing it but I'll have a look, like I've just mentioned, I have been sniffing around those modules a fair bit today and maybe with your pseudo thought I can figure out from that.

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #9 on: August 07, 2015, 03:04:02 AM »
MP4 Animation
I added list to a lampshow in the attract.yaml:
Code: [Select]
        lampshow: [attract_show_1, test]
Then in the attract mode added check to see if lampshow was a list
Code: [Select]
else:
(lyrTmp, duration, lampshow) = self.game.genLayerFromYAML(l)

if(lampshow is not None):

if type(lampshow) is list:
self.game.log("lampshow is a list")
self.sounds.append(lampshow[1])
self.shows.append(lampshow[0])
sl.append(lyrTmp, duration, callback=self.next_show)
else:
self.shows.append(lampshow)
self.sounds.append('')
sl.append(lyrTmp, duration, callback=self.next_show)
else:
sl.append(lyrTmp, duration)

Then added to the next show method:

Code: [Select]
def next_show(self):
self.game.sound.stop(str(self.sounds[self.sound]))
self.game.log("Attract: Playing NEXT show: %s" % self.shows[self.show])
self.game.lampctrl.play_show(self.shows[self.show],  repeat=True)
self.game.log("sasasasaas" + str(self.sounds[self.sound]))
self.game.sound.play(str(self.sounds[self.sound]))
self.show += 1
self.sound+=1
self.show = self.show % len(self.shows)
self.sound = self.sound % len(self.sounds)

Edit: Just had to remove a lot of text in this post :). Did have issues with out of sync mp4 but that was due to a program called handbrake giving me back 13Fps when my AVI file was 15Fps and I DID ask for 15FPS. Just use ffmpeg next time, sigh.

So I am sorted for now. That ^^^ audio isn't a great readable way but working ok, I'm sure you'll come up with a cleaner method. Most attracts don't always contain too much audio, but an option to disable attract audio could be nice.








« Last Edit: August 07, 2015, 03:54:25 AM by dave_h »

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #10 on: August 07, 2015, 04:42:34 AM »
Definitely bad issues pre loading MP4 with VP, avi is fine but these can't be pre-loaded?. It's fine on normal boot with python.

Rosh

  • Moderator
  • ****
  • Posts: 667
  • Josh Kugler
    • View Profile
Re: Movie Layer
« Reply #11 on: August 07, 2015, 12:42:44 PM »
the mp4s will take larges amount of memory, especially if using a larger screen size.  Ideally you want to load them right before use and then dump them when finished.  The idea was that they would primarily be used at the start of modes,  where typically the ball is being held.  If being used at times where that is not the case, then pre-loading at start of game is typically the best option.


dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #12 on: August 07, 2015, 12:56:16 PM »
Cheers Rosh.

It is actually really small 192x64 , 44 seconds @ 500kb and it was just the one file preloading. I've only really encountered the problem when pre-loading mp4 with VP, they're fine when loaded the normal way through terminal and are in sync with audio.

I've not noticed any problems when loading on the fly avi's yet, they're all in sync and load instant.

MOcean

  • Moderator
  • ****
  • Posts: 822
  • Michael Ocean
    • View Profile
Re: Movie Layer
« Reply #13 on: August 07, 2015, 10:23:23 PM »
I just pushed a new version of skeletongame that fixes a framework bug (involving slam tilt), a typo in the samplegame's asset_list, and also adds the 'sound' support feature in the provided attract mode.

dave_h

  • Wizard
  • *****
  • Posts: 185
    • View Profile
Re: Movie Layer
« Reply #14 on: August 07, 2015, 10:29:39 PM »
Nice. Updating....interested to see how you did the sound.