Shop OBEX P1 Docs P2 Docs Learn Events
video (.mtv) player for small LCDs (just an idea for now) — Parallax Forums

video (.mtv) player for small LCDs (just an idea for now)

RaymanRayman Posts: 14,665
edited 2013-02-02 09:42 in Propeller 1
I've got a new 2.4" LCD for Quickstart product coming soon and I was thinking again about how to play videos on it...
I know there are others here working with these screens, so I thought I'd share my idea...

Noticed today that a little MP3 player says it plays videos in .MTV format.
Was't sure what that was, so I Googled it:
http://wiki.multimedia.cx/index.php?title=MTV

Turns out that this is a very good format (16-bit per pixel, uncompressed video).
And, there is a free converter tool out there that will take almost any video and convert it.
The tool will let you do video+MP3 audio or video only.
It reduces the frame size to manageable levels like 160x120 or 320x240.
It also reduces the frame rate to as low as 1 fps.
So, I think this tool is nearly ideal for this purpose.

MP3 audio is a bit of a problem. I do have a new VS1003b based MP3 player coming soon, but I'd really rather the audio be in .wav format for my SDPC project.

One option is to have 2 instances of fsrw going, one to read this video file and another to read the audio from a .wav file.

The read speed of fsrw is the limiting factor here, but I'm fairly confident that I can at least do 4 fps (frames per second) at 320x240 with audio...

Comments

  • NWCCTVNWCCTV Posts: 3,629
    edited 2013-01-31 17:00
    Why would you want to do only 4 FPS. Live TV is 30 FPS, so if you intend on using it to watch videos it is going to be quite jerky.
  • RaymanRayman Posts: 14,665
    edited 2013-01-31 17:04
    Unfortunately, the Prop just doesn't have the power to do things like HDTV... These little MP3 players don't have the power either...
    So, we settle...

    My old Prop video player for TV does 30 fps but with only 8bpp and 160x120 resolution and poor audio.

    So, you can trade things off if you really want higher fps...
  • Oldbitcollector (Jeff)Oldbitcollector (Jeff) Posts: 8,091
    edited 2013-01-31 17:51
    4fps.. fast enough for cartoons and animation. Go for it. Nice find!

    Jeff
  • MagIO2MagIO2 Posts: 2,243
    edited 2013-01-31 20:47
    4fps would maybe be acceptable if you would have double buffers for the display showing constant content of one while loading the next into the other buffer. But those displays only have enough RAM for one screen, so you will constantly see how the content is updated top to bottom. My opinion: This will look just too ugly!
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2013-01-31 21:06
    I have played around with video a fair bit and wrote my own encoder so I could experiment with resolution vs framerate. The limiting factor is the SD read rate. I never got it much bigger than a postage stamp before the frame rate got too jerky. The problem becomes 'hard' in that the next logical step is to think about which compression algorithm could be used that can be decompressed in real time on the propeller. A jpeg decoding algorithm or something?
  • MagIO2MagIO2 Posts: 2,243
    edited 2013-01-31 23:35
    I think saying that SD read rate is the limiting factor is only half of the truth. A full 320x240x16 image needs 153,600bytes, so HUB-RAM is also a limiting factor. I see several problems:
    SD card driver loads data from the SD-card and has to move it to HUB-RAM. There the display driver has to pick the data and send it to the display.

    So, my guess is that a special SD-card & display driver would be the best solution:
    It contains the read-only part of the SD-driver and the sender for the display. The driver loads a sector into COG-RAM and directly sends it from there to the display. For further optimization this mixed driver can be started in 2 COGs, so when one is busy with sending the data to the display the other can already load the next chunk from SD-card.

    Some years ago I already tweaked the SD-card driver to work hand in hand with the display-driver and this combi can load a full screen in 166ms. Still it was using HUB-RAM and I guess it has quite a good amount of time-usage on it's account! So, I'd guess that <100ms should be possible. Not sure whether I used FAT-part of the software or whether I stored the images in a SWAP file, which would also increase speed!

    A hardware solution would be a shift in register. The input is connected to the SD-card, the parallel out is connected to the display. Then the propeller has not to read the data at all. It can drive the shift in at full speed and after 8 bits send it to display.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2013-02-01 01:45
    The number is exactly as you say MagIO2 - 153,000 bytes for an image. So you have to get that off an SD card at that rate. Ballpark one megabit per second.

    The hub ram may or may not be a problem. On the touchburger screen we sent the data out to external ram first and then sent a whole picture out to the display in one burst to minimise flicker.

    Hardware solution would be very cunning. Send setup data to the SD card, but then change over connections with a multiplexor or something to dump data straight out. Maybe with some 74xx gates.

    I've just been reading about jpegs. Mpeg gets 3x more compression by compressing between frames. But take just individual jpegs as a starting point. Data is in groups of 64 pixels and needs to be run through a cosine transform (precalculated lookup table) and I think converted from chrominance/luminance to RGB, and then RGB 24 bit to RGB 16bit. Some of that code may already exist in various forms. There is demo C code available.

    Need a 'back of envelope' speed calculation. Say the jpeg is 1/10th the size, so the time reading off the SD card is no longer the limiting factor. How fast could you decode an 8x8 block, and then output it? It would all be in pasm, and possibly more than one cog. 10 instructions per pixel and then 10 frames a second and it still could work?
  • RaymanRayman Posts: 14,665
    edited 2013-02-01 03:52
    Prop2 may be able to decompress video... No way Prop1 can...

    Memory is not a real problem as most small screens include a video buffer.
    For bigger screens, you can use something like the SSD1963.
  • Dr_AculaDr_Acula Posts: 5,484
    edited 2013-02-01 19:37
    Yes, pity, the code for a jpeg decompressor is more than a few lines http://www.xbdev.net/image_formats/jpeg/jpeg_decoder_source/index.php

    However, that could still be useful as an essential part of an html browser - something I think the propeller is capable of running.

    Re video, 160x120 x 1 byte per pixel seems about the max for a decent framerate.

    So maybe if you give up some video resolution RRRRRR GGGGGGGG BBBBBBBB 24 bit down to RRRRRGGG GGGBBBBB 16 bit and then down to 8 bit RRGGGBB it could work?

    Or expand each pixel out to 4 pixels so it is 320x240, and then dither the colors between frames so that over 4 frames the average is correct for those 4 pixels?
  • potatoheadpotatohead Posts: 10,261
    edited 2013-02-01 21:29
    May I suggest "Ripple Draw" be considered here? For a lot of things, updating the screen in strips can convey higher frame rate motion and resolution at the cost of minor screen image tearing.

    I'll link when I am not on my phone, but a Google for that term and this site will point to some video you might find useful.

    If there is enough RAM to buffer a small strip, it is enough to display ripple draw, which would update at several times the full screen frame rate, depending on the zone or strip size.*Strips worked best with the Prop gf demo, due to how the driver and library code worked. For video, zones might make more sense, depending on how quickly memory can be written and how it can be addressed.

    Jim Bagley implemented very simple RLE type decoding on P1. For grainy video it is of limited use, but it was done.
  • RaymanRayman Posts: 14,665
    edited 2013-02-02 04:40
    baggers did do RLE in a very impressive demo. But, I think RLE doesn't really work well for 16-bit images. Maybe for cartoons...

    Even if it could work, the real advantage here is that this is a quasi-standard format with existing conversion and creation tools.

    Can't find anything on ripple draw...


    BTW: For some music videos, instructional vidoes, or cartoons, I think 1 FPS would be acceptable. Better than nothing anyway....
  • potatoheadpotatohead Posts: 10,261
    edited 2013-02-02 09:42
    1 fps with effects would be like an old slide show multimedia thing. Perfect for music videos, IMHO.

    http://forums.parallax.com/entry.php/381-Much-better-single-buffer-graphics

    Copy paste is extremely annoying on my phone. There it is, FWIW.
Sign In or Register to comment.