Jpeg or Mjpeg decoding
smbaker
Posts: 164
Does anyone have a solution for jpeg decoding on a propeller? Specifically the application I was a looking at is decoding the MJPG stream of a network camera and displaying it on the video out. The network part can be done on the prop fairly easily, but the jpeg decoding (an MJPG stream is simply a sequence of jpeg images) is where I think the problem lies. I'm guessing this is too much computation for the prop to handle; does anyone have experience interfacing a 3rd-party jpeg decoder of some sort to the prop? We're not talking about very large images here, only about 640x480.
Scott
Scott
Comments
Scott, for a normal monitor without memory, assuming you could store the JPEG image off-chip, such as on a memory card or in cam, you'd still have to generate the video signals for the image in real-time. That either means fully decompressing the JPEG data and storing it in Propeller memory and then drawing therefrom--which as Mike observed exceeds the current Prop's HUB ram--or you'd have to continually fetch partial JPEG image data from the memory card and decompress that on the fly to constantly build/rebuild scan lines, which is a lot of data moving around and involves a lot of critical timing issues to drive a display.
However, if your display had it's own built-in memory (as the controller chips for a few small raw LCD panels do), then you'd "just" be tasked with decompressing the JPEG data and sending it to the display. So, perhaps your root question is whether the Prop could handle such decompression of the JPEG data on the "one-time" fly for a memory-equipped LCD without needing all the decompressed data to be stored in the Prop. And the answer is: I have absolutely no idea, but perhaps someone here knows or is able to speculate. For all I know (which is nothing), perhaps the standard JPEG decoding schemes requires access to the full image data in order to get started with the decompression, hopefully not but perhaps, which would render all of this moot.
Ignoring the specific JPEG particulars, for a one-chip VGA still-image solution, perhaps you could hold your breath for the Prop II at 4 or 5 bits per pixel, potentially coming in about a year. I know I am, wisely or not. Other than that, you'd need to add static ram with another chip to the current Prop, which other theads on this forum address, though such approaches obviously somewhat destroy the simplicity.
The Prop II (which is not guaranteed/promised to be released) is rumored to be designed with 256KB or perhaps even 384KB of hub memory. And, for example, 640x320x0.5 bytes per pixel (4 bits) is 153,600 total bytes. But at 4 or 5 bits per pixel, that's only 16 or 32 total screen colors (without scanline tricks, etc.), which seems well below your needs.
If I'm not mistaken, I believe that the conceiver/designer of the new chip has mentioned 5 bits per color (RGB) for a total of 15 bits per pixel (32,768 possible colors), but that's either for QVGA or is using a tiling scheme (breaking up the screen into multiple squares). I believe that he even mentioned double-buffering, so even if only QVGA but double-buffered, that would exceed 256K, so he was probably talking about a tiling scheme (anyone more clear on that?).
If it takes a year or so for the Prop II to come out, I keep greedily hoping for even more functionality, such as a megabyte of ram or more COGS or whatever, but I doubt that will be the case if the process technology stays at 180nm and the die-size doesn't increase (and power draw and cost). But even at only 256KB/385KB (and 8 COGS), a new range of applications will open up, particularly in the visual realm. I'm just worried about other chips passing the new chip by if it doesn't come out soon (or with increased functionality). However, I know that the Propeller concept has it's own unique advantages, particular simplicity, orthoganality and deterministic timing, as well as ease of interfacing that likely still give it an important role to play.
One possibility that occurs to me is that the designers, Chip and crew, have something up their collective sleeves, like a package-on-package memory option that will allow using even more memory than the built-in expanded HUB memory (perhaps even with some dedicated addressing/data/control lines for such piggy-backed memory (though that would seem unlikely or even impossible in an inexpensive package with only a hundred or so pins)). Or, we could get really pie-in-the-sky and hope for 36-bit instructions or so with increased COG ram, but let's not go there (whoops, too late!).
Anyway, a one-chip solution capable of handling still images (or even animation and video) is desparately needed, so I feel your pain. Good luck with your search.
Mike is right.
You would probably be able to do it on one of my Morpheus boards, after I release some 256 color per pixel drivers.
I am currently planning:
256x192 - will be released fairly soon
320x240
400x300
with 256 colors per pixel - which is encoded as 3 bits for Red, 3 bits for Green, 2 bits for blue - so there would be color banding.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full
Morpheus & Mem+dual Prop SBC w/ 512KB kit $119.95, 2MB memory IO board kit $89.95, both kits $189.95
www.mikronauts.com - my site 6.250MHz custom Crystals for running Propellers at 100MHz
Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller
It's been a while since I've worked on JPEG code, but the largest compute requirement is to implement the inverse discrete cosine transform (IDCT) for the decoder.· I believe it requires about 5 or 6 16-bit multiplies per pixel.· The other portions of the algorithm that take significant compute are the decoding of the variable-length codes and the inverse quantization.· However, these parts require less compute than the IDCT.
A simple thumbnail decoder can be implemented fairly easily.· This decoder would just extract the DC coefficients from the codestream, and generate a low resolution thumbnail of the image.· JPEG workes on 8x8 blocks of image data at a time.· The DC coefficient represents the average value of the 8x8 block.· An image constructed from just the DC coeffients would be one-eighth of the resolution horizontally and vertically.· A 640x480 full-res image would produce an 80x60 image from the DC coefficients.· This would fit within the memory of the Propeller, and it would only require a single 8-bit multiply per pixel.· It might be an interesting project to implement on the propeller.
Dave
As an alternative, uncompressed BMP, or TIFF images, or perhaps run-length compressed GIF (and PNG?) images do not require the math, and you may be able to work on the fly. However that won't decode or produce movie files, barring moving GIFs.
For me, the sweet spot of propeller applications is low level control. The parallelism lets hardware be monitored closely if volume of data does not get in the way. I'm working on an audio application, where the arrays are only 1D long so the memory required does not explode so quickly.
Steve