Here are some samples I converted using ImageMagick. They are 3 bits per color plane (stacked 2 Cogs VGA) and 4 bits per color plane (2 COGS with low bits of R2R in 1 COG and high bits in the other COG).
The 4 bits per plane assumes you run 2 VGA COGs and a 4 pole R2R DAC instead of the 2 pole R2R DAC that is used by default.
I didn't dither the images, just to illustrate the palette changes.
I included the default Prop color palette too.
Oh, to address Phil's concern, this isn't cheating, it's just a more innovative use of existing hardware. Using an off-chip VGA framebuffer *is* cheating.
Another thought... So, Ken basically wants to present on a Prop. No brainer style too, just connect battery, projector, and go!
If we did decide it made sense to augment the reference video circuits, wouldn't it make sense to just include a dead simple external RAM, and make a product of it? "Super Quick Start", or something?
I'm thinking the 640x480 space can be made useful with the standard circuit. I'm also thinking it's gonna work well for text, graphs and other abstract things, given some image formulation rules. I don't think it's going to yield images that make sense. There is a line between "eating the dog food", which is a damn good line, and presenting at a level that isn't broken in basic ways.
...too bad we aren't on TV. Very good images can be coaxed outta the standard circuit... tease, tease, tease... (A TV circuit should be on that product, if we make it.) Seriously, that's just a digression. VGA is most practical here.
Here's another five images. I'll get some more together for you. Keep in mind I'm totally game for a redesign of my style to run well in a Propeller. When I created this show I wasn't thinking about limited resources since I was using a Mac with Keynote. Given some new constraints I'd happily adapt. After all, I think the real style of a good presentation is with minimal slide data (almost opposite of what I provided) and real demonstrations.
...too bad we aren't on TV. Very good images can be coaxed outta the standard circuit... tease, tease, tease... (A TV circuit should be on that product, if we make it.) Seriously, that's just a digression. VGA is most practical here.
Yeah, yeah, I know. So much more is possible, more easily as well with composite. My thoughts are that VGA is a more commercial interface, and composite TV seems to raise questions in some customer's minds about longevity, finding a quality OEM display, etc. And it's mostly true that in an educational environment (where the PropBOE goes) there isn't much in the way of composite inputs. Even at Parallax we had to make special efforts to see that our projector supported composite.
Ultimately, the vibrance of a VGA display is much better, at least in my limited experience. Heck, we're six years into Propeller 1 and I'm still uncovering basic features. There's a lifetime of projects in Propeller 1 at my rate of acquisition.
There are only two native VGA modes: 2-color and 4-color. My thought was that each (uncompressed) line would start with a 4-byte palette, followed by 40 longs worth of pixel data, at 16 pixels per long. This works out to 164 bytes per line, or 78720 bytes for a 640x480 image. The trick is to come up with an easy-to-unpack RLE compression method for each line that will bring the total down to something that fits in hub RAM. The point is that the entire compressed image has to fit there, since it can't be streamed out of SD flash fast enough -- or can it?
Each line requires about 25uS. To stream 164 bytes out of SD in 25uS would require a bit rate of 6.56 MHz. Is that doable with SD on the Prop?
Eric Ball made another stellar template... I've got it doing thousands of colors on the standard circuit. (with various saturation levels too) That was some of my play time last weekend. Not ready to show, so I'm just talking a little.
Some of those are really interesting Ken. Doable. The big, detail photos are tough. You could transition from a text slide to a picture, full screen lower resolution, higher color density, then back to do those, I suppose.
Here is how the first slide would look if scaled to 160x120 at 4 bits per plane, and rendered at 640x480. The text would be overlaid with another overlay COG.
I think 3 bits per plane, 2 cogs would be a good compromise, since a 160x120 image fits in 21600 bytes packed.
It's clear that if you made a presentation that used graphics at low res, pixel for pixel (not zoomed), it would provide a very nice look. You would have the other COG driver handling text and such with a 64 color palette, then 12bit color images.
@pedward: I'm feeling a little better about my results over here. Tried to convert it to 32 colors with similar results.
OBC
Apparently you guys all like this challenge in some way. I'm still not clear on whether we could divide up the display and isolate a graphic in a smaller place, maybe providing 2-3 templates for some variety.
A limited markup language with 3-5 layout templates would take some of the pressure off the fonts.
OBC
Fonts saved as BMP graphics must be an expensive use of RAM, so maybe there's a savings in this model. The low-rest VGA_Text driver is more or less acceptable if it could support some graphics . . . some simple graphics.
Here are the sample slides, rendered with four-of-64 colors per line. Clearly, some render better than others, depending upon the amount of horizontal color detail in each line of the original. The streaking is obviously not acceptable -- except on college campuses.
BTW, the question I posed earlier is still open: "How fast can data be streamed by the Prop from an SD card to hub RAM?"
That's actually not all that bad, Phil, and it could provide maximum flexibility rather than a mixed graphic/text driver.
I can already envision how some improvements could be made to the way the slides are generated, making them render better in this format. Maybe I'd use a slightly larger font, less gradient in color changes, etc. But the photo rendered of the Prop Boe-Bot almost looks artistic. It's got sorta a neat look to it.
Sounds like the SD card is the trick to making this approach work right.
Imagine the possibilities here. The slides could be advanced by XBee, or people could participate in the presentation by wearing a Polar heart rate monitor (switching the display to a simple text with bargraph, no graphics). The idea could really turn a presentation upside down. I mean heck, how many of us have been dragged through traditional PPT slide shows when it's the demo you're really after, and the ability to get answers to your questions. Nobody wants to hear a sales guy talk about how great their company is, though there are a few slides that show that as well (Propeller progression path, Parallax business, etc).
The reason I'm so interested in the SD card data rate is that, if it's fast enough, I wouldn't have to run-length encode the data, and the entire slide wouldn't have to fit in RAM at once. But, in either case, yes, the SD card is key to holding the entire presentation.
I think it can be made better, too by having the conversion software sense horizontal zones that could use different palettes -- sort of like what Bill suggested, but not on fixed boundaries. That would eliminate the streaking.
It's around 240KB/s IIRC, based on Kye's benchmark program.
With a qSPI FLASH that has 8MB of storage, you can move a lot more data faster. Steve (Jazzed) has a good idea, since we've been talking about it in context to PropGCC.
I've got some samples of 512Kbit SPI RAM coming too, which would mean 512KBytes of RAM with an 8bit wide interface, 64 bytes transfers at a time. The same board multiplexes 2 of the qSPI flash chips for 8bit wide SPI. Based on what I've seen, they can read 20MB/s if you use the counter to clock the data.
PropGCC has some great, semi-mature kernel code for doing a lot of this already. And they have SPASM drivers working with the C code (VGA demo is one).
Interesting that pedward points to GCC as a mature solution for running the project. I kinda doubt Phil would go along for that experience. David Betz has also suggested GCC for the project.
You know, I'd consider breaking some rules about using a standard PropBOE by adding a qSPI flash for storage, but only after we know it's just not possible with SD. I only have opinions about this because I'm kinda stubborn, and I'd like people to be able to duplicate our efforts with stock hardware if at all possible.
Those look good, Phil. I guess you still need RLE to make them fit into hub ram, or settle for something like 400x300, but Ken's sample slides certainly lend themselves to working well with RLE.
One possible improvement might be to incorporate error diffusion, but that might worsen the RLE ?
I'm using error diffusion vertically, between pixels in the same column, but it does not spread horizontally. That would not be difficult to add, though.
My next plan of attack will be to replace one of the four colors with a new one, every 16 pixels. This fits in a byte handily, since two bits can indicate which color to replace, with the other six indicating the new color. This would increase the uncompressed image size to 97.4 kB, though, and make RLE a lot harder. The required data rate streaming from SD would be 8 Mbaud. The SD SPI interface is capable of that, and I think I can coax 10 Mbaud out of the Prop, since I've done it already with the PropCAM. But I'm not yet familiar with the SD interface, and any overhead could be a deal killer.
In response to hub<>SD card speed, what I do know is that I could successfully record 4000 lines a second of tsl3301 line scan data to SD card, with each line having 128 bytes of grey scale pixels (shared with video buffer)... so thats about 500kB/sec or ~ 4 Mbaud. While that wasn't topping out (I seem to recall 2MB/sec was what the benchmark showed), there was a slight write delay every now and then, effectively causing a couple of lines to be dropped, when viewing the recorded "video" on a PC.
Read rate is either half or double the speed, need to cross check which.
Most newer SD cards can do qSPI, they just have *a lot* more overhead to get them to spit out data than flash does. I don't know if the FSRW driver implements qSPI or not.
First off I thought no way is it possible to do this with stock Prop hardware due to limitations with RAM and the video DAC. However if raw data can be read directly from the SD fast enough then it might also be possible to interleave alternate frames to create blends of colors. One of my early video projects would flip every alternate frame and provided the refresh rate was high enough it allowed the colors to be blended. In another project I treated the SD card as virtual memory and created the file access layer on top of that. By reserving a contiguous area of the SD memory you can use low-level drivers without any further impediments to access the display data for presentation. Mind you, I haven't even sat down to do any calculations yet but I have had a second or two to blab about it
That's kinda what I was thinking, too: set aside an unformatted partition on the SD card for the image data and just use the files in the formatted partition to point to the data. That way, you don't have to worry about clusters and blocks and all that messy file system stuff.
An 8bpp, 640x480 image updating 60 times per seconds needs a 18 MB/s bandwidth.
Even a low quality image like 4bpp, 320x200 needs a 2 MB/s bandwidth.
Then what about wearing out the SD card? Each byte will be read 60 times per second, that's 3600 reads per minute!
The standard VGA driver does provide horizontal and vertical stretching... So, I'll have to look at the trade off between covering a small portion of the screen at 640x480
(maybe mostly with 1-bit graphics) versus covering more of the screen at 320x240 (maybe mostly with 2-bit graphics).
Also, I think animations and transistions may not add much to the memory requirements and therefore be a way to improve the experience with low overhead...
Ahle2,
SD cards, and FLASH memory in general, is only worn out when write to it. There should be no problem refreshing a display from it. Don't forget, many MCUs execute their code from FLASH.
Scrub reading from the SD as SPI mode is too slow for anything useful although a single cog could read raw data from the SD and output either through WAITVID or directly but only in low-res mode.
I've also looked at running serial RAM in parallel before but the SST26VF016 devices that Rayman uses could be used if we use 2 in parallel to get 8-bits and basically bypass any Prop video hardware and just run the 8 I/O on the same 8 I/O of the Prop that's used for VGA output. Once raw video data is loaded for the presentation all that's needed is a fast clock which clocks the data straight out of the flash onto the 8 VGA I/O lines. So H/V synch are encoded in each byte of course and the cog would have the CTR setup to output the desired pixel clock rate which could very easily be set for 640x480. This would still only give us 64 colors but by storing the raw data as a 2 frame set and using the alternate frame blending we can have a far greater color range. The 2 frame set might take around 660kB but even the 16Mbit device could easily accommodate 6 frames in uncompressed raw video data. This is similar to what I have done before years ago with simple PLDs as counters and SRAM setup with raw VGA data. However now the serial clock of the SPI devices does away with external address counters.
I'm assuming here that there is no need to latch the output data from the flash if the data changes cleanly from clock to clock. So this would be extra hardware (2 tiny 8-pin chips) which would use the existing VGA I/O lines plus an extra clock line and a chip enable. The Prop would have to issue a high-speed read command at the start of every 1 or 2 frames and somehow not adversely affect the VGA monitor. An advantage of this external hardware is that you can still run the Prop in it's regular VGA mode if desired and perhaps even overlay this onto the flash output using resistors to couple the outputs perhaps. I think I have some devices here so I will hook them up and try this idea out, it's a very cheap and easy way to introduce some background full VGA graphics without impacting on the Prop's resources very much. At $3 for 2 chips this is very reasonable,
There would be a trick to loading these chips up as this would affect the monitor but you can run in SPI mode for writing new data so it is possible to have the synch lines pulled low while this is done. This whole idea may be full of holes but it's not too hard to try out and see plus I could still add an 8-bit transparent latch if needed.
Using external RAM to hold images might be very useful in tandem with the onboard Prop resources. When a Prop is just doing text, or some basic shapes without a lot of color detail, plenty can be done on a 640x480 screen. Why not combined the two as mentioned above? Have the presentation images setup in external RAM to be streamed in, then just overlay text and symbols / graphs and such with the Propeller itself? A presentation with a few images in it to break things up would then be possible, using the SD card to hold the overall data, images on an external RAM, text and such moved from SD to HUB where needed.
(Parallax Presenter = great product name, BTW Anyone finishing a project, could go all Parallax and present it...)
Just stuffing external RAM through a VGA is not something distinctive, IMHO. Now, Props can do it along with doing a whole bunch of other things, which is. Overlay highlights that, perhaps adding sound would further emphasize that fact.
We have 32k of buffer if we get rid of spin. This can give 320x200x16 colors
To do this, we need a code which will execute in cogs only.
On a propeller demo board we have p0-p8 free; 4 pins will be used for SD, 4 pins still free for intercog communication.
It seems all which was possible on Atari ST (it has 32k framebuffer), should be possible with propeller except 64 colors instead of 512
To do this, we need a dumb display driver (get pixels from hub ram and display them), keyboard driver to control the presentation, SD driver to read bulk data from sd to hub. All these PASM procedures can be loaded to cogs with initial spin code, and then we need to stop spin interpreter cog and use all hub ram for buffer.
Eventually these display cogs can contain color maps in its memory. Dumb display code is short. Color maps can be loaded to hub ram before picture, then passed to cogs.
After redoing my calculations, I realize that streaming out of SD is simply not feasible. In the simplest case, each line will require 164 bytes (2 bits per pixel, plus 4 bytes for the line palette). That's 1312 bits that would have to be read in 31.7 uS, which equates to 41.3 MHz for the SD clock. So anything I do will have to rely on compression, in order to make the whole image fit in hub RAM.
It may also be worth looking into a DjVu-like image format. DjVu uses a low resolution background image, low resolution foreground, and a high-resolution mask layer.
Comments
The 4 bits per plane assumes you run 2 VGA COGs and a 4 pole R2R DAC instead of the 2 pole R2R DAC that is used by default.
I didn't dither the images, just to illustrate the palette changes.
I included the default Prop color palette too.
Oh, to address Phil's concern, this isn't cheating, it's just a more innovative use of existing hardware. Using an off-chip VGA framebuffer *is* cheating.
Original:
64 color:
512 color:
4096 color:
Full color:
64 color:
512 color:
4096 color:
If we did decide it made sense to augment the reference video circuits, wouldn't it make sense to just include a dead simple external RAM, and make a product of it? "Super Quick Start", or something?
I'm thinking the 640x480 space can be made useful with the standard circuit. I'm also thinking it's gonna work well for text, graphs and other abstract things, given some image formulation rules. I don't think it's going to yield images that make sense. There is a line between "eating the dog food", which is a damn good line, and presenting at a level that isn't broken in basic ways.
...too bad we aren't on TV. Very good images can be coaxed outta the standard circuit... tease, tease, tease... (A TV circuit should be on that product, if we make it.) Seriously, that's just a digression. VGA is most practical here.
Ken Gracey
Yeah, yeah, I know. So much more is possible, more easily as well with composite. My thoughts are that VGA is a more commercial interface, and composite TV seems to raise questions in some customer's minds about longevity, finding a quality OEM display, etc. And it's mostly true that in an educational environment (where the PropBOE goes) there isn't much in the way of composite inputs. Even at Parallax we had to make special efforts to see that our projector supported composite.
Ultimately, the vibrance of a VGA display is much better, at least in my limited experience. Heck, we're six years into Propeller 1 and I'm still uncovering basic features. There's a lifetime of projects in Propeller 1 at my rate of acquisition.
Each line requires about 25uS. To stream 164 bytes out of SD in 25uS would require a bit rate of 6.56 MHz. Is that doable with SD on the Prop?
-Phil
Some of those are really interesting Ken. Doable. The big, detail photos are tough. You could transition from a text slide to a picture, full screen lower resolution, higher color density, then back to do those, I suppose.
I think 3 bits per plane, 2 cogs would be a good compromise, since a 160x120 image fits in 21600 bytes packed.
It's clear that if you made a presentation that used graphics at low res, pixel for pixel (not zoomed), it would provide a very nice look. You would have the other COG driver handling text and such with a 64 color palette, then 12bit color images.
4096 colors, 4 bits per plane, 28800 bytes
512 colors, 3 bits per plane, 21600 bytes:
BTW, how did you get the screenshots?
OBC
Apparently you guys all like this challenge in some way. I'm still not clear on whether we could divide up the display and isolate a graphic in a smaller place, maybe providing 2-3 templates for some variety.
OBC
Fonts saved as BMP graphics must be an expensive use of RAM, so maybe there's a savings in this model. The low-rest VGA_Text driver is more or less acceptable if it could support some graphics . . . some simple graphics.
Ken
BTW, the question I posed earlier is still open: "How fast can data be streamed by the Prop from an SD card to hub RAM?"
-Phil
I can already envision how some improvements could be made to the way the slides are generated, making them render better in this format. Maybe I'd use a slightly larger font, less gradient in color changes, etc. But the photo rendered of the Prop Boe-Bot almost looks artistic. It's got sorta a neat look to it.
Sounds like the SD card is the trick to making this approach work right.
Imagine the possibilities here. The slides could be advanced by XBee, or people could participate in the presentation by wearing a Polar heart rate monitor (switching the display to a simple text with bargraph, no graphics). The idea could really turn a presentation upside down. I mean heck, how many of us have been dragged through traditional PPT slide shows when it's the demo you're really after, and the ability to get answers to your questions. Nobody wants to hear a sales guy talk about how great their company is, though there are a few slides that show that as well (Propeller progression path, Parallax business, etc).
The reason I'm so interested in the SD card data rate is that, if it's fast enough, I wouldn't have to run-length encode the data, and the entire slide wouldn't have to fit in RAM at once. But, in either case, yes, the SD card is key to holding the entire presentation.
I think it can be made better, too by having the conversion software sense horizontal zones that could use different palettes -- sort of like what Bill suggested, but not on fixed boundaries. That would eliminate the streaking.
-Phil
With a qSPI FLASH that has 8MB of storage, you can move a lot more data faster. Steve (Jazzed) has a good idea, since we've been talking about it in context to PropGCC.
I've got some samples of 512Kbit SPI RAM coming too, which would mean 512KBytes of RAM with an 8bit wide interface, 64 bytes transfers at a time. The same board multiplexes 2 of the qSPI flash chips for 8bit wide SPI. Based on what I've seen, they can read 20MB/s if you use the counter to clock the data.
PropGCC has some great, semi-mature kernel code for doing a lot of this already. And they have SPASM drivers working with the C code (VGA demo is one).
You know, I'd consider breaking some rules about using a standard PropBOE by adding a qSPI flash for storage, but only after we know it's just not possible with SD. I only have opinions about this because I'm kinda stubborn, and I'd like people to be able to duplicate our efforts with stock hardware if at all possible.
One possible improvement might be to incorporate error diffusion, but that might worsen the RLE ?
My next plan of attack will be to replace one of the four colors with a new one, every 16 pixels. This fits in a byte handily, since two bits can indicate which color to replace, with the other six indicating the new color. This would increase the uncompressed image size to 97.4 kB, though, and make RLE a lot harder. The required data rate streaming from SD would be 8 Mbaud. The SD SPI interface is capable of that, and I think I can coax 10 Mbaud out of the Prop, since I've done it already with the PropCAM. But I'm not yet familiar with the SD interface, and any overhead could be a deal killer.
-Phil
Read rate is either half or double the speed, need to cross check which.
-Phil
Even a low quality image like 4bpp, 320x200 needs a 2 MB/s bandwidth.
Then what about wearing out the SD card? Each byte will be read 60 times per second, that's 3600 reads per minute!
/Johannes
The standard VGA driver does provide horizontal and vertical stretching... So, I'll have to look at the trade off between covering a small portion of the screen at 640x480
(maybe mostly with 1-bit graphics) versus covering more of the screen at 320x240 (maybe mostly with 2-bit graphics).
Also, I think animations and transistions may not add much to the memory requirements and therefore be a way to improve the experience with low overhead...
SD cards, and FLASH memory in general, is only worn out when write to it. There should be no problem refreshing a display from it. Don't forget, many MCUs execute their code from FLASH.
I've also looked at running serial RAM in parallel before but the SST26VF016 devices that Rayman uses could be used if we use 2 in parallel to get 8-bits and basically bypass any Prop video hardware and just run the 8 I/O on the same 8 I/O of the Prop that's used for VGA output. Once raw video data is loaded for the presentation all that's needed is a fast clock which clocks the data straight out of the flash onto the 8 VGA I/O lines. So H/V synch are encoded in each byte of course and the cog would have the CTR setup to output the desired pixel clock rate which could very easily be set for 640x480. This would still only give us 64 colors but by storing the raw data as a 2 frame set and using the alternate frame blending we can have a far greater color range. The 2 frame set might take around 660kB but even the 16Mbit device could easily accommodate 6 frames in uncompressed raw video data. This is similar to what I have done before years ago with simple PLDs as counters and SRAM setup with raw VGA data. However now the serial clock of the SPI devices does away with external address counters.
I'm assuming here that there is no need to latch the output data from the flash if the data changes cleanly from clock to clock. So this would be extra hardware (2 tiny 8-pin chips) which would use the existing VGA I/O lines plus an extra clock line and a chip enable. The Prop would have to issue a high-speed read command at the start of every 1 or 2 frames and somehow not adversely affect the VGA monitor. An advantage of this external hardware is that you can still run the Prop in it's regular VGA mode if desired and perhaps even overlay this onto the flash output using resistors to couple the outputs perhaps. I think I have some devices here so I will hook them up and try this idea out, it's a very cheap and easy way to introduce some background full VGA graphics without impacting on the Prop's resources very much. At $3 for 2 chips this is very reasonable,
There would be a trick to loading these chips up as this would affect the monitor but you can run in SPI mode for writing new data so it is possible to have the synch lines pulled low while this is done. This whole idea may be full of holes but it's not too hard to try out and see plus I could still add an 8-bit transparent latch if needed.
Using external RAM to hold images might be very useful in tandem with the onboard Prop resources. When a Prop is just doing text, or some basic shapes without a lot of color detail, plenty can be done on a 640x480 screen. Why not combined the two as mentioned above? Have the presentation images setup in external RAM to be streamed in, then just overlay text and symbols / graphs and such with the Propeller itself? A presentation with a few images in it to break things up would then be possible, using the SD card to hold the overall data, images on an external RAM, text and such moved from SD to HUB where needed.
(Parallax Presenter = great product name, BTW Anyone finishing a project, could go all Parallax and present it...)
Just stuffing external RAM through a VGA is not something distinctive, IMHO. Now, Props can do it along with doing a whole bunch of other things, which is. Overlay highlights that, perhaps adding sound would further emphasize that fact.
To do this, we need a code which will execute in cogs only.
On a propeller demo board we have p0-p8 free; 4 pins will be used for SD, 4 pins still free for intercog communication.
It seems all which was possible on Atari ST (it has 32k framebuffer), should be possible with propeller except 64 colors instead of 512
To do this, we need a dumb display driver (get pixels from hub ram and display them), keyboard driver to control the presentation, SD driver to read bulk data from sd to hub. All these PASM procedures can be loaded to cogs with initial spin code, and then we need to stop spin interpreter cog and use all hub ram for buffer.
Eventually these display cogs can contain color maps in its memory. Dumb display code is short. Color maps can be loaded to hub ram before picture, then passed to cogs.
If i only have some more free time...
-Phil