Converting the color decoding to PASM2 worked really well. It can output 32 fps at 320x240. I added a VGA ouput driver to observe the output at full FPS. USB video output is limited to 6.5 to 7 fps by the USB bandwidth available. There are some 4:2:0 sampling modes available for UVC, but that would only boost it to 9 fps. Probably should just skip that and go right to JPEG.
Flexspin only?
Also included: A VGA driver that converts 320x240 UYVY to 640x480 RGB. The video generation is completely handled by interrupts, allowing most of the cog cycles to go to color space conversion. The hardware colorspace converter treats the inputs as unsigned to no-go using it for YUV to RGB.
I noticed something weird where the video gets much noisier when the clock is in the 330-340 MHz range. 350 MHz gives a great picture, but I would not run that fast in any commercial application.
Using the FFMPEG vectorscope to check the output I just can't seem to get it perfect. R and Cy are the most off, but if I rotate everything then Yl and B will move away from their marks. Could be some mismatch in ffplay or uvc. Or it could be the signal generator.
I thought it would work too until I tried it. I think I had to fight this issue with my SSB transmitter. Maybe I could do like the transmitter and push the YUV data into the SETCY, SETCI, SETCQ. Having to write 3 registers would skew the RGB signals.
The CSC does allow signed coefficients, a and c.
R = aY + cCr
If Cr=$90 (+16) then it should increase R.
If Cr=$70 (-16) then it should decrease R to be less than a*Y.
The Cr (and Cb) terms must be signed. RGB to YUV works because RGB >= 0.
Also the output values do need to be clamped within [0-255]. I tried to skip that and got rainbow sparkles.
That's sounding really neat Saucy. The little image screenshot you posted above almost gives it a retro videotape vibe but I imagine it's more grainy than that in real life. Videotape was pretty soft back in the day. I'll have to try to hook this up to some composite source and try to take a look. I'm really lacking those sources here except for an old DVD player perhaps.
@SaucySoliton said:
I thought it would work too until I tried it. I think I had to fight this issue with my SSB transmitter. Maybe I could do like the transmitter and push the YUV data into the SETCY, SETCI, SETCQ. Having to write 3 registers would skew the RGB signals.
The CSC does allow signed coefficients, a and c.
R = aY + cCr
If Cr=$90 (+16) then it should increase R.
If Cr=$70 (-16) then it should decrease R to be less than a*Y.
The Cr (and Cb) terms must be signed. RGB to YUV works because RGB >= 0.
That's not a problem, you can add an offset to the signal with the low byte of CY/CI/CQ. (the CSC is of course active even during blanking, so the idle SETDACS value needs to be $00_80_80_00) The actual problem, I just found out, is that the coefficient range isn't enough. Since for R/B, there needs to be a coefficient larger than 1, but for G, negative coefficients are needed. But the signed/unsigned switch is global. Maybe the same DAC range switching trick used to get around the issues with the NTSC mode can be used to get the coeffs all below 1. But that will introduce nasty banding.
Also the output values do need to be clamped within [0-255]. I tried to skip that and got rainbow sparkles.
It takes about 64 clock cycles to compress 1 byte of data. (with very well optimized PASM code) So at 320MHz we can expect an input data rate of about 5MB/sec per cog. A very nice performance boost over the 1MB/sec USB connection.
320x240 UYVY (16 bpp) 32 fps.
320x240 RGB (24 bpp) 21 fps with output downsampled to 4:2:2. USB Video Class says it requires 4:2:2 sampling. I haven't tried anything else yet. I'm not sending RGB to UVC yet. I felt it was necessary to update the code from @ChrisGadd to use 32 bit pointers. I think the USB objects would work fine if their buffers are in the first 64k of memory, but flexspin seemed not to do that if I included image data from a file. I also wanted to experiment with 1023 byte isochronous packets.
@rogloh said:
That's sounding really neat Saucy. The little image screenshot you posted above almost gives it a retro videotape vibe but I imagine it's more grainy than that in real life. Videotape was pretty soft back in the day. I'll have to try to hook this up to some composite source and try to take a look. I'm really lacking those sources here except for an old DVD player perhaps.
Thanks!
The main artifact that I see in the image there is horizontal banding. That is caused by the DC restore and automatic gain control. It seemed like no matter how good of a filter I used to measure the sync and back porch amplitude, the banding was still there. The fix was to average over time as well. This version has filters for DC level, luma gain, and chroma gain. I had to turn off the chroma phase filter after it failed due to completely?? unrelated changes. It was a rather annoying problem to tackle. The noise is less noticeable when the video is displayed at 30 fps.
Wow 32fps seems pretty decent @SaucySoliton. Are you doing parallel Cog JPEG compression? How many JPEG compression Cogs were needed for that and what's the approximate output bitrate over USB?
Comments
Or I think somebody mentioned doing text with font that is compressed…
Converting the color decoding to PASM2 worked really well. It can output 32 fps at 320x240. I added a VGA ouput driver to observe the output at full FPS. USB video output is limited to 6.5 to 7 fps by the USB bandwidth available. There are some 4:2:0 sampling modes available for UVC, but that would only boost it to 9 fps. Probably should just skip that and go right to JPEG.
Flexspin only?
Also included: A VGA driver that converts 320x240 UYVY to 640x480 RGB. The video generation is completely handled by interrupts, allowing most of the cog cycles to go to color space conversion. The hardware colorspace converter treats the inputs as unsigned to no-go using it for YUV to RGB.
I noticed something weird where the video gets much noisier when the clock is in the 330-340 MHz range. 350 MHz gives a great picture, but I would not run that fast in any commercial application.
Using the FFMPEG vectorscope to check the output I just can't seem to get it perfect. R and Cy are the most off, but if I rotate everything then Yl and B will move away from their marks. Could be some mismatch in ffplay or uvc. Or it could be the signal generator.
Images look pretty good though.
It really should work. Are your YUV bytes not unsigned to begin with? If not, you can just XOR with $80 to bias them.
I thought it would work too until I tried it. I think I had to fight this issue with my SSB transmitter. Maybe I could do like the transmitter and push the YUV data into the SETCY, SETCI, SETCQ. Having to write 3 registers would skew the RGB signals.
The CSC does allow signed coefficients, a and c.
R = aY + cCr
If Cr=$90 (+16) then it should increase R.
If Cr=$70 (-16) then it should decrease R to be less than a*Y.
The Cr (and Cb) terms must be signed. RGB to YUV works because RGB >= 0.
Also the output values do need to be clamped within [0-255]. I tried to skip that and got rainbow sparkles.
That's sounding really neat Saucy. The little image screenshot you posted above almost gives it a retro videotape vibe but I imagine it's more grainy than that in real life. Videotape was pretty soft back in the day. I'll have to try to hook this up to some composite source and try to take a look. I'm really lacking those sources here except for an old DVD player perhaps.
That's not a problem, you can add an offset to the signal with the low byte of CY/CI/CQ. (the CSC is of course active even during blanking, so the idle SETDACS value needs to be $00_80_80_00) The actual problem, I just found out, is that the coefficient range isn't enough. Since for R/B, there needs to be a coefficient larger than 1, but for G, negative coefficients are needed. But the signed/unsigned switch is global. Maybe the same DAC range switching trick used to get around the issues with the NTSC mode can be used to get the coeffs all below 1. But that will introduce nasty banding.
That, too.
Yes, if the higher voltage DAC is used, it's possible to do YUV->RGB with the hardware. Of dubious usefulness overall due to lack of clamp logic.
Now with JPEG compression!
It takes about 64 clock cycles to compress 1 byte of data. (with very well optimized PASM code) So at 320MHz we can expect an input data rate of about 5MB/sec per cog. A very nice performance boost over the 1MB/sec USB connection.
320x240 UYVY (16 bpp) 32 fps.
320x240 RGB (24 bpp) 21 fps with output downsampled to 4:2:2. USB Video Class says it requires 4:2:2 sampling. I haven't tried anything else yet. I'm not sending RGB to UVC yet. I felt it was necessary to update the code from @ChrisGadd to use 32 bit pointers. I think the USB objects would work fine if their buffers are in the first 64k of memory, but flexspin seemed not to do that if I included image data from a file. I also wanted to experiment with 1023 byte isochronous packets.
Thanks!
The main artifact that I see in the image there is horizontal banding. That is caused by the DC restore and automatic gain control. It seemed like no matter how good of a filter I used to measure the sync and back porch amplitude, the banding was still there. The fix was to average over time as well. This version has filters for DC level, luma gain, and chroma gain. I had to turn off the chroma phase filter after it failed due to completely?? unrelated changes. It was a rather annoying problem to tackle. The noise is less noticeable when the video is displayed at 30 fps.
Wow 32fps seems pretty decent @SaucySoliton. Are you doing parallel Cog JPEG compression? How many JPEG compression Cogs were needed for that and what's the approximate output bitrate over USB?
This sounds amazing. Unfortunately I won't get to check it out until tomorrow, but sounds very impressive Saucy