P2 NTSC/PAL video input

I want your input to guide future development.
1. What kind of applications would you do with a video input?

2. NTSC or PAL?

3. bt.601 or square pixels?

4. Output resolution and color encoding?

5. High frame rate or reduced cog count?

Comments:
3. bt.601 is 720x480 for NTSC, 720x576 for PAL,
Square pixel is 640x480 for NTSC, 768x576 for PAL
Minimum clock frequency is 234 for bt.601, 221 for sqNTSC, 266 for sqPAL.
I would favor square pixels for machine vision applications.

4. Memory usage is a big concern here.
768x576x8bpp=442,368 bytes.
720x480x8bpp=345,600 bytes.
640x360x16bpp=460,800 bytes.
384x288x16bpp=221,184 bytes.
320x240x32bpp=307,200 bytes.
320x240x24bpp=230,400 bytes.
320x240x16bpp=153,600 bytes.
YCrCb output would save a few clock cycles.

5. Capturing the monochrome video almost totally utilizes 1 cog. I estimate at least 2 additional cogs to decode color in realtime. The capture cog could stop after capturing a frame and convert the output to color.

Based on the above I think I would want 320x240 output for color to allow the possibility of 2 images in memory. 1 for capture, 1 for processing. Higher resolutions just don't make much sense except for really simple algorithms. The capture code operates at 9 instructions/pixel and getting 12 instructions/pixel requires a sysclock over 300MHz.

Here's a test image. It's NTSC, with bt.601 sampling, but with 48 additional pixels on the left side. That's to get some of the color burst. It's a single ADC pin, using the Goertzel hardware for the window filter. Sampling is line-locked, controlled by the Goertzel NCO.
dcr.png

Code is attached. It's alpha quality, beware. Horizontal sync is not optimal. Vertical sync is non-existent.
James https://github.com/SaucySoliton/

Invention is the Science of Laziness

Comments

  • That is pretty impressive!

    Frankly, I am with you on the basics for machine vision and related apps. Square pixels and a modest memory footprint should open up some interesting work.

    If it is not too much trouble, developing with the option for bt601 may have uses. People could choose. I am interested in both. At the least, those interested could learn more and roll their own.

    This on the 2a chip right?



    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • It would be neat to be able to use those cheap NTSC cameras with P2...
    Prop Info and Apps: http://www.rayslogic.com/
  • It should just be a matter of adjusting loop lengths. The bt.601 standard came from research into standards converters. Having the same sample rate and number of active pixels for NTSC and PAL is really convenient. We could make a composite to VGA or HDMI converter. Although with lower quality video and higher price than ones on the market already.

    It is a 2a chip. The 2b should be a little better if using multiple ADCs in parallel. The 2b scope filter could also be used, but it doesn't have the capability to interpolate between samples. There would be some jitter as the streamer is forced to quantize the sample time to a sysclock step. Although would 5 degrees of jitter be noticeable?

    The attached plots are from a multiburst test signal. Looks like the attenuation is not that bad. Although it would be neat to compensate it with a passive pre-emphasis circuit.
    1200 x 900 - 18K
    1200 x 900 - 25K
    James https://github.com/SaucySoliton/

    Invention is the Science of Laziness
  • rjo__rjo__ Posts: 2,027
    edited 2019-09-24 - 13:35:26
    1. Distance sensing
    2. NTSC
    3. Don't care. If non-square pixels, you can reduce the geometry with a look-up table.
    4. If you go for the highest resolution possible, you can always subsample to put an image into memory or on the screen.

    With higher resolution you can sample just part of the image... several lines for instance... to get better
    measurements.

    5. Highest frame rate possible with monochrome images.

    Your code is mostly beyond me at this point. For example, you are getting at least 169 levels of gray from a 128 sample LUT... unless some of that comes from an image initially read into the LUT in Cog0?

    It looks like you are capturing two complete fields at greater than 7 bits.
    But it also looks like you are only getting about half the data from each field?
    The good data looks really good... So, what you have is already useful.

  • rjo__rjo__ Posts: 2,027
    edited 2019-09-23 - 23:27:04
    A one wire camera that doesn't require a USB connection is really attractive:)
  • 1. If the quality ends up good enough, some sort of opensource analog video capture/conversion device would be quite interesting.
    2. ideally both, but NTSC seems more common. ideally also nonstandard signal types, such as NTSC50/PAL60 and 240p
    3. ideally adjustable horizontal res.
    4. YCbCr is fine I guess
    5. In this case, high frame rate.
  • Could we capture a 1280x960 image with this camera?
  • Circlesoft, at 8bits per pixel that's 1,228,800 bytes which will not fit in memory. Even at 4bits per pixel it's too much for memory.
  • Could we capture a 1280x960 image with this camera?

    It says "effective resolution 976x494." So 482,144 pixels. Now getting that many pixels out of NTSC video will require an 18.56MHz sample rate. That means a clock rate of 335MHz. Maybe we could reduce it to 297MHz by removing some features from the code.
    James https://github.com/SaucySoliton/

    Invention is the Science of Laziness
  • potatoheadpotatohead Posts: 9,786
    edited 2019-09-24 - 02:02:45
    Effective, on a composite signal has caveats. The 3.5Mhz color carrier, for NTSC, does limit overall ability to resolve high contrast, small details.

    I wonder if the cameras can be manually switched to monochrome? Without the color carrier, maybe those resolutions make sense.

    I have NTSC monitors that can resolve 900 in monochrome. My color one claims 700, but that is largely monochrome.

    Those are pretty awesome though, as just the camera module looks pretty cheap!

    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • rjo__ wrote: »
    1. Distance sensing
    2. NTSC
    3. Don't care. If non-square pixels, you can reduce the geometry with a look-up table.
    4. If you go for the highest resolution possible, you can always subsample to put an image into memory or on the screen.

    With higher resolution you can sample just part of the image... several lines for instance... to get better
    measurements.

    5. Highest frame rate possible with monochrome images.

    Your code is mostly beyond me at this point. For example, you are getting at least 169 levels of gray from a 128 sample LUT... unless some of that comes from an image initially read into the LUT in Cog0?

    It looks like you are capturing two complete fields at greater than 7 bits.
    But it also looks like you are only getting about half the data from each field?
    The good data looks really good... So, what you have is already useful.

    Could you post the original scenery?
    Great application!

    The LUT contents are used as weights for the incoming bits from the ADC. It was originally intended for the weights to be sine and cosine for Goertzel measurements. I'm using triangular ramps, which I tried to explain here: forums.parallax.com/discussion/comment/1476671/#Comment_1476671 The streamer steps though the LUT at a pre-programmed speed. It is acceptable and useful to step by more than one address at a time. I use the ramp up and ramp down to break the continuous ADC stream into discrete samples. There is some overlap where the weights for one sample decrease while the weights for the next sample increase. To increase the sample rate, one would ramp up and ramp down quicker. What I consider beautiful and elegant is that breaking of the ADC stream into samples can be done at any arbitrary rate. The code adjusts the sample rate to get exactly 858 samples per line.

    Since the sample rate can be adjusted to almost anything, it would be best to just sample according to the number of pixels we want. Although for color video that forces a certain minimum sampling rate, so there would be a reason to subsample by 2 or some other easy factor.

    There is no vertical sync yet. You are seeing bottom of one field, one complete field, and the top of the next field.

    Tomorrow.

    ---
    Also more options sampling at 4x the subcarrier.
    NTSC 753x480 @ 258MHz
    PAL 922x576 @ 320MHz
    James https://github.com/SaucySoliton/

    Invention is the Science of Laziness
  • potatohead wrote: »
    Effective, on a composite signal has caveats. The 3.5Mhz color carrier, for NTSC, does limit overall ability to resolve high contrast, small details.

    I wonder if the cameras can be manually switched to monochrome? Without the color carrier, maybe those resolutions make sense.

    I have NTSC monitors that can resolve 900 in monochrome. My color one claims 700, but that is largely monochrome.

    Those are pretty awesome though, as just the camera module looks pretty cheap!

    Even if the camera is color only, we could get monochrome output with a comb filter. It's just averaging a few adjacent lines, so fast to run but likely to need its own cog. I did some tests with a comb filter on the captured data. The luma output looks great. The chroma, well there's a lot cross-color rainbows on the edges.

    Or just lowpass to remove the color carrier, but then we loose a lot of resolution.
    James https://github.com/SaucySoliton/

    Invention is the Science of Laziness
  • Consider splitting the luma and chroma so that luma is high resolution and chroma is low resolution. That would give something like 640x480 luma resolution with 320x240 chroma resolution, very usable when overlaid.
  • This looks like a really interesting project. Nice one Saucy!
  • If the camera allows that, sure! That's S-video, and it works great. The modules I saw output chroma.

    Saucy is right though. The comb can deliver pretty great monochrome. A cutoff = about 320 pixels tops, and 160 if it's severe.

    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • LtechLtech Posts: 210
    edited 2019-09-24 - 07:05:55
    Remark:

    3. bt.601 is 720x480 for NTSC, 720x576 for PAL,
    Square pixel is 640x480 for NTSC, 768x576 for PAL

    It is a 4/3 square picture, all the actual camera's are 16/9 letterbox

    =1920 pixels wide and 1080 pixels high.
    We can take just a part of it for sample.

    Can't we do hdmi in, it is already digital ?
  • Ltech wrote: »
    Can't we do hdmi in, it is already digital ?

    It would be pretty tricky to get HDMI digital in at SD video resolutions, but perhaps not impossible if the streamer can first accumulate multiple bit samples into each long before transferring to hub. You could probably use the 27MHz HDMI clock as the P2 clock source and multiply by 10.

    You'd have to sample the 3 bits (R/G/B) at 270MHz precisely aligned with the incoming HDMI clock for reliable transfer, and stream all of it into HUB, then have some COG searching for a start of the 10b code of one channel to find the right offset so it can begin to convert back to RGB pixel values using 10b to 8b reverse table on the accumulated bit values over 10 samples. The table could be in LUTRAM if only 9bits are used for its index. But the biggest problems is that you'd only get 5 instructions per pixel which is not much. The FIFO might keep you fed with data and you could probably extract each 10b symbol, but in the end you need time to do a RDLUT and then write a byte back to memory - or do some writes in the horizontal blanking. Seems to me you'd need lots and lots of COGs in parallel to have any hope at all, if there even is a way.

    This would be a very challenging project to attempt. Getting the sample timing nicely phase aligned with the data could be an issue too.
  • evanhevanh Posts: 7,922
    edited 2019-09-24 - 10:58:04
    The much reduced wattage of the revB silicon means that this no longer is putting a cap on max usable clock speed. And stability also appears improved.

    So, if you have a need, current estimates for a revB reliable max clock rate is 360 MHz.

    PS: I still want to test this under heavy load, so it might be revised down ...
    "We suspect that ALMA will allow us to observe this rare form of CO in many other discs.
    By doing that, we can more accurately measure their mass, and determine whether
    scientists have systematically been underestimating how much matter they contain."
  • This is what the scene is supposed to look like. Except for the dark stripe about line 385. That's an artifact from shifting the above image to its proper position in the frame.
    monitor-matrix.png
    Color decoding happens on the PC right now. It makes development much easier, especially since the P2 doesn't have enough memory for a full resolution, full color image.
    768 x 525 - 2M
    James https://github.com/SaucySoliton/

    Invention is the Science of Laziness
  • msrobotsmsrobots Posts: 2,913
    edited 2019-09-27 - 01:14:17
    well I think a lower resolution would make more sense anyways, 320x240x1 would even do it, coming from cheap backup cameras or such. OK some color would be nice say 8 bit but monochrome is more easy to manage.

    One would have HUB space for two of the buffer/cameras and still have space for other code to run.

    Anyway way cool what you are doing here,

    Mike
    I am just another Code Monkey.
    A determined coder can write COBOL programs in any language. -- Author unknown.
    Press any key to continue, any other key to quit

    The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this post are to be interpreted as described in RFC 2119.
Sign In or Register to comment.