Shop OBEX P1 Docs P2 Docs Learn Events
P2v PropCam3d beta... — Parallax Forums

P2v PropCam3d beta...

rjo__rjo__ Posts: 2,114
edited 2016-09-22 05:03 in Propeller 2
I'm not posting pictures because at 90FPS, I'm going to need a better light source:)
The PropCam is connected to and controlled by a Propeller Activity Board(rjoPropCam_i2c.spin). I'm using the serial, snapshot mode for the PropCam.
The P2V strobes the Hsync line to the Propeller Activity Board and then captures the data, constructs the image and displays it. You can control both the exposure and video gain on the PAB. P2V7Propcam.spin is the P2 source. 3MBaud. Hit "o" to start the capture. There is a glitch in the analysis when you change settings on the PropCam when it is already sending data to the P2v. This can be fixed but it is down the list.

Because of sensitivity issues, I expect daylight operations to be somewhere in the area of 45 FPS(depending on the day:)

The next step is to integrate a second PropCam. Because the data is 3bit serial and stuffed into a byte...there is room for two cameras without changing my buffer size... which currently sets just north of 440,000 bytes!!!





Comments

  • rjo__rjo__ Posts: 2,114
    Some samples from PropCam3D (stereo). Frame rates from about 85FPS to just over 250FPS using ambient light .
    Parallel mode, 4 bits/pixel. There is a slight vertical displacement of the right and left images. So fusing these into a single image with your eyes takes a little practice.

    792 x 226 - 50K
  • rjo__rjo__ Posts: 2,114
    edited 2016-05-14 15:05
    oops...

    Uses 2 parallel cogs. The code is found in PropCam_Acq_Parallel.
    
    PropCam_Acq_Parallel
                
                mov acq_lpf,#101
                mov acq_line_ptr,##_Screen_Buffer
                cmp which_nib,#0 wz
        if_z    add acq_line_ptr,##128
                wrfast #0,acq_line_ptr
                mov sr,which_nib
                mul sr,#4
                getct PCA_timer2
                call #doVSYNC
    
                waitx ##_integration_time_plus          
    
                waitx #348  'Propcam integratiion time plus hblank=acquisition time
              
    
    acq_line    
                
                rep @end_line_ACQa,##128 'readout 
               
                mov parallel_pixel,ina 
                shr parallel_pixel,sr
                shl parallel_pixel,#4
                wfbyte parallel_pixel
        
    
    end_line_ACQa
    
                add acq_line_ptr,#320
                wrfast #0,acq_line_ptr
                waitx #304
                djnz acq_lpf ,#acq_line
    
    
    
                getct PCA_timer3
                sub PCA_timer3,PCA_timer2
                wrlong PCA_timer3,##_data_exchange
          
                     
                      ret
    
  • Did you see this module from Terasic, Rjo?

    I mention it because it probably wouldn't be that hard to get running with a P2, and they state a fairly high resolution at 60fps, and talk about two cameras. The data ends up being parallel

    I know you're kind of OK between Phil's PropCams and webcams via the PC, just thought I'd throw this one into the mix

    http://www.terasic.com.tw/cgi-bin/page/archive.pl?Language=English&CategoryNo=68&No=1011&PartNo=2
  • RaymanRayman Posts: 14,641
    edited 2016-05-27 11:46
    That thing is $99 Tubular, that's pretty outrageous...
    Also, using a MIPI camera is a little ridiculous when the MIPI bus length looks less than 1 cm and then needs to MIPI to parallel converter...

    The camera modules at techtoys.com.hk are only $19, last I looked.

    On the other hand MIPI might what all camera modules are in the future.
    Wonder if 160 MHz P2 could read MIPI directly...
  • Well I'm happy to pay you at least the $80 difference if you can make a webcam work with the P2, but lets get the HIDs working first : )

    I think main thing is to get stuff talking, then look at removing intermediate chips if possible

    Have you seen the mipi vision of how they envision stuff might be connected? Enough to keep us busy for years



  • rjo__rjo__ Posts: 2,114
    edited 2016-05-28 12:57
    I had not seen that ... thanks.

    I have a couple of the D5M's, they work as advertised. I like OmniVision. For general robotic applications, I think most of their chips work just fine. For real "vision" applications, however, the last time I looked, OmniVision had one chip that would work "fine"... has a global shutter, nice resolution, ball grid layout, etc... can't remember the name of it and I could be wrong. In general Omnivision doesn't make a global shutter chip that is in the price range I find interesting.

    On the other hand, Aptina does and Aptina is now owned by onsemi.com/, without whose help, we would be in deep trouble around here. My prejudice is to work with friends until they poop on you:) ONSEMI also has a range of really pricey chips, so getting a good working relationship with them might also put a fellow in the right place to do some serious business.

    The opportunity at the cheap end is that there isn't a chip that Aptina makes with a global shutter that finds its way onto a breakout board for less than $199... I think. And even then, the info available to a hobbyist is always just a little on the scant side. Seems like everyone that makes a board also makes a board that is difficult to use with anyone else's products but their own... can't blame them, just isn't exactly what we need.

    I'm not going to try to mention specific chips, because I don't have time to be right tonight, but there is an Aptina chip with a global shutter and really good light sensitivity, etc. etc.

    Having said all of that, I wouldn't trade any of them for a PropCam... It has just the right resolution, just the right speed and just enough light sensitivity to be almost perfect. Even if it had more resolution, most of the time I wouldn't be using it...the more you use, the more you have to calculate... and before long you have a bottleneck somewhere... it is best to minimize the data to just what you actually need... and in my case that just happens to be the PropCam.

    For use in advanced vision processing for challenging robotic application, the PropCam could easily become a gold standard... but for that to happen, people have to actually use it... educators, students, small assembly lines.

    I think the PropCam needs a little more glitz and glamor to seduce the right people, but that is about it.

    A motorized lens would be nice:)


  • I look at PropCam, and think about buying it, learning it, using it, but somewhere in the back of my mind, I think I read somewhere that no more of them are being built. when current stock is gone I will need to find/learn another, Is PropCam a live product?
  • Parallax does not currently stock the PropCam; but it's very much a live product, and I have plenty in stock. Unless/until Parallax resumes selling them, I can furnish them from here, as long as it's a domestic shipment.

    -Phil
  • rjo__rjo__ Posts: 2,114
    rabaggett wrote: »
    I look at PropCam, and think about buying it, learning it, using it, but somewhere in the back of my mind, I think I read somewhere that no more of them are being built. when current stock is gone I will need to find/learn another, Is PropCam a live product?

    I was confused about this as well. In fact, some time ago, I was all set to order several units from Parallax when I saw the notice on the product page.
    I assume that this is a temporary situation, but until whatever is resolved, it would be helpful if Parallax would add a notice that the units are available and link to Phil's private message.

  • rjo__rjo__ Posts: 2,114
    edited 2016-05-30 12:55
    In order to do the stuff I really want to do with PropCam I need SDRAM or HR and possibly parallelized P2's. With the recent family expansion, I think I have enough FPGA boards laying around to prototype a system with 8 P2v's... but waiting for mass storage is killing me:)

    Come on guys!!!
  • rjo__rjo__ Posts: 2,114
    I have updated to P2Vv10 and switched from NTSC to VGA (640x480 8bit). The serial service has been updated, but the supporting Windows 10 serial terminal app has not been updated.

    I am using the PropCams in serial mode. I have the parallel mode working... but the program is already too big and I'm starting to throw stuff out before I add more.

    This update implements the initial steps in stereo-analysis, using the Pixel Processor. Until we have access to large memory, I will be sticking to area statistics for correspondence matching. This is not fully integrated, yet. Area statistics are less accurate than my favorite approach... but far faster:)

    Commonly, a second order differential is performed on each image before correspondence matching is done. I hate this because it contributes errors around the edges. Another way of mixing the stew is to do object recognition first and then correspondence matching... I hate this because near field objects in complex scenes can be hard to identify. So, I won't be taking either of these two approaches. Until we have access to large memory, I'll be doing area matching that simply ignores the edges. It yields sparse data... but pretty reliable results.

    I'm not sure this approach will be any better than the others... but at least I don't hate it.

    My goal is to eventually do as much as is possible at around 90 frames a second. And then to do as much as is possible without time restrictions.

    Right now I have the cameras turned off, because I'm not sure anyone has them. Instead the program loads the bmp image and cycles through the steps of digital subtraction and thresh-holding... intoxicating stuff, I promise:)


  • rjo__rjo__ Posts: 2,114
    oops,

    The code was written assuming parallel optics... which is how I have my PropCams set up. The image used in the demo, however, comes from a stereo-camera with a convergence point. The exact stereo-offsets being considered also come from the PropCam set up. So, the demo conveys the basic idea, but is not rigorous in any sense:)
Sign In or Register to comment.