Elev-8 Commlink Serial Protocol

Comments

• JasonDorie Posts: 1,930

  2017-09-20 18:32

  Are you trying to decode the full data stream sent by the Elev8?
  
  The data is sent as 8 bits, no parity, 1 stop bit (almost the universal settings for serial data). Packets are sent with three u16 values as the header (0xAA55, type, length), followed by the data body as a multiple of 16-bit values, and a single 16 bit checksum. The 0xAA55 code lets me more easily find the beginning of a packet if the stream is interrupted. The total packet size is "length" bytes, which includes the extra 8 bytes for the header and checksum. All packets are rounded up to the nearest multiple of two bytes in length, so if you're trying to send a single byte you need to pad it with an extra one. This was done because the checksum runs twice as fast on u16's as on u8's, and almost everything I send is a 2 or 4 byte quantity anyway.
  
  All the "type" values are in the DoDebugModeOutput() function.
  
  One caveat to be aware of - I make no attempt whatsoever to guarantee that there is enough room in the serial buffer before I send, so if you exceed the available bandwidth, you'll overwrite and corrupt previous data that's still pending. I've spaced all the data out and made sure the packets are small enough that this doesn't happen in practice. This is what the "phase" value is used for in the DoDebugModeOutput() function - I use a counter that rolls over to keep track of which blob of data I'm sending on this update - it spaces out both the data stream and the CPU usage required to do it. If you're adding a new chunk of data to the stream that's more than a couple bytes you'll need to account for this.
  
  The simplest way would be to remove an existing packet and take over that "slot" with your data, but use a different "type" value in the header. For example, phase 7 is the "desired quaternion" data, which is probably not needed much. You could send your data instead, or you could even alternate - send your data every other update, so groundstation still shows the desired orientation often enough to be useful.
  
  Does that all make sense?
  
• jakpapy Posts: 8

  2017-09-24 22:35

  Thanks for replying! Yes, that makes sense and is quite informative.
  
  I'm not trying to decode the full data stream, but instead replace it with values that I would like to observe and record in Labview during flight. I have commented out CheckDebugInput() and DoDebugModeOutput() and instead filled in its place:
  
  if(Radio.Aux1>500 && dataUpdate==1){
  COMMLINK::StartPacket( 0, 4 );
  COMMLINK::AddPacketData(&testVar1, 2 );
  COMMLINK::AddPacketData(&testVar2, 2 );
  COMMLINK::EndPacket();
  COMMLINK::SendPacket(1);
  dataUpdate = 0;
  }
  
  where dataUpdate is a flag stating that there is new data in testVar1 and testVar2 that has been calculated on another cog ready to be transmitted. Therefore, it should only send data approximately at the rate of the other cog (50Hz). I found that when SendPacket(0) is used to transmit over usb, Putty terminal can read it, but for some reason Labview has a problem. When I connect via the Xbee, Labview is able to read it just fine (which is the important thing as I will not be flying while connected to USB, I just don't know why that problem exists).
  
  The problem I am now running into is that it seems that the Xbee induces a lot of lag such that it seems there is a backlog of data in some buffer and the data arrives a second or two after an event occurs. I configured the LEDs on-board to turn on when given the same event that is transmitted and the LEDs turn on with no lag and then a second or two later the data appears on the receiving end of the xbee. I noticed this seems to be a problem in Groundstation even when running this with the master code without any of my changes. I also already checked that both xbees are configured correctly to the 57600baud. Are you familiar with such an issue?
• JasonDorie Posts: 1,930

  2017-09-25 06:14

  If you're doing that every frame, you're likely hitting FTDI's magic buffering scheme. The data is partly spread out the way it is to prevent FTDI's driver from clustering it together. If you exceed 62 bytes (I think?) within a certain time frame, they wait until the receive buffer is full (4Kb) before passing it to the host. It's obnoxious.
  
  And I'm not sure what update rate you're talking about - Elev8's main loop is 250hz, or 200hz for the absolute latest branch from GitHub.
  
  It's possible you're hitting an XBee buffering issue too - their transparent mode does something similar, so it'd be worth looking through the XCTU (XTCU?) interface for settings to do with latency and buffering. I don't recall what we settled on - it's been a while since I played with the XBees.
• jakpapy Posts: 8

  2017-09-26 04:52

  It is good to know about the FTDI's buffering scheme, that is likely the problem! I'm surprised that is occurring though, I'm only transmitting at maximum 4 bytes (plus the additional bytes for start sequence, length, checksum etc) at 50hz - that's only 4800 bits per second (plus stop bits).
  
  Sorry, that update rate is describing a new cog I've set up that is able to process sensor data at a rate of 50hz maximum. That sensor data is saved to a global variable and in the main loop, only transmitted when new data is made available. I've found that if I only transmit at a rate of 5-10 hz (still reading from the sensor at 50hz) there doesn't seem to be any lag.
• jakpapy Posts: 8

  2017-10-30 23:32

  I'm back with a semi-related question. I want to collect data from what is going on inside my implementation on a different cog. I thought about using the datalogthread cog, but I couldn't seem to get it working. Additionally, I've found that I cannot seem to use the COMMLINK commands from inside my cog, so I have to use them in the main loop. My understanding is that the transmission of data happens on the S4 cog so using the COMMLINK on the main cog should not cause lag no matter how long the message is/baud rate that is used but I managed to set off the alarm for the main loop taking to long to run a couple times. Additionally, as I should be using volatiles for variables shared between cogs, commlink doesn't seem to like them and I have to use a workaround (I get an error saying something along the lines of "error: invalid conversion from 'volatile void*' to 'void*' [-fpermissive]").
  
  Long story short:
  How would you recommend for me to collect the values of 12 static volatile int variables on another cog for post analysis? If the transmission/recording can take place on that cog, only 2 of those variables need to be volatile.
  
  Thank you!
  
• jakpapy Posts: 8

  2017-10-30 23:42

  Actually, it seems as though trying to transmit this data using COMMLINK does trigger the alarm for slow loop time even more when using my workaround for the volatile issue. Therefore, I think it is necessary to use a different robust method for transmitting or recording the data, preferably on another cog.
• JasonDorie Posts: 1,930

  2017-11-02 17:48

  You'll be fighting a few things here:
  
  1) The transmission of existing data is timed and spaced out so it never blocks when sending
  2) The TX buffer isn't very large, and I know that by the time I trigger my next send there will be enough room, so I won't have to wait when adding new data
  3) The main loop timing is fairly tight, and it's running in CMM mode which is interpreted code
  4) XBees are not fast
  
  When connected via serial, the max throughput is about 11 Kb/sec. You never want to actually send that much data, because you need "gaps" to get the FTDI chip to pass the data to the host and not buffer it all. With an update rate of 250hz, it doesn't take very long to hit that.
  
  Even just copying data to the serial buffers and computing the checksums takes time (especially in CMM mode) and if you're trying to send 12 longs in a single update without removing an existing send of roughly the same size, you'll hit all these problems.
  
  Your 12 longs are 48 bytes, + packet overhead. The TX buffers are only 64 bytes (https://github.com/parallaxinc/Flight-Controller/blob/master/Firmware-C/elev8-main.cpp#L556) so unless you've allowed enough time for whatever was sent in the cycle before you to transmit, you may end up waiting to put new data in there. You shouldn't need anything robust to collect the data if it really is just a few longs, but the transmission timing is tight because of the bandwidth limits. I doubt you need to mark your data as volatile either - the debug data sender isn't in a tight loop where variables would be cached in registers, so there's no point - they'll always be reloaded when that code runs anyway.
  
  The largest single packet I send is 24 bytes. If you were to remove the output of cases 5, 6, and 7, and replace those with the first, second, and third parts of your 12 longs, you'd be well within the limits of the system.
  
• jakpapy Posts: 8

  2017-11-03 20:42

  I'm not too concerned with the frequency of data transmission as I can space them out - transmission of these 12 variables can happen even as slow as 5-10 hz, so I can send one variable at a time even.
  
  I'm sending 12 ints, not longs. Also, part of my concern is that if I want to do things right, these SHOULD be volatile as they are calculated on a different cog and must be shared to be sent, but the COMMLINK functions are not prepared for volatile- do you think it would be OK to use static int instead of the recommended static volatile int?
• JasonDorie Posts: 1,930

  2017-11-03 21:36

  Int and long are both 32 bit on the prop. Short is only 16 bit, so if you can downgrade to shorts it's worth doing.
  
  Volatile is a hint to the compiler that the variables should not be cached in registers, and should be reloaded on every access to ensure that they are current. None of the other variables transmitted by that code are marked volatile either, because it's a large enough function, and not in a loop, so the chance of any of those variables being cached is next to nil - I wouldn't worry about it - static int will be fine.