Virtual link 10Mbit AC serial with fast update ---- now 20Mbit!
Peter Jakacki
Posts: 10,193
There were some examples previously which demonstrated high-speed serial using simple start bits. While they performed they relied upon gaps in transmission to permit an unambiguous start detection.
I had an application where I needed as high a speed as possible, as high an update rate as possible, and it had to be AC coupled so it had to be DC balanced (just as many highs as lows).
Achieving a high update rate means that the transmissions need to be back to back but therein is the problem, which bit is the start bit? (Imagine you are looking at a sample of a continuous bit stream).
Well, first off the AC coupling was handled simply by transmitting each bit as a complement in addition to itself so a high would be transmitted as a high then a low and a low as low than a high, simple.
This means that in a data stream that there would be no legal condition where any of the bits stayed high or low for more than two cycles. Also each data bit can be checked for integrity against it's complement.
By generating a start condition that exceeded the maximum two cycles I could easily detect it in the middle of a stream. A passive stop condition was added to balance the DC component and also allow time
for processing the data. In fact the stop condition may not allow enough time so redundant data bits may be generated before or after the stop condition to allow the receiver enough time to process.
Alternatively the data would be processed by another cog.
What's this good for? Well I have my own commercial uses for it but it would be a neat way to link multiple Propeller chips at high-speed together over many kinds of medium creating a virtual-link.
A variation of this is suitable for half-duplex buses as well so a single Prop I/O can be used as a virtual link and continually ping-pong data back and forth without any real overheads.
Other variations of this allow for multi-bit data terminated by a stop condition where the data is left justified so a CR character would be sent as 4 bits then a stop. A null is simply a start and stop.
I may be jumping the gun here but it's working very well in my application and I like to see if there is some interest to develop some useful objects.
Here is a waveform sample showing the zoomed view of a single packet as well. The blue trace is a simple diagnostic signal showing when the packet has been accepted to be processed. In my sample
waveform I have some redundant bits before the start condition as I have some processing to do. The start condition is 300ns wide and then followed by 9 data bits (plus their complements). I am only
using the first five data bits in my application. As you can see each packet is stacked end-on-end and the number of highs and lows are equal.
Update: Feb 17th
I was playing it safe with 10Mbit serial but I decided to make it faster and push it to 20Mbps instead. I was worried about the sampling stability but by throwing in an extra zero data bit after the start condition I could resynch to the middle of the input sample. The original start and stop conditions were both 300ns long which theoretically could be reduced to 150ns each but due to other overheads I have them set at 200ns each. So the payload overhead besides the complementary data bits amounts to 500ns/packet. Each data bit requires 100ns so 32 bits would take 3.7us with a sustained rate of 270,270 32-bit packets per second. Of course 32-bits of data requires 64-bits which actually means it would take 6.9us/packet or 144,927 packets/second.
Now I have to shape it into a usable general-purpose object. My screenshot "vlink20.jpg" shows me transmitting 6 bits of data with an update rate of just under 1 million packets per second and I didn't need to pad my packet with dummy data either as I now do it more efficiently for my application.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
Post Edited (Peter Jakacki) : 2/17/2010 8:09:30 AM GMT
I had an application where I needed as high a speed as possible, as high an update rate as possible, and it had to be AC coupled so it had to be DC balanced (just as many highs as lows).
Achieving a high update rate means that the transmissions need to be back to back but therein is the problem, which bit is the start bit? (Imagine you are looking at a sample of a continuous bit stream).
Well, first off the AC coupling was handled simply by transmitting each bit as a complement in addition to itself so a high would be transmitted as a high then a low and a low as low than a high, simple.
This means that in a data stream that there would be no legal condition where any of the bits stayed high or low for more than two cycles. Also each data bit can be checked for integrity against it's complement.
By generating a start condition that exceeded the maximum two cycles I could easily detect it in the middle of a stream. A passive stop condition was added to balance the DC component and also allow time
for processing the data. In fact the stop condition may not allow enough time so redundant data bits may be generated before or after the stop condition to allow the receiver enough time to process.
Alternatively the data would be processed by another cog.
What's this good for? Well I have my own commercial uses for it but it would be a neat way to link multiple Propeller chips at high-speed together over many kinds of medium creating a virtual-link.
A variation of this is suitable for half-duplex buses as well so a single Prop I/O can be used as a virtual link and continually ping-pong data back and forth without any real overheads.
Other variations of this allow for multi-bit data terminated by a stop condition where the data is left justified so a CR character would be sent as 4 bits then a stop. A null is simply a start and stop.
I may be jumping the gun here but it's working very well in my application and I like to see if there is some interest to develop some useful objects.
Here is a waveform sample showing the zoomed view of a single packet as well. The blue trace is a simple diagnostic signal showing when the packet has been accepted to be processed. In my sample
waveform I have some redundant bits before the start condition as I have some processing to do. The start condition is 300ns wide and then followed by 9 data bits (plus their complements). I am only
using the first five data bits in my application. As you can see each packet is stacked end-on-end and the number of highs and lows are equal.
Update: Feb 17th
I was playing it safe with 10Mbit serial but I decided to make it faster and push it to 20Mbps instead. I was worried about the sampling stability but by throwing in an extra zero data bit after the start condition I could resynch to the middle of the input sample. The original start and stop conditions were both 300ns long which theoretically could be reduced to 150ns each but due to other overheads I have them set at 200ns each. So the payload overhead besides the complementary data bits amounts to 500ns/packet. Each data bit requires 100ns so 32 bits would take 3.7us with a sustained rate of 270,270 32-bit packets per second. Of course 32-bits of data requires 64-bits which actually means it would take 6.9us/packet or 144,927 packets/second.
Now I have to shape it into a usable general-purpose object. My screenshot "vlink20.jpg" shows me transmitting 6 bits of data with an update rate of just under 1 million packets per second and I didn't need to pad my packet with dummy data either as I now do it more efficiently for my application.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
Post Edited (Peter Jakacki) : 2/17/2010 8:09:30 AM GMT
801 x 600 - 355K
801 x 600 - 293K
Comments
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
www.mikronauts.com E-mail: mikronauts _at_ gmail _dot_ com 5.0" VGA LCD in stock!
Morpheus dual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory/IO kit $89.95, both kits $189.95 SerPlug $9.95
Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz
Las - Large model assembler Largos - upcoming nano operating system
Nicely done! You've almost got Manchester encoding going there. But I think your method is much simpler to decode with the Prop, especially at 10 Mbaud. Without the start/stop synchronization, you'd have to resort to some sort of HDLC or SDLC protocol.
-Phil
It looks good and could be very useful, especially if it will work like a 10Base2 bus with just 1 pin per propeller.
Doug
Have you actually tested throughput? When you say 10Mbit/sec, does that mean the signal runs at 20MHz since each bit has a high and a low?
How does this compare to BeauNet? More or less throughput?
Thanks,
Doug
In my application the data is capacitively coupled into differential drivers and receivers but I am looking at trying this out on a standard ethernet transformer in half-duplex mode as it would be a useful way to isolate the signal. A RS-485 transceiver may be a useful driver/receiver for this too as well as on it's own. The half-duplex mode only requires a single cog and there is time to process the information received as the protocol relies upon ping-pong communications where one side transmits and just waits for a return packet etc. This of course cuts down on the throughput but still provides a high-speed link. Ping-pong Vlink relies on timeouts and random timer transmit on line idle to initiate or reestablish communications. Once it has sent the packet and the other side replies then it all just flows continually and smoothly.
Thanks to everyone for your compliments, I'm simply building on work others have done in this excellent forum. Any ideas on applications? Maybe even using another Prop as remote I/O and accessing the pins as simple variables from the application code. Of course the remote Prop could be doing some other smarts as well.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
The most usefull thing I can think of to do with this is to create a prop to prop DMA transfer driver.
For example. Using you link one propeller chip could dump the contents of a small section of its memory over the serial line to be read in by another prop chip. That prop chip then could upload that data dumped into its main memory.
It would be like having automatically updating variables in main memory that would originate from other props.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Nyamekye,
The protocol I had in mind was to use 12bit characters where the upper 4 bits were used to identify the I/O controller (like 0001=TV, 0002=serial, 0003=VGA, etc). This could just be a mod to Peters' object.
Kye: For prop-prop DMA, I would use 32bit characters as that would be faster. However, this would not be general use, but rather targetted use.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:
· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz
Have I got that right, or is yours packetized also?
Thanks,
Doug
I guess the cool kids have it all figured out...
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
JMH
http://forums.parallax.com/showthread.php?p=691952
Doug
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
Is it really possible to transfer large buffers of say 256 bytes between Props, HUB to HUB, at a sustained 20MBit/s ?
I'm having a real bad does of flu today and can't even begin to think about it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
For me, the past is not over yet.
Check the link that hinv left a couple of posts back.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
How does your scheme differ from Manchester encoding?
I am a little confused. Before you said "so that translates to 4.571Mbps", and now you say you doubled the bitrate to 20M, and now you say 144,927 32bit-packets/second which calculates to 4637664bits/sec.
It would be less confusing it you differentiated between actual data bits and "transitions" in the signal, or some other term because bits are data in my vocabulary.
For instance, Ethernet originally ran at 10Mbit, and the throughput on some systems was 1.1MByte/Sec(9.227440bits/Sec)at the application level(ftp). With that kind of bandwidth, we did remote display on over 30 NCD Xterminals (pushing it a bit) on the same lan.
Thanks,
Doug
Post Edited (hinv) : 2/17/2010 2:34:09 PM GMT
As you know, Manchester encoding might look similar but it decodes differently as it relies upon hi/lo and lo/hi transitions at bit center to indicate logic levels. Unless we had some extra hardware then that's a bit hard for the Prop to do so while I achieve zero DC average I keep it simple for detecting actual data bits.
Now, there is a lot of confusion regarding bit rates etc. While BeauNet says it is a 14.5 Meg Baud that does not really convey much about actual throughput. It is far better to say it is a 20Mbps serial stream and that the effective "data" bit rate is 14.5Mbps (I think). That way you know the baseband signaling rate that your interface has to handle and you get to know just what that translates to in actual data. Imagine if you had a system that generate 100Mbps bursts of data yet it's overhead meant this was reduced to 10Mbps actual data. It's important to know that your interface can handle the 100Mbps stream (50MHz) but that you will only get around 1M bytes per second in actual data.
Remember my objectives that I had previously stated so what is important for me is to know the baseband signaling rate, that it is a continuous "AC" stream, and that I can synchronize easily by detecting the unique start condition at any point in the transmission. There is also another advantage with my scheme in that although it is not differential or bi-phase which is not polarity sensitive, I can however detect inverted polarity easily as my unique stop condition precedes the start and in software I can flip the polarity based on their states.
BeauNet on the other hand is quite suitable for direct I/O to I/O connection over very short distances with a higher effective "data" (not bit) rate than my scheme which makes it more suitable for these backplane and chip to chip schemes.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
I think what I meant to say "signaling rate" vs bit rate. Those two are then distinct of throughput. But the real question is if you doubled the signaling rate, the throughput should also double?
I have never seen the kind of protocol overhead you are talking about. Even with all of the layers on ethernet, when it went from 10BaseT to 100BaseT, throughput went from 1.1MBytes/sec to 11MB/Sec on some machines that had really tuned implementations.
I think what you have is real good. I would love to play with it. I could then answer my own questions. Do you plan on releasing the source?
Take Care,
Doug
Peter, did I get that right?
-Phil
$0d transmitted as asynch : 1111111 0 10110000 1 111111 STA 01234567 STP $0d transmitted as complementary bit : 111 01 01 01 01 10 10 01 10 000 STA 0 0 0 0 1 1 0 1 STPNotice that the CB method transmits msb first and simply transmits an XOR of that data bit immediately after. So you end up transmitting twice as many bits as there are data bits but the reason for this is to allow transmission over AC coupled media. Notice also that in the data field of the CB method there are no bit patterns that have any bit stuck for more than 2 cycles. Using that to our advantage we create a start condition that stands out from the jungle by making it 3 cycles of high bits. To balance the start condition and acheive an average zero DC component we need to transmit a "stop" or gap as 3 cycles of low bits.
So this CB method is a simple complementing bit extension upon a normal serial stream with the other addition of start and stop conditions. The software for this is simple although timing is critical. I was thinking about a general object but for most uses it would seem that BeauNet would be more practical. It's because I am transmitting over single-mode fiber over very long distances and the receivers are sensitive AC coupled devices that I need this type of serial stream. If the frequency is too low the gain will be poor and I just get a lot of noise out of the receivers.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
I am still confused about the data rate not doubling since you doubled the signaling rate.
I can't wait to see what you come up with for code. Are you going to wait until it is polished and put it into obex, or post the code and get input?
Have you thought about how to communicate with an ARM with this "protocol"?
Doug
You speak of inverting every second bit in the stream for the purpose of balancing the high/low periods to eliminate any DC component. This is in order to feed the stream through transformers, or possibly AC coupled circuitry.
To address that issue, I would suggest that at the speeds you are dealing with that a similar approach could be implemented by sending just a standard async ASCII byte, followed by that byte's complement. The start and stop bits are already complements of each other, so together they add no DC. What this achieves is utilizing standard UART technology with "instantaneous" DC components lasting for only the duration of one 10 bit (start 8 data stop) period, and the DC in that balanced off by the following complementary byte. Although I have not tested this, I believe the short durations for which the DC is not balanced out will not affect the transmission medium significantly. Furthermore, when your stream stops, a DC component will be generated.
Continuously streaming standard UARTs however do not address the other issue you are dealing with; detecting the start of a byte commencing at an arbitrary point in the stream.
Presumably your stream does not last forever, hence there is a starting point that can be readily detected, and that could even be enhanced with a special sequence of bytes such as sync characters, or their equivalent.... fast square wave / slow square wave starting characters of an ethernet packet.
There are many ways to achieve high throughput..... my suggestion is just another one.
Cheers,
Peter (pjv)
My application does require continuous transmission or at least dummy pulses otherwise the resulting noise is obscene. What I have created is a virtual link across single-mode fiber that can run up to 40km (25miles) per span. As one of the tests I did I connected a Propeller board at the remote end and connected the local end to my PC and programmed the Propeller in the normal fashion, time after time after time it all worked without a hiccup. In this case the RS-232 port was virtualized so that every signal on the DB9 was mirrored on the remote end and vice versa. The virtual link does not care what format or baud rate the serial port uses but the link ends up working just like a very very long but perfect cable.
@Doug, I think you are getting confused between protocols and serial "formats". Protocols encompass the "this is the way we do things" but not necessarily the "how it gets done". Preambles are fairly common on many serial formats that need to stabilize hardware (RF receivers) or software, to synchronize their clocks etc, and sometimes it's just padding. Anyway, it is not necessary to compare this with Manchester encoding, this format (and protocol) has a purpose and if you understand the purpose then all the rest gels.
Since I doubled the signaling rate I have of course doubled my data rate. I can now transmit up to 1 million packets per second using the 20Mb/s.
To all. I admit that the bits per second thing can have many interpretations but I have explained in previous posts why it's important to know the signaling rate and also the throughput. For instance, BeauNet delivers 14.5Mb/s over a 20Mb/s channel, so the channel needs to handle 20Mb/s and the throughput is around 75%. With RS-232 there are many systems that may say they work up to 115.2Kbaud for instance but they cannot produce or handle that sustained rate, sometimes even after one character. These systems may have huge gaps between characters and require all kinds of delays between characters or after lines. We also know there are many advertised USB2.0 devices of which USB2.0 is capable of 480Mb/s for "high speed" devices but even the so called "full speed" devices can fail to deliver a 12Mb/s throughput.
My link layer that delivers 20Mb/s is probably not much different from BeauNet (Really, I haven't checked it yet) but what I do with the bits and at the higher levels is different. All digital transmissions uses 1's and 0's anyway.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
Please understand with my suggestion I was not trying to be difficult but offer another approach. Not knowing all the details of your application causes one to make some assumptions, and those might be incorrect.
So then as I now understand it, during the time when you have no need for data to be transmitted, the "idle" time you will be transmitting garbage data just to keep the receiver happy. And presumably this data is again complementary pairs of bits. Is there then no time when the channel is truly idle and rests for more than two bit times in a high or low condition?? If not, then I'm curious how you eliminate 100% of the data handling/processing overhead.
Cheers,
Peter (pjv)
Only 512 possible 10bit vaules are used, so a lookup table would use under 1kB of cog ram.
8b/10b is a line code that maps 8-bit symbols to 10-bit symbols to achieve DC-balance and bounded disparity,
and yet provide enough state changes to allow reasonable clock recovery.
This means that the difference between the count of 1s and 0s in a string of at least 20 bits is no more than 2,
and that there are not more than five 1s or 0s in a row.
·
Normally this would be out of the league of a Propeller chip as these devices are driven at such high bit rates and nothing in the datasheets ever suggests anything different. I figured I might even be able to modulate it at a higher frequency (128MHz) if I needed to but that would have complicated it with having a demodulator as well. This is another example of the flexibility and versatility of the Propeller chip, but one does have to think laterally and understand the Propeller and the application.
For general-purpose use though I will look at coupling this through ethernet transformers with minimal hardware, perhaps just a fast RS485 chip. The software can be made to correct polarity inversions and the interface is inherently isolated. As well, I have bit-by-bit error checking built in!
BTW, there is no need for me to transmit garbage just to keep the line busy as I am using this as a virtual link so the more samples/sec the better.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*
Post Edited (Peter Jakacki) : 2/19/2010 1:16:41 AM GMT
Hi Tony, yeah, I'm aware of scrambling and encoding data for DC balance but remember this is high-speed traffic using the Propeller and I wanted to keep it as simple as possible, there are so few clock cycles to do any kind of processing in plus operating at such a low bit rate compared to 155Mb/s I felt better in ensuring that every bit had an edge to maximize the detection in the receivers. DC balance is but one of the requirements, the "frequency" as I have mentioned is important too.
In the final analysis it's whether it does the job that really counts, and it's does.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
*Peter*