MAGIC32 - 32 magic channels of impossible jitter-free fast serial in 1 cog (transmitter now working)
Peter Jakacki
Posts: 10,193
I've decided to give this its own thread after mentioning my work in "Full Duplex Serial: Is it really full duplex?" and my Tachyon thread. Since it is a work in progress this would be a good place to post information about its progress and of course this is all "impossible" 
(Now isn't that the fun part of any project?)
The goal is to have any mix of up to 32 receive or transmit channels running in parallel at up to 115,200 baud from a single cog. At present they are all locked at the same baud rate but even if it proves a little too difficult to implement mixed baud rates the fact is that there is nothing stopping anyone from launching another cog with the same object.
I will skip the description of how this operates at present and instead show the logic analyser output of a cog hammering away with 32 transmit channels at 115,200 baud. Only 16 are shown out of 20 I/O pins that are enabled as outputs but it makes no difference to the driver if only 1 or all 32 are active.
To give it a distinguishing name I've decided to call it MAGIC32 rather than "FDS something" as the technology is based on Clark's third law which states that sufficiently advanced technology is indistinguishable from magic although in fact the fun part of doing this is the magic. Ain't the Propeller are a lot fun?
EDIT: MAGIC is an acronym for "Multiple Asynchronous Grouped Interface Communications" (or something like that).
The practical side of this is that you can make any or all 32 I/O a transmit or receive which is handy for me as a lot of the work I do also involves multiple serial ports usually over RS485/422/232 or just plain logic levels.
Here is the digital logic analyser capture of 32 channels transmitting strings at 115,200 baud from a single cog, some I/O pins are not enabled of course. After refining this part I will move on to the receiver section which will prove to be quite tricky to remove the "im" part so that it will be "possible".
BTW, this works just as well at over 1M baud but I need to eventually match the receivers so 115,200 is my goal.
(Now isn't that the fun part of any project?)The goal is to have any mix of up to 32 receive or transmit channels running in parallel at up to 115,200 baud from a single cog. At present they are all locked at the same baud rate but even if it proves a little too difficult to implement mixed baud rates the fact is that there is nothing stopping anyone from launching another cog with the same object.
I will skip the description of how this operates at present and instead show the logic analyser output of a cog hammering away with 32 transmit channels at 115,200 baud. Only 16 are shown out of 20 I/O pins that are enabled as outputs but it makes no difference to the driver if only 1 or all 32 are active.
To give it a distinguishing name I've decided to call it MAGIC32 rather than "FDS something" as the technology is based on Clark's third law which states that sufficiently advanced technology is indistinguishable from magic although in fact the fun part of doing this is the magic. Ain't the Propeller are a lot fun?
EDIT: MAGIC is an acronym for "Multiple Asynchronous Grouped Interface Communications" (or something like that).
The practical side of this is that you can make any or all 32 I/O a transmit or receive which is handy for me as a lot of the work I do also involves multiple serial ports usually over RS485/422/232 or just plain logic levels.
Here is the digital logic analyser capture of 32 channels transmitting strings at 115,200 baud from a single cog, some I/O pins are not enabled of course. After refining this part I will move on to the receiver section which will prove to be quite tricky to remove the "im" part so that it will be "possible".
BTW, this works just as well at over 1M baud but I need to eventually match the receivers so 115,200 is my goal.
1366 x 768 - 106K
Comments
I could see demand for two forms - An All-in-one COG, and a Two COG design of Separate Tx/Rx COGs.
The all-in-one focuses on size, whilst the 2-COG design is more about speed.
Mention of details like Buffers/Fifos /Handshakes/Baud Granularity are also good in UART Libraries - ie spec it like a HW UART for users.
I've just got a few more things to add to it to be able to use it effectively in testing the serial comms for now and I may even add an option to buffer to a file on the SD card.
Now that is all the magic I need to see
Looks like I have to stick to the standard character encoding too to avoid any display problems but most of the decent terminals support color. Once this is up and running properly I will start using to debug the multiple serial channels and I expect to be enhancing the analyzer in-between debugging the comms.
The side benefit of having this as regular text is that it is easy to copy & paste into a document, albeit without the color from what I have seen so far. The display update speed at 2M baud is under 200ms for a 64 sample window and 360ms for a 200 sample as shown. This can be certainly improved but for the moment is more than adequate otherwise I risk getting sidetracked having more fun
What's also cool about the analyzer too is that I can type in some new code from the terminal to enhance or customize it, right there on the spot.
BTW, for MAGIC32 the buffering methods I will be using for the transmit will buffer buffers. Huh? Well it buffers characters or pointers to strings which can be considered buffers themselves, and also buffers pointers to virtual memory so they can access files directly. There's no point in unpacking a string byte by byte into a buffer, then picking out the characters from the buffer again. So no need for massive buffers, we have them embedded in the application already.
The challenge is in getting meaningful data on them from 32 separate sources with appropriate synchronization and in that regard you aren't very forthcoming.
The requirement for buffers is application dependent and most applications do require buffering and almost all the rest greatly benefit from the simplified semantics of buffered output. Your approach places that burden on the application, which usually has substantial impact on correctness, code size and performance.
Still, you require some sort of producer/consumer model to synchronize the flow of data. What's your approach for that with 32 non-synchronized producers?
How's your code size and performance budget looking for receive?
I think everyone here can see the problems Peter is up against. Including Peter himself.
It's very easy to come up with reasons why something will not work. I'm sure that whatever comes out of this will be interesting even if it does not fully meet the stated goal.
BTW k, look at the timing diagram I first posted and read the text if you can as it's in ASCII. The ASCII characters are loaded into the FIFO as strings, of course there is no dumb 1Mhz wriggling going on. How can you say I haven't been "very forthcoming" when I have posted plenty of information about it even though I haven't even spent a full day on the code yet and am still in the throes of changes. I don't play silly games but a tough but realistic (to me) challenge I'm up for. The silly games I leave to the armchair critics
BTW, here is a quick capture of the transmitters from the still incomplete SPLAT logic analyzer:
The history of technology development is littered with the debris of those who boast of amazing potential and then deliver mediocre results, frequently fleecing many of much in the process. Occasionally it's outright fraud, but in most cases it's a combination of arrogance and ignorance, and there are new examples every day. Aereo is a classic example of ignorance. Apple's Saphire debacle is a fine example of arrogance.
While you could have developed an implementation and then presented your results, as is the approach required in peer-reviewed science, you instead have chosen to make preposterous claims that do not stand up to even casual analysis, and then you dribble out pictures that purport to show something, but prove nothing.
And then when asked for details, you wave your hand and scoff, denigrating the questions and the questioner in the process. There has been no information, not even any data; there have been no facts at all.
And to be clear, there is no challenge here - you've laid down your own gauntlet. This is your silly game. You're making up the rules as you go along, don't snipe at those providing color commentary for your audience.
Someone said "It's very easy to come up with reasons why something will not work". That's simply not the case in the face of a demonstration of one's claims.
And while it is true in some instances and particularly so when the claims are ridiculous, what's unfathomable is that one would continue to dig a hole in the face of such reasons rather than to discuss them on merit and point out errors if they do exist. Socratic method and technocracy are the hall marks of great engineering and great engineers flock to them both. Avoidance is something else altogether.
What's happening here is akin to watching a 10 year old build a tower to the moon out of 2 x 4s. We all know it's a fool's errand at best. While failure is inevitable, it's exact form difficult to predict and ranges from the comedic to the tragic. The entertainment is of value only to a sadist. And the opportunity cost is potentially huge. Who stands by or, worse yet, encourages such lunacy on behalf of the naive?
You have now descended into insinuations of bad intention "fraud", and or personal commentary "arrogance and ignorance", "10 year old", "lunacy"
Please don't do that here. It's annoying. I don't appreciated it at least. Thank you.
Peter is free to do what he likes, as are we all. He is welcome to discuss it here, as are we all I hope. He has indicated that whatever the final result is, success, failure, or something in between will be published here.
That peer review of which you speak does not generally happen until the work is done and presented for review. That is often how science works. That's not to say that scientists involved in some research or other don't discuss what they are up to among themselves at the forums and conferences they have.
SORRY BUDDY, you live in some other world or universe. None of this stuff I do now or have done before I do for money or glory, just for the fun and for the challenge of it or are you too shallow and jaded to see that????? Don't bother to comment any longer, you're not welcome any more. You need to seriously moderate yourself.
I like the way you are pushing the limits.
Propeller has a long history of impossible things done and for sure the journey is exciting.
I cannot understand some comments, I didn't check recently the forum guidelines (you know all this politeness stuff..), but regardless of the rules this is quite disturbing.
Massimo
That would still be an advance on what we have now. It could have it's uses.
Or, it's all done for the fun of pushing the limits.
As long as it's usable without Forth I'll be very happy
I don't pay much attention to form rules much. But simple common courtesy is always a good guide. Being aware of whom you are speaking to helps.
Not really worth replying to the peanut gallery. Most every one here knows how much you two have contributed to the Parallax world. I have personally learned a great deal from you two.
I try to keep my mouth shut and only ask questions but sometimes...
Any way guys keep up the good work!!
The entire tone of the post which seems to have generated such ire is drawn directly from Peter's posts, which I simply presumed were the bastion of courtesy in these forums based upon his prolific postings here.
It never occurred to me that what's game for he would not be game for me.
I stand corrected and respectfully withdraw anything that might have been misconstrued as aspersions.
I am, however, still wondering how one gets from "I'll skip the description of how it operates" to wondering why others might see that as not being forthcoming. Perhaps someone would have the common courtesy of explaining that to me?
Nobody has any obligation to be more forthcoming here or anywhere on the net than they want to be.
Best thing is to watch the progress, if we are interested, and offer constructive advice if we can.
Only to understand the meaning of MAGIC is worth reading this threat. ;-)
I think its fun for someone to declare they are going to do the apparently impossible, and then share their daily progress. It's early days yet. I say lets watch from the sidelines and cheer him along and celebrate if he succedes and comiserate if he fails. Whatever happens, lessons will be learned, skills increased. I would prefer this to be a pleasant experience, not one of having to skip past posts with accusations and defenses.
If nothing else, the idea of presenting "ascii art in minicom" had me reaching for my notepad to give this a try sometime! - need to find out the control codes for colour etc.
Dave
One may be uninformed of the facts, naïve about the facts or stubborn in the face of the facts. But by being so, one is not disagreeing. In saying the earth was the center of the universe, the Vatican was not disagreeing with Galileo, they were being steadfast in their dogma and refusing to evaluate the evidence and defend their position on merit. They were first uninformed and then stubborn. But there was no disagreement. They were just wrong.
I'm nowhere did I suggest anyone ought be forthcoming, I simply point out that he is not; to which he took great objection and you rushed to his defense. But the hand wave of "I'll skip the description" is as far from forthcoming a position as one could possibly adopt without directly insulting the intellect of the audience. I've asked for someone to have the common courtesy to explain it otherwise, but neither you nor anyone else seems willing or able to do so.
ErNa,
I believe the colloquial translation to English is "You can lead a horse to water, but you cannot make it drink".
KMyers,
I've tried to see it you put it before I spoke out, but in my mind, it's just is not. It comes down, unfortunately, to the meaning of "it" in this context. "It" in this context is a 32-port full duplex serial driver, suitable for use in place of the existing 4 port driver and that operates at 115200 baud.
That is not what's being worked on. We know this because that is not remotely possible. The work in progress is so far off that stated goal that no one could ever possibly mistake one for the other.
Building a bicycle to go to the corner store while saying you're working on a rocket to mars is simply not an accurate representation of the truth.
Is what's being done of merit? Who cares! Certainly not me. Anyone can certainly go off and code up anything they choose and have a reasonable expectation of not drawing critique from anyone.
But to make audacious claims in public places, defend them with smoke and mirrors and then belittle those who question those claims - I can see no merit in that at all. Can you?
I'm all in favour of countering false claims. Especially if they are intended to deceive for profit or power. So far we have not seen any of that here as far as I can tell. Only an attempt to do the impossible. Results pending the end of the experiment. Sounds fair enough to me. Yep, sure can. As I said the before goal may not be fully met but Peter has done some amazing things and something of merit may well come out of this.
And yet, when I see a 10 year old or a 20 year old or an 80 year old building a tower to the moon out of 2x4, legos, or jell-o, I do not ask why. I do not tell the person it can not be done. I either smile and walk on by, or I stop what I'm doing and help! Will we reach the moon? 99.999% chance, no! Will we have fun building the world's tallest tower of jell-o? YEA WE WILL!
Ask questions all you want. Pry for information. Be inquisitive. Explain your doubts and why you think it will be difficult. There's nothing wrong with that. But you do not get to show your anger from unsatisfactory responses here. Peter has no obligation to show you his work today or a year from now or ever. If he says it's possible, and you say it's impossible, there is no room for an argument until one of you proves the other wrong. Peter is still working on his proof, and you have failed to show any proof - only conjectures (even if those are educated conjectures, they're not proof).
ksltd, I think your initial comment was at least going in the right direction in positing skepticism.
The problem was your labeling Peter as being less than forth coming, and then compounding it with lecturing.
I personally don't see any way that Peter can make this work, without some sort of restrictions on the tx/rx-side. However, I and many other are happy to go along for the ride to see what he comes up with.
If you want to question him or his personal challenge, then why not do it the accepted and civil way, by keeping it technical?
Its one thing to be skeptical and prod the firmament.
Its another thing to insinuate on your first post.
To fail to point out the obvious to the misinformed is to do a massive social injustice. The entire purpose of community is to focus energies of the individual on those places where they will yield benefit for the many
Only a sadist smiles and walks on by.
And I agree that there are no obligations of any sort. But at the same time, once one makes outlandish claims in public, there is a reasonable expectation that someone may point them out. And when that happens, to feign surprise is a bit much, wouldn't you agree?
You are totally correct in pointing out the reasons why you think this venture is impossible. I don't think you are the only one.
The issue I have is in your turning a technical debate into a personal attack. That is not helpful to anybody and annoying to have to read.
As a moderator, I'm tasked to keep the conversations here polite, respectful, etc. no matter how technically outrageous the topic might seem to some. If you don't like what's being described, either go read and comment on other topics you feel better about or limit your comments to observations of fact or a specific technique critique. Personal criticisms will no longer be tolerated here. Peter is experienced and has contributed a lot. This would not be the first time he (and others) have made outrageous sounding proposals and eventually reduced them to practice after a lot of work ... rendered gratis ... and/or described the limitations of the technique involved after a lot of work to characterize it.
There was a recent thread on PWM, where this was done.
You split the problem into 'organize bits' and 'playback' and the playback can be precisely paced over many channels because it is reduced to a simple copy.
The complex stuff is in the 'organize bits' tho for PWM you tend to need high edge precision, but relatively infrequent updates.
This is why figures like sustained throughput and buffer size matter.