Would you use the Propeller to simulate a neuron?
Rsadeika
Posts: 3,837
Every couple of years or so, it seems like the topic of AI comes up. We have a couple of threads dealing with neuronal arrays, and/or other configurations. My question is, is the Propeller really suited for this job? Maybe we should stick to the intended purpose of the chip, an intermediary for a sensor attachment.
I did not want to disrupt the other threads, I think their are a lot of opinions about this subject. So, lets share the ideas here.
Thanks
Ray
I did not want to disrupt the other threads, I think their are a lot of opinions about this subject. So, lets share the ideas here.
Thanks
Ray
Comments
From the following link...
["The [URL="http://en.wikipedia.org/wiki/Human_brain"]human brain[/URL] has a huge number of synapses. Each of the 1011 (one hundred billion) neurons has on average 7,000 synaptic connections to other neurons. It has been estimated that the brain of a three-year-old child has about 1015 synapses (1 quadrillion)."]
http://en.wikipedia.org/wiki/Neuron
Then from the next link...
["The processing power of average [URL="http://library.thinkquest.org/C001501/the_saga/compare.htm"]human brain [/URL]is about 100 million MIPS. (MIPS -Million computer Instructions Per Second) it means it can handle the process, which a 1640 GHz Pentium computer can handle...
...Since the human brain have 100-trillion-synapses, and each synapses can handle one MB of data, the human brain can store 100 million Megabytes of data..."]
http://mi-programacion.co.cc/processing-power-and-speed-human-brain-vs-computer
The typical response has always been to quote the statistics about the brain, and it's synapses. So, lets see if I divide 7 (cores) into a hundred billion, how many Propeller chips will I need? I think that at this point Parallax is salivating profusely.
I still have not seen any thread that would deal with just that one lonely neuron. But then may be we should deal with an astrocyte first, before moving to the neuron?
Ray
Architecture is the vision of the developer.
I find that the Propeller is much more than an intermediary for a sensor attachment. My Spinneret Web Server project does not contain a single sensor.
Ray
Been working with some machine vision - pretty difficult I would say.
And also, ANN is 2D. I don't know much about 3D representation, however some of the researchers had been doing that already.
http://www.cs.cmu.edu/afs/cs.cmu.edu/usr/mitchell/ftp/faces.html
I compiled the code without any problems using gcc (Cygwin), but haven't tried it on the supplied images. I need to read the documentation.
A combination of ANN with other AI-based algorithms are done too, like Gabor and SVM (Support Vector Machines).
The Viola-Jones algorithm seems to detect faces quite fast, uses integral image processing (and some other algorithms inside as well) and only involves addition/subtraction, but some modification must be done on higher-resolution images to cut down on the false-positives. OpenCV has onboard functions which does that nicely.
The attempt to use Propeller to simulate neurons requires splitting the task in a distributed manner. This has not been the path of neural network development because it *didn't have to be* the path. People and things tend to follow the "path of least resistance."
There is value in the attempting to simulate neurons on Propeller because of things that can be realized along the way and it *may produce* some usable results. I personally have many, many, other more useful things to do. And my time will be spent in productive ways which means neural networks will not be my highest priority.
The thing is, you have people here who honestly believe the Propeller can be a "Brain" and have been convinced of this by a vapor ware advertisement practicing propaganda machine .... I personally think it's stupid to believe that any combination of Propellers can behave like a "Brain" other than running programs created by people with brains inclined to write programs.
There is nothing wrong with exploring. However, expectations, prognostication, and vapor should be modest until there is some proof otherwise.
If anyone wants to help in productive ways other than being some propaganda machine, I welcome it.
In general two approaches to simulating neurons have been used.
1 - Build an electronic circuit that mimics neuron behavior.
2 - Use software running on a fast computer (or cluster of computers) as Leon mentioned.
Option 1 has been tried but has been generally dropped in favor of 2 due to the high cost and complexity.
Leon has stated in this thread that:
"A typical ANN used for pattern recognition uses a multi-layer feed-forward Perceptron with at least three layers (one input layer, one output layer and at least one hidden layer), and is generally implemented on a single fast processor with fast RAM. The Propeller doesn't seem to be particularly suitable."
While his assertion that the prop is not particularly suitable may be true, I am not so sure of that.
The folks on this forum have done some pretty amazing things with the prop. Using PASM for neuron simulator code that runs in parallel on individual cogs and uses hub memory for connection tables could prove to be a viable proposition. The best way to find out is to build it, write the code, and go through the usual rounds of improvements.
Personally I think the prop has a lot going for it for this application.
1 - Eight independent cpu's
2 - Very flexible I/O
3 - Shared hub memory accessible by all cogs on the chip
4 - Very simple hardware for a board with an array of props.
While the limited memory of the prop is a disadvantage that disadvantage is not as great as it seems, particularly for this application. The only way to be sure is to build it and find out.
1. Communicate with another neuron.
2. Communicate with other sensors.
3. Hold data.
4. When in an array, has a learning capability.
So, how would the Propeller fit the situation. Basically the Propeller has eight cogs, and 32 i/o pins. I will address each point mentioned for the neuron.
1. Inter-neuronal communication. I believe Beau did some experiments with communicating two Propellers. If I remember correctly he something set up where he had it up to 20Mbs, which is not to shabby. So, the networking part, in a very general format, is doable.
2. Sensor communication. This is a no brainer, plenty of examples for doing just that.
3. Data source. I am not sure if I have seen any examples of holding external data in cog or hub memory. This could get a little iffy.
4. Learning capability. This is the hard one, I have not seen any code, that shows that it would be feasible using a Propeller setup.
Since most of this usually pertains to the subject of robots, how practical would this be to implement on a robot? In another thread I saw mention of a 64 Propeller array. The Boe-bot sure would look interesting with a tower of 64 proto boards.
I am all for experimentation, but I have not seen any real functioning implementations. So, somebody give me some real examples of a functioning implementation of a small array of Propellers.
Ray
This is a major goal. Not quite there yet.
There is some character recognition Spin code that uses a neural network. It is a single Propeller program and there are very few comments in it. It may be hard to digest, but the demo picture looks good.
http://forums.parallax.com/showthread.php?93467-Character-recognition-project-*finished*&p=643109&viewfull=1#post643109
The loud noise in the forest that you didn't hear - does that mean it didn't happen?
Do you really think there's only one intended purpose for the Propeller chip?
I doubt that would be implemented as a stack of 64 protoboards. If I wanted to do something like that it would be with 4 boards having an array of 4x4 props on it.
The original discussion started on Humanoido's big brain thread about a week ago, and was transferred to it's own thread four days ago. Don't you think expecting real examples in that short a time frame might be unreasonable?
@Leon
Again, don't you think expecting real examples in that short a time frame might be unreasonable? Also, please define what you mean by parallel processing. Do multiple props controlling a CNC count as parallel processing?
At the time the transputer came along it was a big deal. Very high tech and powerful so a lot of folks jumped on it for parallel processing and communications. By the time the prop came out parallel processing had moved to multiple x86 systems with large memories so the prop has never been considered for such use. No surprise there is not much example software for using arrays of props for parallel processing systems.
@ All,
I have no doubt an array of props can be used to perform useful parallel processing. All of the pieces are already in place. All that is left is to assemble them and get them working together.
SIMD requires communications between the chips to distribute the same set of instructions to all the chips, synchronize the chips, and then distribute the data to each chip. Each of the three individual steps has already been done. All that remains is to put them together.
I am not aware of any examples, however at this point I have to say you are really nit picking. Any micro from the smallest 4 bitter on up that has some form of I/O can be connected together and programmed for both MIMD and SIMD processing. It may be wildly impractical to do so, and the result may be excruciatingly slow, but it can be done.
If Leon was ignored here like over on avrfreaks, he wouldn't have over 5000 posts.
Something to think about.
What are the good points I've made, so that I can continue in the same vein?
I'm not ignored on AVR Freaks. I make a lot of contributions, as I do on other forums, which seem to be appreciated.
I disagree with your first point, and as for the second point, once I gave it some thought MIMD has been done on a single prop by every project that uses more than one cog so there is no reason it cannot be done with multiple props. As for SIMD that has also been done on a single prop, the perfect example being the video drivers that use multiple cogs. Again, if it can be done on one prop it can be done on multiple props.
Where are the arrays of Propellers, doing things that are done with other parallel processing devices?
Reverting to the topic of this thread, has anyone got any positive things to say about using Propellers for neural net applications? I am reminded of Dr Johnson's famous retort: "Sir, a woman's preaching is like a dog's walking on his hinder legs. It is not done well; but you are surprised to find it done at all."