Trying to solve a problem leads to all kinds of solutions. Some solutions may actually match the problem. Other solutions can match more compelling problems. I don't want to derail Humanoidio's enthusiasm for doing what he enjoys. Indeed something useful may come out of it one way or another.
I'm willing to wait and see, too. I admire Humanoido's enthusiasm and hard work and would never want to discourage him in his -- admittedly ambitious -- quest. That's never been an issue with me.
But when he reports actual progress, without sufficient detail to reproduce his accomplishment, then dodges direct questions about said progress, doesn't show any source code, and wages personal attacks upon those who express doubt, I begin to smell something fishy. I'd love to give Humanoido the benefit of the doubt. But if he expects to continue stringing us along for another 800 posts, it's got to be with something more substantial than what we've seen so far: i.e. schematics and original source code -- even if it's for something he tried that didn't work.
Am I expecting too much? If there's strong consensus that I am, I'll let it go. I'm not really enjoying this role one bit.
I'm with Phil on this, I haven't seen a single line of code for anything as simple as a neural net that Humanoido has running on a single Propeller.
He seems to think that by adding more and more Propeller chips his contraption will start doing something useful, in fact he is wasting his time and money. He is giving the rest of us some amusement, however.
Phil, I also have a fair amount of skepticism that wiring up propellers together will achieve a meaningful result.
As I pointed out eariler in this thread, most neural nets are simulated in software. This is because they work by re-wiring connections between neurons in response to an input trying to produce a desired output. Any two neurons in the net can make or break a connnection as a result of multiple rounds of training. So if you have neurons on separate propellers that want to form a connection it would require a change to the physical wiring repeated several times. This is an impractical way to train a neural net.
Dogpile!! cool!!... [you see: Ttailspin take a running dive into the pile]...
How about some ground rules here?
What is the cutoff point for the number of post's in a thread like this? After all, the freudian slips start way way back,
so how long should one wait before they "call" or go "all in" to stop a thread like this?
What was it that made this thread different? Why would it be allowed to go on so long?
and why stop now? is it because the results are not instant in gratification?
Is this big brain impossible to achieve even half of what has been suggested?
In other words, Is half of what Humanoido writes down true?
Do We really expect results from this big brain? if so, Where is YOUR code that YOU introduced to this thread?
After all, He has requested help in this more than once..
I have an excuse, I can't write code to save my life... what is Your excuse?
I have "training" in Reading between the lines, Yes, there is a small confidence game afoot,
Some paraphrased white papers, duplicate pictures, "stream of conscience" posting style,
and we won't be seeing anything from that "Failed Hard Drive", and as for moving to the states, well....
That seems to be beside the point for now..
What is needed now is YOUR Code input...
If this sounds a little bit like hero worship, Well... I am allways impressed by someone who can carry
a thought for more than paragraph...
Humanoido's on-going AI collage reminds me a lot of Todd Siler's efforts back in the 1980's. Siler was a research affiliate at MIT who used paintings and other artwork to tinker with whatever metaphorical connections he could dream up between the anatomy of the human brain and the design of particle accelerators, fusion devices, etc. In his book, Breaking the Mind Barrier, Siler weaves back and forth from science to art and, in my opinion, seems to do neither discipline much justice. Yet, his effort to make connections should be applauded, I guess. At one level, I sorta get what Siler was saying - maybe it was just a repackaged version of that old Platonic idea of "Forms" being something we tap into when we think and create and so it's maybe no surprise we see the same metaphorical structures arise again and again in different pursuits - but on a lot of levels, it seemed like Siler just talked a lot but never really said anything.
However, what I do appreciate is that a lot of creativity occurs intuitively. Sometimes the best ideas in the world can spring up from people just goofing around, not really knowing what the heck they are doing, but following an instinctive drive to do something - anything! - besides sitting around and watching TV.
I may have some doubts about this approach producing any results that could be described as intelligent or conscious behavior but I see no reason to force a stop to this thread. Unless I missed something Humanoido has never been rude, abusive, or tried to con anyone, at least no more so than many others on the forum. Many discoveries have been the serendipitous result of research into something else.
@ Martin_H,
As long as all the props have some form of hardware connection scheme wiring changes are not required for neurons to create or break connections to neurons on other props. In most of the research I am aware of the creation/destruction of connections is done in software.
@ Humanoido,
It would be nice to see some concrete information about hardware, inter-prop physical connections, neuron software, how neuron connections are made, etc.
Again (since I must not be expressing myself well), I have no quarrel with this thread in general, its length, its placement in the Robitics forum, Humanoido's objectives, or even his methods. Maybe something will come of it; maybe it won't. That's part of the grand adventure that he's sharing with us. And -- let's face it -- with more than 20,000 views, this thread has attracted a lot of interested readers. At least he now includes citations to the external references he uses, and I laud him for finally recognizing that necessity.
My point is not about objectives or methods or wanting to stifle creativity -- none of that. It's simply about honest reporting of any progress that's made and the evidence required to demonstrate it. When presenting such evidence, schematics are the hieroglyphs we all understand, and source code is our lingua franca. These are the communication tools that cut through the more-than-ample hyperbole afoot here and which would assure me, at least, that any reported progress is real. Unfortunately, though, that crucial element has not only been missing, but asking for it results in evasion, more smoke, and playground-style taunts. Hence my incredulity.
Every time I see Humanoido's updates I can't help thinking that it might all be a very elaborate joke. Or, he really has discovered something and it's his creation that is doing the posting.
Hard to imagine logging in and not see the "Fill the Big Brain" post way up near the top of the robotics forum.
We all have different reasons for wanting to zoom to the last post. indeed, I would say 20,000 different reasons.
And I get that "we want proof", but I think, if I managed to keep up, We are going to have to write our own proof..
-Tommy
P.S. You guys are very shiny beacons today, thank you.
As long as all the props have some form of hardware connection scheme wiring changes are not required for neurons to create or break connections to neurons on other props. In most of the research I am aware of the creation/destruction of connections is done in software.
Agreed. A Ring Topology is probably best suited for this since any "node" can send a message to any other node; any number of nodes is possible without worrying about bus arbitration or some Super HUB message space allocation, etc.... A Ring infrastructure would be fairly slow relatively speaking, but the benefits out-weigh that disadvantage. When I have time, I'll do more software in that area with TetraProp.
I still don't really get what it is about. I see a lot of humongous words and pictures showing dozens of Propeller chips on a breadboard, but what can Humanodio actually do with it? Where is the killer app? I could wire together a ton of propellers and sensors and claim that i was some kind of genius that has discovered the answer to life, but that wouldn't be fair would it? I think it is time to play your hand son!
kwinn, the thought of having the props on a bus with an inter-neuron message scheme had occurred to me. Unless the bus speed was really high, I think bus contention would slow things down to much.
Do We really expect results from this big brain? if so, Where is YOUR code that YOU introduced to this thread?
After all, He has requested help in this more than once..
I have an excuse, I can't write code to save my life... what is Your excuse?
It sounds like accusation, but it is not meant to be, and I do apologize for my poor choice of words.
It is never a great idea to generalize anything.
I have noted the contributions by many to this thread, and that is why I was so taken back by a suggestion to close.
Sorry, I panicked, and started frantically scrawling my message of shiny beacons...
kwinn, the thought of having the props on a bus with an inter-neuron message scheme had occurred to me. Unless the bus speed was really high, I think bus contention would slow things down to much.
I agree a single bus would be much too slow. I was thinking more along the line of multiple buses in a cube or hypercube arrangement. Each prop would connect to an x, y, and z bus for a cube arrangement, and routing messages from prop to prop would require going through a maximum of 3 props. So dedicate 1 cog and 3 pins for messaging while leaving the rest for I/O and simulating neurons.
IIRC our visual system inputs data from the retina at 8-12 Hz, so 7 cogs may be able to simulate several neurons at that rate.
Yikes, this is turning into quite a dogpile. Any way to reduce the boil to a mere simmer?
Erco, I don't see any profanity or name calling so far. It looks like a very good discussion so far, with both sides making some good points. This is what I consider an enjoyable topic and debate.
kwinn, any bus scheme will require each propeller chip to have a unique bus address, and each neuron will also require an address unique to a propeller. This is starting to sound similar to an IP address and port used in the TCP protocol. Once the bus topology is no longer simply connected, it will also require a message routing scheme. If your network was a tree, the propeller's address could embed the path to it. You route a message by passing it up the tree, when a node sees its address in the leading portion of the destination address it knows to begin routing down the tree.
Actually that sounds like an interesting problem, we ought to simulate it in software and see if it works.
Since the propeller doesn't have virtual memory you could use the RAM address as the port ID, as long as you never relocate neurons. In computers with virtual memory RAM address isn't unique and you don't have access to the physical address.
Is there existing work where "the RAM address" or other ID is used to convey messages? It makes sense, but I would like to be on the same page so I understand more clearly what model you're talking about.
I like the tree idea for speed. Are there redundant paths in your tree? Does the tree ever get "virtually reorganized" (Heap or Splay for example)?
jazzed, in object oriented programming the RAM address is used to select the destination. This works fine within an address space, but not across address spaces. On most modern operating systems you can create a global section which maps the same chunk of memory to multiple processes. So can you extend this across processes using this technique. There are network memory caches like Extreme Scale which let you extend this in a limited manner across physical machines. But under the covers network caches use TCP/IP.
Back to propellers, the tree would be the physical organization of the propellers, not the logical organization of the neurons within the network. For the physical network I would have all propellers have two I2C buses. A propeller is the master of its child bus, but a slave on its parent's bus. This would let you construct a tree with as many child nodes as you desire, and many levels deep as well. The thinking is that you would not reorganize the physical layout one you set it up.
A looming problem is how you assign a different address to each propeller. With TCP/IP you use either static configuration or DHCP. Static configuration would be a major headache as each propeller would need to be loaded with the same software. A DHCP like scheme might be possible via pushing down addresses from the parent. The node without a parent knows to initiate the address discovery. A parent would poll its children for messages. and in turn pass messages up to its parent based upon the routing scheme I outlined.
The next challenge would be creating the neuron objects and having them be able to wire up to each other. This would require some kind of additional discovery scheme and is frankly where I fall down on this whole things.
jazzed, in object oriented programming the RAM address is used to select the destination....
Sure, but I thought you had some insight about how this could be used for Neuron communications. A Neuron would probably not rely on some RPC mechanism since it is not described as an algorithmic entity but rather a "weight trainee" in early works. Granted I have not researched this for a while. Maybe the concept has changed. Having some reference works as a starting point would be valuable. I suppose I should do more reading in the list I posted.
The next challenge would be creating the neuron objects and having them be able to wire up to each other. This would require some kind of additional discovery scheme and is frankly where I fall down on this whole things.
This is indeed the next level up challenge. There are many nicely understood ways to do the physical layer such as your I2C tree or a Ring of Rings ... Maybe Tree-Rings ??? Ducking for cover .... I2C with Propeller as master and as slave is reasonably fast, but I think we can do better.
Layers above the physical infrastructure such as a fundamental connection protocol that allows Neural traffic are required. TCP/IP and UDP are layer 3 items and serve the upper layers well. If a Neural net is a pure stimuli/training/reaction network, I'm not sure we really need to go above the MAC Layer 2 to achieve results in a Ring.
A TCP/IP/UDP Layer 3 would probably be required for routing if a Tree is used. The thing is, scarce Propeller resources that might be used for higher protocols would remove resources required to do the interesting work.
The only problem in a Ring is the speed of data. Considering our own neural message passing is thought to be a physical chemical reaction, the speed of a Ring is probably not to bad.
Redundancy is probably not required, but if one key part fails, one could have half a brain or some larger percentage.
As far as node ID's go, for a tree approach the device Phil pointed out would be quite useful for uniqueness and quick, mostly reliable startup. However, using Rings for node assignments would be easier from a hardware perspective and redundancies would necessarily be built into the system.
I'm optimistic for experiments, but pessimistic for actually achieving and end goal of "self awareness". It's probably a waste of time, but the things discovered along the journey may be useful.
I've pointed out one Ring sharing methodology before, and it essentially says: 7 COGS are used to implement 1 to 4 neurons which pass information to/from HUB, and the 8th COG is used for handling inter-propeller messages. Now that division of labor may not be the eventual outcome, but it seems to be a fair starting point.
I'm not insanely committed to any approach though. Any other ideas on how to proceed?
My original post was inspired by jazzed's TetraProp and something I read a while back about our visual system. The gist of that article was that when our eyes detect a line at a specific angle the visual cortex fires a specific neuron or group of neurons. To me this implies that the image our eyes see is transmitted to the visual cortex as an array of some sort.
To have a neural net make sense of an image, sound, or other input I would expect that where in that array of inputs the signal comes from would also be important. With that in mind I thought an array of propellers, with each propeller having a fixed address based on it's location in the array would be the simplest solution from a hardware and software perspective.
Having an array of props along with a simple xyz bus structure will also leave a large number of pins free for I/O.
Finally, two questions. Is it worth populating the center of an array with props since it will be difficult to bring those pins to the edges for use as I/O? Can they be used for processing data?
No sign of Humanoido since this kerfuffle started.
jazzed, it's been 26 years since I've implemented a neural network, so I'm digging deep into memory. We modeled the neural network as a sparse array. Each array row was a neuron and the array columns represented connections to other neurons in the network. The input sensor data was turned into a vector and the output of the network was also a vector. Matrix multiplication was used to transform the input into the output via the neuron network matrix. We had a training phase which is how cells in the matrix were populated to represent network connections.
My thinking has been that a single propeller chip doesn't have enough RAM to hold the matrix and manipulate it. So you would need more than one chip and partition the matrix among them. You would need some kind of networking layer to enable coordination on the problem. So yes I was thinking an RPC mechanism would be used to enable the matrix operations. However, a distributed system like this is a whole lot of overhead for a math problem.
This brings me to the reason I am somewhat pessimistic about this project. An Intel Atom mini-itx board with a 1 GB stick of RAM running Linux would likely be cheaper and better suited for this task than a whole bunch of propeller chips. I'd use a single propeller chip to interface with sensors and get the data to the Atom chip which could do the heavy lifting.
Comments
But when he reports actual progress, without sufficient detail to reproduce his accomplishment, then dodges direct questions about said progress, doesn't show any source code, and wages personal attacks upon those who express doubt, I begin to smell something fishy. I'd love to give Humanoido the benefit of the doubt. But if he expects to continue stringing us along for another 800 posts, it's got to be with something more substantial than what we've seen so far: i.e. schematics and original source code -- even if it's for something he tried that didn't work.
Am I expecting too much? If there's strong consensus that I am, I'll let it go. I'm not really enjoying this role one bit.
-Phil
He seems to think that by adding more and more Propeller chips his contraption will start doing something useful, in fact he is wasting his time and money. He is giving the rest of us some amusement, however.
As I pointed out eariler in this thread, most neural nets are simulated in software. This is because they work by re-wiring connections between neurons in response to an input trying to produce a desired output. Any two neurons in the net can make or break a connnection as a result of multiple rounds of training. So if you have neurons on separate propellers that want to form a connection it would require a change to the physical wiring repeated several times. This is an impractical way to train a neural net.
IMO this has nothing to do with stifling creativity and all to do with how we behave as forum members.
How about some ground rules here?
What is the cutoff point for the number of post's in a thread like this? After all, the freudian slips start way way back,
so how long should one wait before they "call" or go "all in" to stop a thread like this?
What was it that made this thread different? Why would it be allowed to go on so long?
and why stop now? is it because the results are not instant in gratification?
Is this big brain impossible to achieve even half of what has been suggested?
In other words, Is half of what Humanoido writes down true?
Do We really expect results from this big brain? if so, Where is YOUR code that YOU introduced to this thread?
After all, He has requested help in this more than once..
I have an excuse, I can't write code to save my life... what is Your excuse?
I have "training" in Reading between the lines, Yes, there is a small confidence game afoot,
Some paraphrased white papers, duplicate pictures, "stream of conscience" posting style,
and we won't be seeing anything from that "Failed Hard Drive", and as for moving to the states, well....
That seems to be beside the point for now..
What is needed now is YOUR Code input...
If this sounds a little bit like hero worship, Well... I am allways impressed by someone who can carry
a thought for more than paragraph...
-Tommy
However, what I do appreciate is that a lot of creativity occurs intuitively. Sometimes the best ideas in the world can spring up from people just goofing around, not really knowing what the heck they are doing, but following an instinctive drive to do something - anything! - besides sitting around and watching TV.
I may have some doubts about this approach producing any results that could be described as intelligent or conscious behavior but I see no reason to force a stop to this thread. Unless I missed something Humanoido has never been rude, abusive, or tried to con anyone, at least no more so than many others on the forum. Many discoveries have been the serendipitous result of research into something else.
@ Martin_H,
As long as all the props have some form of hardware connection scheme wiring changes are not required for neurons to create or break connections to neurons on other props. In most of the research I am aware of the creation/destruction of connections is done in software.
@ Humanoido,
It would be nice to see some concrete information about hardware, inter-prop physical connections, neuron software, how neuron connections are made, etc.
Again (since I must not be expressing myself well), I have no quarrel with this thread in general, its length, its placement in the Robitics forum, Humanoido's objectives, or even his methods. Maybe something will come of it; maybe it won't. That's part of the grand adventure that he's sharing with us. And -- let's face it -- with more than 20,000 views, this thread has attracted a lot of interested readers. At least he now includes citations to the external references he uses, and I laud him for finally recognizing that necessity.
My point is not about objectives or methods or wanting to stifle creativity -- none of that. It's simply about honest reporting of any progress that's made and the evidence required to demonstrate it. When presenting such evidence, schematics are the hieroglyphs we all understand, and source code is our lingua franca. These are the communication tools that cut through the more-than-ample hyperbole afoot here and which would assure me, at least, that any reported progress is real. Unfortunately, though, that crucial element has not only been missing, but asking for it results in evasion, more smoke, and playground-style taunts. Hence my incredulity.
-Phil
Hard to imagine logging in and not see the "Fill the Big Brain" post way up near the top of the robotics forum.
We all have different reasons for wanting to zoom to the last post. indeed, I would say 20,000 different reasons.
And I get that "we want proof", but I think, if I managed to keep up, We are going to have to write our own proof..
-Tommy
P.S. You guys are very shiny beacons today, thank you.
That might be it. If nothing else, Humanoido has succeeded in creating a PPBPACYM (Phil Pilgrim Button-Pushing And Chain-Yanking Machine).
-Phil
Ttailspin, we've all contributed code and knowledge to this adventure!
I feel it's in bad taste to make claims that you know are not true.
It is never a great idea to generalize anything.
I have noted the contributions by many to this thread, and that is why I was so taken back by a suggestion to close.
Sorry, I panicked, and started frantically scrawling my message of shiny beacons...
-Tommy
I agree a single bus would be much too slow. I was thinking more along the line of multiple buses in a cube or hypercube arrangement. Each prop would connect to an x, y, and z bus for a cube arrangement, and routing messages from prop to prop would require going through a maximum of 3 props. So dedicate 1 cog and 3 pins for messaging while leaving the rest for I/O and simulating neurons.
IIRC our visual system inputs data from the retina at 8-12 Hz, so 7 cogs may be able to simulate several neurons at that rate.
Erco, I don't see any profanity or name calling so far. It looks like a very good discussion so far, with both sides making some good points. This is what I consider an enjoyable topic and debate.
Actually that sounds like an interesting problem, we ought to simulate it in software and see if it works.
I often thought that the Spinneret would make a much better brain. One Propeller could have access to many TCP/UDP devices. Heck the whole internet.
I like the tree idea for speed. Are there redundant paths in your tree? Does the tree ever get "virtually reorganized" (Heap or Splay for example)?
Back to propellers, the tree would be the physical organization of the propellers, not the logical organization of the neurons within the network. For the physical network I would have all propellers have two I2C buses. A propeller is the master of its child bus, but a slave on its parent's bus. This would let you construct a tree with as many child nodes as you desire, and many levels deep as well. The thinking is that you would not reorganize the physical layout one you set it up.
A looming problem is how you assign a different address to each propeller. With TCP/IP you use either static configuration or DHCP. Static configuration would be a major headache as each propeller would need to be loaded with the same software. A DHCP like scheme might be possible via pushing down addresses from the parent. The node without a parent knows to initiate the address discovery. A parent would poll its children for messages. and in turn pass messages up to its parent based upon the routing scheme I outlined.
The next challenge would be creating the neuron objects and having them be able to wire up to each other. This would require some kind of additional discovery scheme and is frankly where I fall down on this whole things.
This is indeed the next level up challenge. There are many nicely understood ways to do the physical layer such as your I2C tree or a Ring of Rings ... Maybe Tree-Rings ??? Ducking for cover .... I2C with Propeller as master and as slave is reasonably fast, but I think we can do better.
Layers above the physical infrastructure such as a fundamental connection protocol that allows Neural traffic are required. TCP/IP and UDP are layer 3 items and serve the upper layers well. If a Neural net is a pure stimuli/training/reaction network, I'm not sure we really need to go above the MAC Layer 2 to achieve results in a Ring.
A TCP/IP/UDP Layer 3 would probably be required for routing if a Tree is used. The thing is, scarce Propeller resources that might be used for higher protocols would remove resources required to do the interesting work.
The only problem in a Ring is the speed of data. Considering our own neural message passing is thought to be a physical chemical reaction, the speed of a Ring is probably not to bad.
Redundancy is probably not required, but if one key part fails, one could have half a brain or some larger percentage.
As far as node ID's go, for a tree approach the device Phil pointed out would be quite useful for uniqueness and quick, mostly reliable startup. However, using Rings for node assignments would be easier from a hardware perspective and redundancies would necessarily be built into the system.
I'm optimistic for experiments, but pessimistic for actually achieving and end goal of "self awareness". It's probably a waste of time, but the things discovered along the journey may be useful.
I've pointed out one Ring sharing methodology before, and it essentially says: 7 COGS are used to implement 1 to 4 neurons which pass information to/from HUB, and the 8th COG is used for handling inter-propeller messages. Now that division of labor may not be the eventual outcome, but it seems to be a fair starting point.
I'm not insanely committed to any approach though. Any other ideas on how to proceed?
My research has been rather light but I believe a neuron architecture would work nicely for certain sensory systems like vision. Maybe not all though.
I've probably been working too much with the Spinneret lately but heck TCP/UPD works great.
To have a neural net make sense of an image, sound, or other input I would expect that where in that array of inputs the signal comes from would also be important. With that in mind I thought an array of propellers, with each propeller having a fixed address based on it's location in the array would be the simplest solution from a hardware and software perspective.
Having an array of props along with a simple xyz bus structure will also leave a large number of pins free for I/O.
Finally, two questions. Is it worth populating the center of an array with props since it will be difficult to bring those pins to the edges for use as I/O? Can they be used for processing data?
jazzed, it's been 26 years since I've implemented a neural network, so I'm digging deep into memory. We modeled the neural network as a sparse array. Each array row was a neuron and the array columns represented connections to other neurons in the network. The input sensor data was turned into a vector and the output of the network was also a vector. Matrix multiplication was used to transform the input into the output via the neuron network matrix. We had a training phase which is how cells in the matrix were populated to represent network connections.
My thinking has been that a single propeller chip doesn't have enough RAM to hold the matrix and manipulate it. So you would need more than one chip and partition the matrix among them. You would need some kind of networking layer to enable coordination on the problem. So yes I was thinking an RPC mechanism would be used to enable the matrix operations. However, a distributed system like this is a whole lot of overhead for a math problem.
This brings me to the reason I am somewhat pessimistic about this project. An Intel Atom mini-itx board with a 1 GB stick of RAM running Linux would likely be cheaper and better suited for this task than a whole bunch of propeller chips. I'd use a single propeller chip to interface with sensors and get the data to the Atom chip which could do the heavy lifting.