Thanks @cgracey ! It's all powered by your Propellers! A Raspberry Pi sends the audio to a FM radio transmitter. It also sends E1.31 ethernet packets synchronized to the music to the 8 P2 light controllers I have throughout the yard. The 4 singing trees are all powered by two P1s. They get DMX512 data from the P2 controller running the arches. There is another 1200 pixel matrix, 350ft of lights on the fence, and 8 tree wraps that just wouldn't fit in frame for the video.
A single P2 could run everything, but the display is spread out over about 36,000sq ft. Power distribution and ws2811 signaling is the limiting factor. It's just easier, cheaper, and more reliable to distribute those with ethernet cords, 110v extension cords and additional P2 controllers.
The camera just doesn't capture how crisp it all looks. By the time you adjust the camera to capture the color, you lose the temporal detail of the lighting. The entire display plays at 40 frames per second. It looks incredibly fluid. Better than I had ever hoped.
The fun that I have had learning to program the P1 and P2 to do this stuff has truly been one of the best joys of my life.
Cool! I love working on this stuff, too. It's like playing Sudoku everyday, but you get to invent the solution to a problem, which is like a puzzle, in itself, and then you solve the puzzle. Lots of creative adventure.
I wish we could effectively monetize that experience so we could build more chips. It would be great to have a P2 that runs at maybe two GHz and has tons of memory, more cogs, and even better analog.
Congratulations. This is really impressive, not only the great show itself but also the story behind it.
@ke4pjw said:
The fun that I have had learning to program the P1 and P2 to do this stuff has truly been one of the best joys of my life.
Being creative is one of the basic needs of humans. The Propeller is not only a very good example of an end product of such creativeness but also a great tool for others to be creative.
It's that time of year again! I have worked up a version 2 of the Blitzen 24 Edge. These are the fixes and additions.
1) Crosstalk between the ethernet and outputs 16-24. Now using 4 layer board with ground and power planes. (fix)
2) Reset of the P2 does not power cycle the SD card. Now using the 5V regulator enable pin with a switch to powercycle the Edge. (fix)
3) Not enough thermal dissipation on power inputs. (fix)
4) OLED display screen burn-in. (fix)
5) Screw Terminals on power inputs. (new)
6) Rotary Encoder with button for diagnostic menu selections. (new)
7) Remove need for 12V/5V jumper and remove optional separate power input for controller circuitry (new)
8) PCB now has standard 4MM holes for standoffs.
Also, this is a new layout using KiCAD. I love KiCAD. Unfortunately I had to increase the board size by 5MM. It may require some modification to the ChannelMaster enclosures to fit properly.
Got components in for the V2 boards. So far I have confirmed that the OLED and Ethernet are wired correctly. I also confirmed that my reset scheme works! The reset button power cycles every component on the board, not just the P2.
@Rayman I modified the Microbit footprint. Unfortunately, it was a right angle, so in addition to enumerating the pins in the schematic, I had to modify the silkscreen too. I will dig it up after work and post it here.
I will say that KiCad is nice in that it kind of forces you to do the schematic first. I came from using ExpressPCB/ExpressPCB+. After doing a couple boards in KiCad, man, I really love it. The only thing I miss is the multiple tabs for schematics. It has similar functionality but is too "configurable" for my taste.
Well, the 4 layer board almost solved my "glitching". Whenever I started the cog that outputs P32-P39, it caused interference (somehow) with the ethernet which resulted on all outputs. I believe this caused issues with ethernet, even though this is a 4 layer board and all traces for ethernet are on the top (red) side of the board. The fix for this was to use less drive on both ethernet and pixel output. I don't understand how those two could interact, because there is a solid groundplane and powerplane between them.
The only remaining issue I have is Port 24 (P39) It glitches severely. Almost like it has a bad connection. I replaced the driver IC to make sure it wasn't a problem and reflowed the solder joints. Using a bodge wire solves the problem. Also placing my finger on the bottom of the board just below the edge connector where the P32-P38 traces are solves the issue.
Port 24 (P39) is the victim with the aggressors are P32-P38. I just can't figure out why P0-P15 are A OK, but these aren't. Maybe it is the proximity of P32 to the ground side of the reset button.
If anyone could tell me where I messed up, I would greatly appreciate it.
Is p39 driven directly or though a pull-up ? Maybe adding a load resistor close to the chip it’s driving helps? If it’s a slow signal there maybe some capacitive load too…
Although a 4 layer board has better shielding against capacitive crosstalk it means less distance to the ground plane and therefore higher parasitic capacitance and higher peak currents when signals are switching. And you can't shield magnetic cross-talk easily. So even with 4 layer boards it's sometimes not good to route sensitive signals with long traces in parallel.
So with "ethernet signals" you mean the data bus going to the WIZ module? Reducing the drive strength on the data bus generally helps as you already found out. On my CNC board I had to insert series termination resistors between the P2 and the SPI bus lines. This reduces the slopes of the switching edges and reduces crosstalk and ringing. Unfortunatelly the P2 has no series termination in the range of 100 Ohm built into the smart pins. the 1.5k resistors are a bit high and might slow down timing critical signals too much. So I inserted hard wired 68 Ohm resistors which give a good match to a 100 Ohm line together with the ~30 Ohm internal drive resistance of the MOSFETs.
I'd try to analyze the signals with a scope so you can optimize the terminations to get the optimum minimum edge slope vs. minimum delay balance.
@Rayman said:
Is p39 driven directly or though a pull-up ? Maybe adding a load resistor close to the chip it’s driving helps? If it’s a slow signal there maybe some capacitive load too…
P39 is directly driving a SN74ACT241N. It is a non-inverting buffer used to act as a buffer and bring the output up to 5V. The signals going into this IC are 800kHz. It just seems really odd that I have two other circuits exactly like this one that has no problems. The only difference is all 8 of the inputs to this IC reside on the same side of the board, whereas the other two IC like this have 4 on top and 4 on bottom.
@Rayman said:
Also make sure it’s not a power supply issue…
I looked at this, but dismissed it. I will scope it again and take a look.
@Rayman said:
Are the signal layers next to each other?
If so and doing it again, might want to put a ground or power layer between them...
Stackup is signal (top), Ground, Power, signal (bottom)
Groundplane is a single solid layer.
Powerplane is separated where my fuses are. This is to accommodate 12v and 5v. All signals on the bottom have a solid 5V powerplane.
@ManAtWork said:
Although a 4 layer board has better shielding against capacitive crosstalk it means less distance to the ground plane and therefore higher parasitic capacitance and higher peak currents when signals are switching. And you can't shield magnetic cross-talk easily. So even with 4 layer boards it's sometimes not good to route sensitive signals with long traces in parallel.
This must be the problem. My concern was the return paths (or lack of) which usually are the cause of issues. I was surprised the issue is with the slower 800kHz signals.
So with "ethernet signals" you mean the data bus going to the WIZ module? Reducing the drive strength on the data bus generally helps as you already found out. On my CNC board I had to insert series termination resistors between the P2 and the SPI bus lines. This reduces the slopes of the switching edges and reduces crosstalk and ringing. Unfortunatelly the P2 has no series termination in the range of 100 Ohm built into the smart pins. the 1.5k resistors are a bit high and might slow down timing critical signals too much. So I inserted hard wired 68 Ohm resistors which give a good match to a 100 Ohm line together with the ~30 Ohm internal drive resistance of the MOSFETs.
Luckily I don't see any problems with the ethernet, other than it was a victim. Those are all on the top side of the board, closest to the ground plane. I still don't see how they could possibly be victimized by the traces on the bottom.
I'd try to analyze the signals with a scope so you can optimize the terminations to get the optimum minimum edge slope vs. minimum delay balance.
Clearly I have more learning to do
I appreciate everyone's feedback. I was just surprised to have the have the same issues on the same pins with the new layout. I am starting to think my use of pins P32-P39 are cursed.
@Rayman said:
if those red and blue traces are separated by power and ground planes, very hard to imagine EMI type things...
I'd guess power supply or code problem.
So I have single COGs that drives 8 outputs. The cogs are started and stopped depending on if the output is enabled in the configuration. As soon as the COG that drives P32-P39 starts, that is when "glitching" starts. Maybe it has to do with something in my code and the outa/outb boundary. I investigated that last night, but didn't get anything fixed until I turn down the drive.
Placing my fat finger on those 6 lines where I placed the arrow for the aggressors makes the problem disappear. So, I am leaning tward my inexperience in layout. But I will dig further. I am going to comb my code and config to see if another may have ahold of P39. It makes no sense to me that P38 is A-OK but P39 is getting crosstalk. P39 is further away than any of the others.
@Rayman said:
Did just notice something a little suspect in the layout...
On IC4, the wire to pin 13 looks perhaps too close to pins 11 and 12...
Is this withing the keepout? Could it actually be touching?
Oh man, I thought I had corrected that. I must have ctrl-zed too much at some point. Fixing that in KiCAD now.
They are not touching and do not appear to interfere with each other based on my testing. (I pain painstakingly tested each port in multiple configs looking for cross talk)
I appreciate the help. When you spend hours and hours trying to figure a problem like this, it's frustrating.
This is a mystery then... Touching with finger is an interesting thing. Sure not also touching pins of Edge connector?
If not, seems like adding a tiny capacitance, say 10 nF or so to the problem pins should also fix it.
I'm guessing you don't have an o'scope? This is when I'd dig that out...
Any chance the code is leaving the problem pins floating for a while?
I have one, but looking at that signaling doesn't reveal much, as the triggering is difficult to set on this type of signal. It is analog. Looking to get a digital scope. I think that might better help capture and review the problem.
Touching the pins on the Edge does nothing. Touching the other side of the board in the same area does nothing. If I add a length of wire to P39 at the Edge connector, the problem also goes away. A bodge wire solves the problem. I just can't see what I could have done in the layout to make this happen.
I'm not seeing any other layout issues... I think the code must be letting the pin float somehow...
Guess it's easy to make a mistake with "addpins". Might look for any of those...
I think, debugging this with an oscilloscope is the only way to avoid blind-guessing and shooting into the dark. With a scope it should be relatively clear to identify the reason and a countermeasure for the interference. For example, capacitive cross-talk can be idetified by an overshoot at the victim signal while the agressor has a rising edge. If it's a supply problem there should be a downward dent in the victim signal, instead.
It's difficult to trigger on the actual error condition in real data streams, though. I'd suggest generating worst-case patterns, for example output rising edges on all agressor signals simultanously with different phases of the victim signal.
Placing my fat finger on those 6 lines where I placed the arrow for the aggressors makes the problem disappear
Fingers add a bit of capacitance and a weak pullup or pulldown resistor. That can dampen crosstalk or ringing. I also often had the case that touching a signal with the scope probe does cause the problem to go away. This is of course annoying while trying to debug something. But you could still see over/undershoot or crosstalk, it just becomes smaller.
@ke4pjw said:
@cgracey saved Christmas! WRPIN ##$00000900 is the secret sauce for curing my glitches on P39! Thank you so much for your time and advice today!
So $900 is "drive high 1.5k" and "drive low 1.5k", IIRC? Good to hear you got it fixed!
@ManAtWork said:
So $900 is "drive high 1.5k" and "drive low 1.5k", IIRC? Good to hear you got it fixed!
Yep! The "drive low 1.5K" was what I was missing. I didn't even know that was a thing. So the transitions going from low to high were what was causing "ringing" I suppose. I also suspect the geometry of P39 being as far away from the ground pin on the Edge connector played a part. It is my understanding that the return current path will follow under the trace. That means it has to wind its way under all of the pins to get from P39 to the GND path on the Edge card connector.
Well, now that I have the outputs sorted, thanks to Chip, I am on my way to a 2.0 release. I am naming this release "Rocklin". The 1.x was named "Music City", where I designed it. I want to keep with a theme of all the places people are at that helped make this thing a reality.
The rotary encoder selects between 3 menus.
1) Displays IP, Ethernet Status, E1.31 Status, Dropped packets, Decoded Packets, Mac Address
2) QR Code with a link to the Web Interface.
3) Software Version.
The OLED now powers off after 30 seconds. When the rotary encoder selects a new menu or the button is pushed, it will power back on.
Here is a quick video of me using the new features:
Comments
Just fabulous, Terry
Thanks @Tubular !
It wouldn't be Christmas without showing off a Christmas song. So, this is my submission to the Xlights Around The World Project
Now I am off to finish version 2 of the light controller.
That's a pretty amazing presentation! Good job, Terry!
Thanks @cgracey ! It's all powered by your Propellers! A Raspberry Pi sends the audio to a FM radio transmitter. It also sends E1.31 ethernet packets synchronized to the music to the 8 P2 light controllers I have throughout the yard. The 4 singing trees are all powered by two P1s. They get DMX512 data from the P2 controller running the arches. There is another 1200 pixel matrix, 350ft of lights on the fence, and 8 tree wraps that just wouldn't fit in frame for the video.
A single P2 could run everything, but the display is spread out over about 36,000sq ft. Power distribution and ws2811 signaling is the limiting factor. It's just easier, cheaper, and more reliable to distribute those with ethernet cords, 110v extension cords and additional P2 controllers.
The camera just doesn't capture how crisp it all looks. By the time you adjust the camera to capture the color, you lose the temporal detail of the lighting. The entire display plays at 40 frames per second. It looks incredibly fluid. Better than I had ever hoped.
The fun that I have had learning to program the P1 and P2 to do this stuff has truly been one of the best joys of my life.
Merry Christmas!
--Terry
Cool! I love working on this stuff, too. It's like playing Sudoku everyday, but you get to invent the solution to a problem, which is like a puzzle, in itself, and then you solve the puzzle. Lots of creative adventure.
I wish we could effectively monetize that experience so we could build more chips. It would be great to have a P2 that runs at maybe two GHz and has tons of memory, more cogs, and even better analog.
Congratulations. This is really impressive, not only the great show itself but also the story behind it.
Being creative is one of the basic needs of humans. The Propeller is not only a very good example of an end product of such creativeness but also a great tool for others to be creative.
It's that time of year again! I have worked up a version 2 of the Blitzen 24 Edge. These are the fixes and additions.
1) Crosstalk between the ethernet and outputs 16-24. Now using 4 layer board with ground and power planes. (fix)
2) Reset of the P2 does not power cycle the SD card. Now using the 5V regulator enable pin with a switch to powercycle the Edge. (fix)
3) Not enough thermal dissipation on power inputs. (fix)
4) OLED display screen burn-in. (fix)
5) Screw Terminals on power inputs. (new)
6) Rotary Encoder with button for diagnostic menu selections. (new)
7) Remove need for 12V/5V jumper and remove optional separate power input for controller circuitry (new)
8) PCB now has standard 4MM holes for standoffs.
Also, this is a new layout using KiCAD. I love KiCAD. Unfortunately I had to increase the board size by 5MM. It may require some modification to the ChannelMaster enclosures to fit properly.
Should be testing next weekend.
Nicely done, Terry!
Got components in for the V2 boards. So far I have confirmed that the OLED and Ethernet are wired correctly. I also confirmed that my reset scheme works! The reset button power cycles every component on the board, not just the P2.
Looks good!
I’ll have to try Kicad soon…
How did you get the edge connector?
Was it there already or did somebody here make it?
@Rayman I modified the Microbit footprint. Unfortunately, it was a right angle, so in addition to enumerating the pins in the schematic, I had to modify the silkscreen too. I will dig it up after work and post it here.
I will say that KiCad is nice in that it kind of forces you to do the schematic first. I came from using ExpressPCB/ExpressPCB+. After doing a couple boards in KiCad, man, I really love it. The only thing I miss is the multiple tabs for schematics. It has similar functionality but is too "configurable" for my taste.
Well, the 4 layer board almost solved my "glitching". Whenever I started the cog that outputs P32-P39, it caused interference (somehow) with the ethernet which resulted on all outputs. I believe this caused issues with ethernet, even though this is a 4 layer board and all traces for ethernet are on the top (red) side of the board. The fix for this was to use less drive on both ethernet and pixel output. I don't understand how those two could interact, because there is a solid groundplane and powerplane between them.
The only remaining issue I have is Port 24 (P39) It glitches severely. Almost like it has a bad connection. I replaced the driver IC to make sure it wasn't a problem and reflowed the solder joints. Using a bodge wire solves the problem. Also placing my finger on the bottom of the board just below the edge connector where the P32-P38 traces are solves the issue.
Port 24 (P39) is the victim with the aggressors are P32-P38. I just can't figure out why P0-P15 are A OK, but these aren't. Maybe it is the proximity of P32 to the ground side of the reset button.
If anyone could tell me where I messed up, I would greatly appreciate it.
Is p39 driven directly or though a pull-up ? Maybe adding a load resistor close to the chip it’s driving helps? If it’s a slow signal there maybe some capacitive load too…
Also make sure it’s not a power supply issue…
Although a 4 layer board has better shielding against capacitive crosstalk it means less distance to the ground plane and therefore higher parasitic capacitance and higher peak currents when signals are switching. And you can't shield magnetic cross-talk easily. So even with 4 layer boards it's sometimes not good to route sensitive signals with long traces in parallel.
So with "ethernet signals" you mean the data bus going to the WIZ module? Reducing the drive strength on the data bus generally helps as you already found out. On my CNC board I had to insert series termination resistors between the P2 and the SPI bus lines. This reduces the slopes of the switching edges and reduces crosstalk and ringing. Unfortunatelly the P2 has no series termination in the range of 100 Ohm built into the smart pins. the 1.5k resistors are a bit high and might slow down timing critical signals too much. So I inserted hard wired 68 Ohm resistors which give a good match to a 100 Ohm line together with the ~30 Ohm internal drive resistance of the MOSFETs.
I'd try to analyze the signals with a scope so you can optimize the terminations to get the optimum minimum edge slope vs. minimum delay balance.
Are the signal layers next to each other?
If so and doing it again, might want to put a ground or power layer between them...
P39 is directly driving a SN74ACT241N. It is a non-inverting buffer used to act as a buffer and bring the output up to 5V. The signals going into this IC are 800kHz. It just seems really odd that I have two other circuits exactly like this one that has no problems. The only difference is all 8 of the inputs to this IC reside on the same side of the board, whereas the other two IC like this have 4 on top and 4 on bottom.
I looked at this, but dismissed it. I will scope it again and take a look.
Stackup is signal (top), Ground, Power, signal (bottom)
Groundplane is a single solid layer.
Powerplane is separated where my fuses are. This is to accommodate 12v and 5v. All signals on the bottom have a solid 5V powerplane.
This must be the problem. My concern was the return paths (or lack of) which usually are the cause of issues. I was surprised the issue is with the slower 800kHz signals.
Luckily I don't see any problems with the ethernet, other than it was a victim. Those are all on the top side of the board, closest to the ground plane. I still don't see how they could possibly be victimized by the traces on the bottom.
Clearly I have more learning to do
I appreciate everyone's feedback. I was just surprised to have the have the same issues on the same pins with the new layout. I am starting to think my use of pins P32-P39 are cursed.
if those red and blue traces are separated by power and ground planes, very hard to imagine EMI type things...
I'd guess power supply or code problem.
Did just notice something a little suspect in the layout...
On IC4, the wire to pin 13 looks perhaps too close to pins 11 and 12...
Is this withing the keepout? Could it actually be touching?
So I have single COGs that drives 8 outputs. The cogs are started and stopped depending on if the output is enabled in the configuration. As soon as the COG that drives P32-P39 starts, that is when "glitching" starts. Maybe it has to do with something in my code and the outa/outb boundary. I investigated that last night, but didn't get anything fixed until I turn down the drive.
Placing my fat finger on those 6 lines where I placed the arrow for the aggressors makes the problem disappear. So, I am leaning tward my inexperience in layout. But I will dig further. I am going to comb my code and config to see if another may have ahold of P39. It makes no sense to me that P38 is A-OK but P39 is getting crosstalk. P39 is further away than any of the others.
Oh man, I thought I had corrected that. I must have ctrl-zed too much at some point. Fixing that in KiCAD now.
They are not touching and do not appear to interfere with each other based on my testing. (I pain painstakingly tested each port in multiple configs looking for cross talk)
I appreciate the help. When you spend hours and hours trying to figure a problem like this, it's frustrating.
PS: That did not trip my keepout rules, which were set to what the manufacturer can do, however better safe than sorry.
This is a mystery then... Touching with finger is an interesting thing. Sure not also touching pins of Edge connector?
If not, seems like adding a tiny capacitance, say 10 nF or so to the problem pins should also fix it.
I'm guessing you don't have an o'scope? This is when I'd dig that out...
Any chance the code is leaving the problem pins floating for a while?
I have one, but looking at that signaling doesn't reveal much, as the triggering is difficult to set on this type of signal. It is analog. Looking to get a digital scope. I think that might better help capture and review the problem.
Touching the pins on the Edge does nothing. Touching the other side of the board in the same area does nothing. If I add a length of wire to P39 at the Edge connector, the problem also goes away. A bodge wire solves the problem. I just can't see what I could have done in the layout to make this happen.
I'm not seeing any other layout issues... I think the code must be letting the pin float somehow...
Guess it's easy to make a mistake with "addpins". Might look for any of those...
I think, debugging this with an oscilloscope is the only way to avoid blind-guessing and shooting into the dark. With a scope it should be relatively clear to identify the reason and a countermeasure for the interference. For example, capacitive cross-talk can be idetified by an overshoot at the victim signal while the agressor has a rising edge. If it's a supply problem there should be a downward dent in the victim signal, instead.
It's difficult to trigger on the actual error condition in real data streams, though. I'd suggest generating worst-case patterns, for example output rising edges on all agressor signals simultanously with different phases of the victim signal.
Fingers add a bit of capacitance and a weak pullup or pulldown resistor. That can dampen crosstalk or ringing. I also often had the case that touching a signal with the scope probe does cause the problem to go away. This is of course annoying while trying to debug something. But you could still see over/undershoot or crosstalk, it just becomes smaller.
@cgracey saved Christmas! WRPIN ##$00000900 is the secret sauce for curing my glitches on P39! Thank you so much for your time and advice today!
So $900 is "drive high 1.5k" and "drive low 1.5k", IIRC? Good to hear you got it fixed!
Yep! The "drive low 1.5K" was what I was missing. I didn't even know that was a thing. So the transitions going from low to high were what was causing "ringing" I suppose. I also suspect the geometry of P39 being as far away from the ground pin on the Edge connector played a part. It is my understanding that the return current path will follow under the trace. That means it has to wind its way under all of the pins to get from P39 to the GND path on the Edge card connector.
Seems strange that that would help, but glad you got it working!
Well, now that I have the outputs sorted, thanks to Chip, I am on my way to a 2.0 release. I am naming this release "Rocklin". The 1.x was named "Music City", where I designed it. I want to keep with a theme of all the places people are at that helped make this thing a reality.
The rotary encoder selects between 3 menus.
1) Displays IP, Ethernet Status, E1.31 Status, Dropped packets, Decoded Packets, Mac Address
2) QR Code with a link to the Web Interface.
3) Software Version.
The OLED now powers off after 30 seconds. When the rotary encoder selects a new menu or the button is pushed, it will power back on.
Here is a quick video of me using the new features: