Shop Learn P1 Docs P2 Docs
Using a Raspberry Pi as an IoT gateway for our P2 Projects — Parallax Forums

Using a Raspberry Pi as an IoT gateway for our P2 Projects

Stephen MoracoStephen Moraco Posts: 202
edited 2022-01-19 23:38 in Propeller 2

I introduced a new project during our Live Forum today.

Initial features like sending email, sending named values to the attached RPi, and reading named values from the RPi as viewing P2 values from the web backend of the RPi are all working.

I'm posting this here so we have a thread where we all can comment.

Any model RPi should work as long as it is running Raspberry Pi OS.

My git repo ironsheep/P2-RPi-IoT-gateway is live! You can find:

  • RPi Setup Instructions
  • Installation Instructions
  • Demos for each working feature (send email, display 1-wire on a web-page, etc.)
  • Upcoming features - I'm awakening these one-by-one - keep an eye on the repo!

On the future features list are:

  • SMS
  • Twitter
  • MQTT
  • MQTT w/Advertisement (for use with Home Assistant)

My existing Home Assistant Projects have shown me already how to add MQTT/HA support. ;-)

Comments

  • JonnyMacJonnyMac Posts: 8,163

    Sorry, I missed it -- very busy with work. Looking forward to giving it a try.

  • pik33pik33 Posts: 1,735
    edited 2022-01-20 07:21

    I have the same idea, but without RPi OS. I use Ultibo instead, a bare metal environment for Raspberry Pi. . I connected the Pi to P2 via UART and it works as a keyboard and mouse interface. The Ultibo powered RPi can be also a mass storage, Ethernet and Wifi interface for a P2.

    While we have less flexibility using Ultibo than the OS - no Python with its libraries, etc - the big advantages of OS free RPi are (1) fast - 3-5 seconds - boot (2) no SD wear/system shutdown problems. You can simply switch it off as the Ultibo doesn't write anything on the SD on its own.

  • @pik33 Useful to know, thank you!
    It would be fun to see a project contributed with all the necessary effort to get that environment running described for us with step-by-step instructions. Then our community would have alternate choices when bringing up a gateway. There are a number of mentions of Ultibo but I've not seen any contributions yet.

    I'm focused on this approach as, in my past, I spent years as a Debian developer so I have a lot of experience with this OS. (Those packages that we install with apt? I created and maintained a number of them.) I also have a small farm of RPi's running this way in my house. So I use, write about, and teach what I know best. As you can see here: Prior RPi work I have many open-source projects for this environment some of which have thousands of users.

    Regarding the uSD write issues, I simply use a high write capability uSD card in RPi's where I need to care about such things. (Like using one as a security cam, etc.). This is mentioned in my RPi setup instructions.

    Again, thanks for mentioning this alternative!

  • pik33pik33 Posts: 1,735
    edited 2022-01-25 12:17

    I have a project open, but this is the very early stage.

    https://github.com/pik33/P2-retromachine

    As it is today, there is Ultibo directory in this repository. It contains the source code for Ultibo/RPi Zero and, in the Release directory, 4 files to copy onto a blank FAT32 formatted SD. These are 3 basic firmware files and a custom compiled kernel.img. These are all files needed to run the interface - the program works as a keyboard and a mouse interface for a P2. RPi zero has to be connected to a P2 via its UART which works at 1.92 Mbps

    The P2 side of this interface is 'retrocog.spin2' file in Propeller directory. It runs at its own cog, receiving data from the interface and also from standard Flexprop serial terminal (at 2 Mbps) and a MIDI breakout board. To test this, received data are available at (unused in Flexprop) hub addresses $30, $34, $38, $3C. The main program has to check if there is non-zero value. If yes, then read it and write zero, then the retrocog will place the next data from its buffer, if there are any.

    I will create a topic for this project when I manage to solder the device for it: I 3D-printed a box for a P2 Edge, Zero 2, MIDI breakout and a lot of DB connectors to play. To do is to solder all the planned wires and do some basic tests.


    Ultibo as the envirinment can be installed on Windows machine (easiest way: use setup.exe) or Linux (using the install script) - including a Raspberry Pi OS. I use the RPi: a shell script can rename Linux kernel and replace with Ultibo one, then reboot. Ultibo program can then restore Linux kernel before rebooting - no SD card swapping needed until a bug prevents the program from restoring Linux kernel.

    I managed to write an audio and camera drivers for Ultibo: https://github.com/pik33/SimpleAudio , https://github.com/pik33/SimpleCamera

  • hinvhinv Posts: 1,149

    @"Stephen Moraco" This looks like quite a useful tool. Will this gateway allow the pi to boot the P2 via the pi UART? If so, I could run FlexProp on the Pi, and that would open up a lot of possibilities.

    @pik33 same question to you.

  • pik33pik33 Posts: 1,735
    edited 2022-07-24 16:55

    To run flexprop with the Pi you need a linux running on the Pi and connect the Pi via P62,P63. Flexprop has way too many dependencies to port it to Ultibo :( but with Raspbian it works well. I already have a Pi configured for P2 programming.

    However I use a configuration which allows reboot from Ultibo to Raspbian and from Raspbian to Ultibo, so yes, I can do this.

    The device I mentioned before is still not completed. I was unable to get PSRAM working and I didn't have EC32. Now I have the EC32, and there are CLK/3 drivers available. Maybe I will try to use 4 chips and CLK/3 in the device.

    The idea of using RPi+P2 combo in one device and programming the P2 with a Pi is interesting, but I think I need a more powerful Pi than a Zero. Either Zero 2 or a full 3B+. The main problem is you cannot buy any Raspberry Pi now, but I have several Pis to experiment with. I have a Zero 2 - moving to Zero 2 doesn't need to print a new box. Simply replace a Pi. There is however a bug in Ultibo which affects RPi Zero 2 and its USB subsystem. The bug is reported and confirmed, so I am waiting for the patch.

    I have several new ideas now :)

    Maybe a PCB with an edge connector for P2 and 40-pin connector for a Pi?

  • @pik33 said:
    To run flexprop with the Pi you need a linux running on the Pi and connect the Pi via P62,P63. Flexprop has way too many dependencies to port it to Ultibo

    Does it? I'm pretty sure the actual flexspin compiler has literally zero dependencies (aside from having bison installed on your build machine).

  • pik33pik33 Posts: 1,735
    edited 2022-07-24 20:04

    Ultibo is a Pascal environment. It can link C (and not C++) static libraries ( .a file) compiled for arm-none-eabi. Only dependencies that are accepted are libc and libm.

    If there is a chance that the compiler can be compiled with these restrictions, then yes, we can have a OS-less combo machine :)

    I recompiled libmad mp3 decoder for Ultibo. The most time consuming thing was to remove all errno, logging and and printf to stderr stuff. After deleting them all from the code, Ultibo accepted the .a file I compiled. Then I lost a modified source somewhere :( I recompiled also libxmp for module playing: this was simpler to do, less to remove. The workflow was like this: compile to .a, try to link to Ultibo code, got a linker error, find the offending stuff, tell several bad words, remove the (bad word), repeat.

  • Wuerfel_21Wuerfel_21 Posts: 3,038
    edited 2022-07-24 20:12

    @pik33 said:
    Ultibo is a Pascal environment. It can link C (and not C++) static libraries ( .a file) compiled for arm-none-eabi. Only dependencies that are accepted are libc and libm.

    If there is a chance that the compiler can be compiled with these restrictions, then yes, we can have a OS-less combo machine :)

    There isn't just a chance, that's literally just what it does.

    gcc -g -Og -Wall -fwrapv -I. -I./backends -I./frontends -I./build -DFLEXSPIN_BUILD -o build/flexcc flexcc.c cmdline.c build/common.o build/case.o build/spinc.o build/lexer.o build/uni2sjis.o build/symbol.o build/ast.o build/expr.o build/dofmt.o build/flexbuf.o build/lltoa_prec.o build/strupr.o build/strrev.o build/strdupcat.o build/to_utf8.o build/from_utf8.o build/preprocess.o build/functions.o build/cse.o build/loops.o build/hloptimize.o build/hltransform.o build/types.o build/pasm.o build/outdat.o build/outlst.o build/outobj.o build/spinlang.o build/basiclang.o build/clang.o build/outasm.o build/assemble_ir.o build/optimize_ir.o build/inlineasm.o build/compress_ir.o build/outbc.o build/bcbuffers.o build/bcir.o build/bc_spin1.o build/outnu.o build/nuir.o build/nupeep.o build/outcpp.o build/cppfunc.o build/outgas.o build/cppexpr.o build/cppbuiltin.o build/directive.o build/expand.o build/mbchar.o build/mcpp_eval.o build/mcpp_main.o build/mcpp_system.o build/mcpp_support.o build/version.o build/becommon.o build/brkdebug.o build/spin.tab.o build/basic.tab.o build/cgram.tab.o -lm
    

    Only -lm.

    Bigger issue is that the code is totally not re-entrant and will leave all of its garbo on the heap after its done.

  • roglohrogloh Posts: 4,461
    edited 2022-07-25 05:24

    I like the sound of that. I found Ultibo boots really fast on a RasPi. If it could become a debug host for a P2 and build code as well that could be useful down the track and could make a nice somewhat self contained setup without requiring a full PC for example. You might be able to have the P2 interact with a Pi and run external compilation jobs then reload into RAM and execute, or save to local filesystems on the P2. Things like that might become possible.

    I picked up a Raspi400 recently because I noticed it was getting harder to get a hold of the Pi4's, but haven't used it in anger yet. Does Ultibo work on these systems as well pik33?

  • pik33pik33 Posts: 1,735

    USB 3 subsystem still doesn't work on Pi4 family. The best machine for Ultibo is still a 3B+

  • @hinv said:
    @"Stephen Moraco" This looks like quite a useful tool. Will this gateway allow the pi to boot the P2 via the pi UART? If so, I could run FlexProp on the Pi, and that would open up a lot of possibilities.

    Yes, it will the way you suggest with a little work (the code in the repo is not set up that way but could be.)...

    Regarding possibilities, yes! They are already there! I've been building for the P2 with FlexProp running on the RPi or running on my Macs or even running under windows for longer than this project existed thanks to Eric's (and others) excellent work on FlexProp. Let me explain:

    My development environment is on large (many cores, lot's of ram & storage) mac's. I use VSCode as my coding environment (as you might expect from my other efforts for this community ;-) I use SSH extensions under VSCode and remotely log into the RPi which is running Raspios (Debian GNU Linux derivative) VScode simply downloads its extensions to the RPi when I first connect. The P2 is simply connected via USB on the RPi. I then have FlexProp running on the RPi compile and download to the P2 all from my mac keyboard controlled by VSCode.

    Actually, this is only one of the arrangements I use to build code for the P2. I can (2) do this all on the mac again using VSCode and FlexProp both running on the max w/P2 connected via USB to the mac or I can also (3) run Windows in a virtual machine on my mac in which I run PropTool and PNut using them to download to the USB-connected P2. Of course, I do (4) have a rather smallish older actual Windows machine that mounts my dropbox files and in this case I still develop on my mac but this 4th form is now my less favorite of all of these environments I use.

    I mention all of this as VSCode along with the wonderful FlexProp effort makes it really easy to develop for the P2 on any of our platforms including RPi's, and no you don't have to access them remotely as I am so used to doing ;-) The RPi's work as a standalone development environment with VSCode (or your favorite editor) along with FlexProp too.

    Does this help?

    -Stephen

  • pik33pik33 Posts: 1,735
    edited 2022-07-25 13:50

    @Wuerfel_21 said:

    @pik33 said:
    Ultibo is a Pascal environment. It can link C (and not C++) static libraries ( .a file) compiled for arm-none-eabi. Only dependencies that are accepted are libc and libm.

    If there is a chance that the compiler can be compiled with these restrictions, then yes, we can have a OS-less combo machine :)

    There isn't just a chance, that's literally just what it does.

    gcc -g -Og -Wall -fwrapv -I. -I./backends -I./frontends -I./build -DFLEXSPIN_BUILD -o build/flexcc flexcc.c cmdline.c build/common.o build/case.o build/spinc.o build/lexer.o build/uni2sjis.o build/symbol.o build/ast.o build/expr.o build/dofmt.o build/flexbuf.o build/lltoa_prec.o build/strupr.o build/strrev.o build/strdupcat.o build/to_utf8.o build/from_utf8.o build/preprocess.o build/functions.o build/cse.o build/loops.o build/hloptimize.o build/hltransform.o build/types.o build/pasm.o build/outdat.o build/outlst.o build/outobj.o build/spinlang.o build/basiclang.o build/clang.o build/outasm.o build/assemble_ir.o build/optimize_ir.o build/inlineasm.o build/compress_ir.o build/outbc.o build/bcbuffers.o build/bcir.o build/bc_spin1.o build/outnu.o build/nuir.o build/nupeep.o build/outcpp.o build/cppfunc.o build/outgas.o build/cppexpr.o build/cppbuiltin.o build/directive.o build/expand.o build/mbchar.o build/mcpp_eval.o build/mcpp_main.o build/mcpp_system.o build/mcpp_support.o build/version.o build/becommon.o build/brkdebug.o build/spin.tab.o build/basic.tab.o build/cgram.tab.o -lm
    

    Only -lm.

    Bigger issue is that the code is totally not re-entrant and will leave all of its garbo on the heap after its done.

    First try: of course a lot of errno and stderr. As expected. A lot of broken references to standard functions was however not expected. It seems this will need a lot of work to make this compiler Ultibo compatible

  • If your link fails on stderr your libc is succ.

  • pik33pik33 Posts: 1,735

    Ultibo simply doesn't have such thing as stderr, so this is not a libc problem, this is the environment problem. This is bare metal, FreePascal based thing. While it can accept libraries coded in C, it has a lot of limitation. Ultibo, however, accepts opening and accessing files from C based code, but its libc is restricted.

    As I wrote earlier, I managed to port libmad to Ultibo, where most time consuming process was removing all errno and stderr stuff. This I expected. However libmad was intended to be a library, while spin2cpp is intended to be a program. So the first thing I had to do was to rename the main :)

    As libmad is a library, its makefile has options for build a static library, so this was much easier. make, try to link, remove all these unnecessary things that were broken.

    My gcc/makefile programing experience is still too low. What I always used for programming everything, everywhere was Lazarus. That means the broken math function references the most probably are the result of my lack of knowledge ("how to compile this stuff as a library").

  • pik33pik33 Posts: 1,735
    edited 2022-07-25 14:55

    Second try: now it is much better: no broken references for standard functions any more. There are however broken references to __ctype_something_loc and several other things.

  • hinvhinv Posts: 1,149

    @rogloh said:
    I like the sound of that. I found Ultibo boots really fast on a RasPi. If it could become a debug host for a P2 and build code as well that could be useful down the track and could make a nice somewhat self contained setup without requiring a full PC for example. You might be able to have the P2 interact with a Pi and run external compilation jobs then reload into RAM and execute, or save to local filesystems on the P2. Things like that might become possible.

    I picked up a Raspi400 recently because I noticed it was getting harder to get a hold of the Pi4's, but haven't used it in anger yet. Does Ultibo work on these systems as well pik33?

    With a PiStorm type protocol, the Pi, which is connected via 26 GPIO's could also serve up RAM, probably at some good performance, and without an OS on the Pi, maybe even somewhat jitter free.

  • roglohrogloh Posts: 4,461
    edited 2022-07-27 03:59

    @hinv said:
    With a PiStorm type protocol, the Pi, which is connected via 26 GPIO's could also serve up RAM, probably at some good performance, and without an OS on the Pi, maybe even somewhat jitter free.

    OMG do I now have another memory driver to write? I know about the SMI on the Pi but not much about PiStorm.

    https://iosoft.blog/2020/07/16/raspberry-pi-smi/

    The thing about SMI is that is seems to be more in line with an async register style transfer bus with its low address pin count and designed to be controlled from the Pi. Maybe an address phase from the P2 to the Pi could be done with SPI and then have the data transfer from Pi->P2 go via SMI (or the Pi could pull data from the P2 using SMI in case of writes to memory). The async nature of it probably makes it harder to transfer at high speed unless the Pi's DMA clock source or its SMI transfers could be driven from a P2 IO pin perhaps, so they can remain in sync during the streamed transfer...? But I quite like the idea of a large amount of RAM for cheap using a Pi Zero running Ultibo or something fast to boot. It does chew quite a lot of pins though vs PSRAM.

    My driver is limited to 256MB right now per memory bus because the top nibble of the address defines the type of memory request instead of being upper address bits. I've already found a 2Gbit flash chip from Winbond that would max this size out, and a Pi acting as a memory device obviously could too.

  • @hinv said:

    With a PiStorm type protocol, the Pi, which is connected via 26 GPIO's could also serve up RAM, probably at some good performance, and without an OS on the Pi, maybe even somewhat jitter free.

    Most of this discussion should probably be made to be its own thread as it really might be hard to find in this RPi gateway software/mechanism thead.

    But, having said that...
    @hinv this can also be done with the RaspiOS in place By using an LKM as I've done in My Tri-panel LED driver LKM experiment.

Sign In or Register to comment.