Shop OBEX P1 Docs P2 Docs Learn Events
Python for P2 — Parallax Forums

Python for P2

Let's move the discussion of implementing Python or MicroPython on P2 here instead of cluttering up the "I want a P2! And how are we going to get these to you?" thread.
«134

Comments

  • Excellent idea. I'm sure Ken will support this thread :D

    David, thanks for mentioning XBYTE again. I'd long lost the conversation where this was discussed.

    Repeating myself a bit, but here's a summary byte code list distilled from https://docs.python.org/2/library/dis.html#bytecodes :smile:
    STOP_CODE
    NOP
    POP_TOP
    ROT_TWO
    ROT_THREE
    ROT_FOUR
    DUP_TOP
    UNARY_POSITIVE
    UNARY_NEGATIVE
    UNARY_NOT
    UNARY_CONVERT
    UNARY_INVERT
    GET_ITER
    BINARY_POWER
    BINARY_MULTIPLY
    BINARY_DIVIDE
    BINARY_FLOOR_DIVIDE
    BINARY_TRUE_DIVIDE
    BINARY_MODULO
    BINARY_ADD
    BINARY_SUBTRACT
    BINARY_SUBSCR
    BINARY_LSHIFT
    BINARY_RSHIFT
    BINARY_AND
    BINARY_XOR
    BINARY_OR
    INPLACE_POWER
    INPLACE_MULTIPLY
    INPLACE_DIVIDE
    INPLACE_FLOOR_DIVIDE
    INPLACE_TRUE_DIVIDE
    INPLACE_MODULO
    INPLACE_ADD
    INPLACE_SUBTRACT
    INPLACE_LSHIFT
    INPLACE_RSHIFT
    INPLACE_AND
    INPLACE_XOR
    INPLACE_OR
    SLICE+0
    SLICE+1
    SLICE+2
    SLICE+3
    STORE_SLICE+0
    STORE_SLICE+1
    STORE_SLICE+2
    STORE_SLICE+3
    DELETE_SLICE+0
    DELETE_SLICE+1
    DELETE_SLICE+2
    DELETE_SLICE+3
    STORE_SUBSCR
    DELETE_SUBSCR
    PRINT_EXPR
    PRINT_ITEM
    PRINT_ITEM_TO
    PRINT_NEWLINE
    PRINT_NEWLINE_TO
    BREAK_LOOP
    LOAD_LOCALS
    RETURN_VALUE
    YIELD_VALUE
    IMPORT_STAR
    EXEC_STMT
    POP_BLOCK
    END_FINALLY
    BUILD_CLASS
    WITH_CLEANUP
    STORE_MAP
    HAVE_ARGUMENT
    
  • octettaoctetta Posts: 123
    edited 2018-11-25 17:09
    Also, I remembered that spinsim was the P2 capable simulator https://developer.parallax.com/spinsim/, but it doesn't seem to support XBYTE yet.

    Maybe I'll have some time to boost that code with a pull request. Other than pulling and building, I don't know much about that code yet.

    I see that DavidB and DaveH are the primary contributors, is that correct?
  • MicroPython is Python 3, so here ya go: https://docs.python.org/3/library/dis.html#bytecodes :)
  • I know this is sure to spark a debate, but what's the "correct" target?

    Python 2.7 or Python 3.x

    At least in my little work sphere, 2.7 is the Python of choice.
  • I think to fulfill the educational need Ken is talking about, P2 really needs to have MicroPython (+P2-specific libraries) so that all of the existing tutorials and materials that are being developed for the language subset in MicroPython can be re-used.

    Some other implementation of Python might be interesting for power users, but I don't think it would drive P2 adoption in the same way.

    The best situation would be to coordinate with the MicroPython upstream authors to develop a P2 implementation that could be accepted upstream. Then P2-based boards would get listed on their website as supported, etc.
  • octetta wrote: »
    I know this is sure to spark a debate, but what's the "correct" target?

    Python 2.7 or Python 3.x

    At least in my little work sphere, 2.7 is the Python of choice.

    We should strive for 3.x. Python 2.7 is officially end-of-life at the end of 2019, a little over a year away.
  • FYI, it looks like MicroPython also uses byte codes. They are symbols prefixed by "MP_BC_". However, it might not be that helpful to use XBYTE to implement the byte code interpreter if the byte codes operations are fairly high level. It's worth considering but I think it would probably be best to just get the C byte code interpreter working first.
  • As far as the question of 'will it fit', here's what the MicroPython FAQ (https://github.com/micropython/micropython/wiki/FAQ) has to say:
    TL;DR: 128K ROM/8K RAM is the recommended minimum configuration for doing something "real". You can easily go below (or above) that, see below.
  • jmgjmg Posts: 15,140
    David Betz wrote: »
    FYI, it looks like MicroPython also uses byte codes. They are symbols prefixed by "MP_BC_". However, it might not be that helpful to use XBYTE to implement the byte code interpreter if the byte codes operations are fairly high level. It's worth considering but I think it would probably be best to just get the C byte code interpreter working first.

    Certainly makes sense to use a more traveled path first.
    Does anyone have numbers for bytescodes-per-second that Micropython examples out there attain ?
  • Clock LoopClock Loop Posts: 2,069
    edited 2018-11-25 19:29
    It would be nice to implement MQTT....

    https://github.com/pfalcon/micropython --->3.0 for microcontrollers.

    Then it becomes even easier to make a book, (like yes a book made from a tree) on the shelf, have internet.

    Then it can tweet me when its accellerometer detects a long enough period of no movement.
  • As you look at this, you must also look into editors. We don't want to create our own again, ever. Creating our own tools for Python, Blockly or C will isolate our business.

    https://codewith.mu/ for MicroPython? Is it a possibility.

    Also, Tubular must join this discussion.

    Ken Gracey

  • Ken, that editor is awesome sauce!

    https://codewith.mu/en/howto/microbit_settings

    Looks to me like a P2 integration is possible. May even be practical.

    The BBC Micro Bit has modest specs:

    256K Flash, 16Kb Static Ram (that's nice and fast, at least)
    16Mhz Arm CPU, single core.

    A P2 board, with a Bluetooth, or some other networking option, and firmware setup could potentially emulate a MicroBit, meaning there should be ample room for a meaningful Python environment.

  • Thonny has support for MicroPython as well. There is basic support in the editor and then a Thonny plug-in for each board to add board-specific items.

    https://bitbucket.org/plas/thonny/wiki/MicroPython

    Thonny is the editor that RPi Foundation includes on their Linux distro for the Pi, so it should be reasonable well supported for educational use.
  • Roy ElthamRoy Eltham Posts: 2,996
    edited 2018-11-25 21:06
    We use Visual Studio Code for our Python editing at work. VS Code is free AND cross platform, and has a nice extension setup with many extensions supporting most languages.

    https://code.visualstudio.com/#alt-downloads
  • Ken Gracey wrote: »

    Also, Tubular must join this discussion.

    OzProp and I are going to give a short talk to the local MicroPython people this Wednesday.

    We'll be in a better place to comment, after that.

    Sorry it can't happen sooner, Ken!

  • jmgjmg Posts: 15,140
    edited 2018-11-26 00:18
    Ken Gracey wrote: »
    As you look at this, you must also look into editors. We don't want to create our own again, ever.

    'Editor' means many things, to many people.
    If you look carefully at the web link you gave, there are [Debug] and [Run] buttons, and [Run] appears to launch a Python Console. (PC of course)

    You will not find an 'editor' that already includes a P2 Debugger, behind that button.
    Ken Gracey wrote: »
    Creating our own tools for Python, Blockly or C will isolate our business.
    'tools' also means many things to many people.
    You can avoid creating Text Editors, given those exist already, but no one else is going to have a P2 Debugger sitting around.
    Just like you had to a create a P2 downloader, & P2 Assembler, in order to test P2, you will need to take charge of some of the tools, and failing to do so will isolate your business.

  • jmg wrote: »
    You will not find an 'editor' than already includes a P2 Debugger, behind that button.

    Sure, but if your users are writing in micropython like this example then a P2 debugger would be the wrong level of abstraction for a P2 debugger.
    Ken Gracey wrote:
    Creating our own tools for Python, Blockly or C will isolate our business.

    Product differentiation is a double-edged sword. On the one hand you get to sell "all the chips"[tm], but on the other hand you get to hold all the support, build all the tools, create all the content, generate all of the evolution / growth.

    He's not saying "Don't build P2 tooling". He's shying away from building micropython tooling which is specific to the P2.

    There's already a wheel factory, there's no need for us to make a new one as long as when we port micropython to the P2 we remember to make it round.

  • I've been working with MicroPython on the ESP32 -- it would make me very happy if it could be ported to the P2.
  • jmgjmg Posts: 15,140
    __red__ wrote: »
    ... He's shying away from building micropython tooling which is specific to the P2.

    Showing just how 'tools' & 'tooling' also means many things to many people.

    P2 is a unique microcontroller with its own opcodes and hardware, aside from running an ARM or ESP32-emulator, (woefully slow & partial), you certainly are going to have to build tools that are specific to P2.
    (even that slow emulator choice, is building a tool specific the P2 !)


  • jmg wrote: »
    'tools' also means many things to many people.
    You can avoid creating Text Editors, given those exist already, but no one else is going to have a P2 Debugger sitting around.
    Just like you had to a create a P2 downloader, & P2 Assembler, in order to test P2, you will need to take charge of some of the tools, and failing to do so will isolate your business.

    Education is a forgiving market in many ways. Customers could be happy with serial communication being a "debugger" for example. I can't see many of them setting breakpoints and traces (right term?) in the levels we currently support. They also don't like investing in anything too proprietary, like Spin/ASM. We tried to market the language to them and for several reasons we failed, only to recover years later. They decide what they want based on what's popular in industry and what their state/national standards may dictate. Usually, the standards favor languages that run on PCs, not microcontrollers.

    The kind of "tool development" I'd like to avoid is the programming environment, IDE, front end, or editor. These are just the words I use from my experience at Parallax, but those of you who've programmed other things will have a more complete vocabulary to describe what I'm talking about. The graphical side.

    I recognize there's work we'd need to do to make the P2 architecture be accessible in these pre-existing systems, perhaps through customizing existing front-end tools. But we'd really like to minimize that work. A company of our size can't create entirely new systems that compile and run on Linux, Windows, OSx, Chrome, etc. We need to partner with whatever systems are already doing that as much as possible. We've sunk many hundreds of thousands (millions?) into supporting these custom systems for the BASIC Stamp and Propeller already.

    If we can leverage any existing efforts, like Mu, we must. Not only do we stay afloat by doing this, we're using systems that our educational customers already know. This is why BlocklyProp has become so popular.

    We are already going to need to update the Propeller Tool for the P2. And the code base we're using has been problematic to compile on OSx. I forget the specifics - maybe there was a licensed component in the tool somewhere. Aside from what Chip requests for Spin/ASM we hope to minimize the tool development.

    We fully expect to get involved in compilers, assemblers, and interpreters as we have in the past, for multiple languages.

    Ken Gracey
  • potatoheadpotatohead Posts: 10,253
    edited 2018-11-26 01:21
    Regarding Micro Python, and potentially Python in general, what has education said they want / need?

    Secondly, has there been anything Parallax knows they need / want, perhaps they don't know yet?



  • jmgjmg Posts: 15,140
    edited 2018-11-26 01:50
    Ken Gracey wrote: »
    The kind of "tool development" I'd like to avoid is the programming environment, IDE, front end, or editor.
    If we can leverage any existing efforts, like Mu, we must.

    Makes perfect sense, so let's download Mu, and take it for a quick spin.

    Certainly not 'small' at 400MB, and 8,850 Files, 705 Folders (!) - but it does seem to download and install fine, but maybe not netbook/bottom end friendly ?
    ie more suited to the 'better resourced' student.

    It has 4 'modes' in there already, on a simple pick list, and some are remote / cross development focused. That's a good sign.
    Some screen grabs below :

    Mu_Microbit.PNG - shows buttons for Flash (code download) and files and REPL (remote command line read-evaluate-print loop), and a plotter.
    Mu_AdaFruit.PNG - shows Serial link, and plotter
    Mu_PyGameZero.PNG - more game-centric buttons,
    Mu_Python3_PC.PNG - PC Python support

    Assuming you have a P2 that can actually run Python, it looks like support, to Microbit levels, is some minor clone/edits away.
    A Python P2 downloader (ie a P2 downloader, written in Python) already exists.
    Ken Gracey wrote: »
    Education is a forgiving market in many ways. Customers could be happy with serial communication being a "debugger" for example.
    I can't see many of them setting breakpoints and traces (right term?) in the levels we currently support.

    Yes and no. They can perhaps tolerate primitive serial print type debug, if they know no better. However, you should always keep an eye on what could be better :)

    eg Take a look at the Mu image below, Mu_Python3_PC.PNG
    That's Mu running Python native on a PC, with [Debug] enabled.
    Notice we now have Step Over, Step in, Step Out, and over to the right, we have a Debug inspector, that shows VARs and their current values.
    That red dot is a breakpoint I toggled with a click next to the line number. Code runs to that, then pauses and updates the VARs

    See how that is all rather easy, and gives a level of 'live code' that print-debug lacks ?

    Now, imagine that display/menu, for code running on a P2. That's more work, but not impossible.

    It is also typical of what other Microcontroller vendors offer in the way of Debug support, so should be on a P2 roadmap.
    There are P2 Debuggers in various forms already, this would need those existing Mu Debug buttons/panes, spliced to use an external Debug, over a fast serial link.
    294 x 121 - 7K
    152 x 112 - 4K
    364 x 115 - 11K
    1140 x 520 - 37K
  • jmg, excellent research. You're pandering to the lazy now (me) but I really appreciate it.

    Seems that Mu already has some very basic type of debugging support with serial including a plotter.

    If you don't have a Micro:bit I'd be happy to send you one so you can see how it works with this tool. Just drop me your shipping info kgracey@parallax.com and I'll get it on the way.

    And yes, I concur about looking ahead about what might be needed vs what we have. I agree entirely with this thought. I've also got a few caveats to this thinking when it comes to anticipating needs, but they're not relevant at this moment. In summary: the "visionary" limitations I impose on our educational support are assumptions about future educational needs. The failed efforts are costly, and we've already used up our allocation of creativity for looking ahead on the P2. Everything else we create must have a more known outcome and less risk.

    Ken Gracey
  • Circuit Python (Adafruit's variant of micro python) is a great example to review, if you are looking for a simple environment for Python coding.

    The best example of simplicity is how several of Adafruit's boards (those with the UF2 bootloader https://learn.adafruit.com/welcome-to-circuitpython/non-uf2-installation) allow dropping CircuitPython source files onto their USB-based storage directory (The board shows itself as a USB thumb drive) and the board reboots and runs your code!

    You can also use the Mu editor to build your sources and run them via a serial connection to the Circuit Python-capable Adafruit boards https://learn.adafruit.com/welcome-to-circuitpython/installing-mu-editor

    Circuit Python Overview:
    https://learn.adafruit.com/welcome-to-circuitpython/overview

    I'm currently using Adafruit's Neo Trellis M4 https://adafruit.com/product/4020, which is a Circuit Python-capable board with a 4X8 array of RGB-LED buttons, accelerometer, serial-IO and lots of features. The simplicity of coding for this board, without need for an IDE, compilers, loaders, etc. is encouraging. I'm pretty sure the education world is going to see this as a prime example of the way things should be!

    BTW: For all of Adafruit's Circuit Python-capable boards, you can write your code as Arduino sketches and load them using the Arduino IDE, if you feel a need for C/C++ compiled code, rather than python scripts.

    Just saying... No need to re-invent the whole wheel :wink:

    dgately
  • Is Circuit Python smaller than the minimal build on the MicroPython repository? I think we should try to get the simplest and smallest version working first. It really ought to be possible to run the minimal build on a P1 with an external flash chip. I realize that isn't a configuration that is supported by Parallax anymore now that the C3 is no longer being sold but there are other boards with external flash like the DNA that should be able to run it.
  • David Betz wrote: »
    Is Circuit Python smaller than the minimal build on the MicroPython repository? I think we should try to get the simplest and smallest version working first. It really ought to be possible to run the minimal build on a P1 with an external flash chip. I realize that isn't a configuration that is supported by Parallax anymore now that the C3 is no longer being sold but there are other boards with external flash like the DNA that should be able to run it.

    Circuit Python appears to be just a fork of micropython with changes to add Adafruit's choice of chips (Atmel SAMD... M1, M4, etc...) and boards (Feather, ESP-based boards. others). The minimal port source looks to be the same as micropython sources on GitHub... https://github.com/adafruit/circuitpython/tree/master/ports

    So yes, getting a minimal build for P1 (and then P2) is probably a good idea to start this up.


    The ability to access the FLASH as USB storage is obviously a matter for a bootloader and available memory/storage... A nice feature, but not pertinent to getting python running on current propeller boards. Though, you should always be looking forward.

    dgately


  • For IDEs and Python, you might want to consider partnering with someone like Trinket.io. There is in-browser python support and they have a simple IDE. Some integration could result in having a browser-to-device development model.

    What would be even cooler is if there was some self-hosting ability of a P2 board that could offer the IDE over USB.

    You can implement networking over USB, the Beaglebone Black does this. Their model is "plug it into your computer, then go to this address in your browser", it's UI is completely self hosting and just works when you plug it into the USB port.

    With the P2 having the potential to do full speed USB, you could have an environment that runs from SD card and self-hosts off the P2.

    Sure the resources would be limited in memory and cogs to less than bare metal, but it would be a lot friendlier.

    The dev work to make a self-hosting setup like that is non-trivial (writing a USB driver, then network driver, then HTTP server, plus all the backend integration to make the HTTP server actually do stuff).

    Heh, so in the end the "cool" self-hosting idea is shot out of the water due to development requirements.

    So, back to using a browser hosted solution like Trinket.io

    The MU editor does seem to be rather heavyweight, I think a browser based solution offers a better solutions, especially for the educational market that might use Chrome books and other lightweight computers.
  • jmgjmg Posts: 15,140
    dgately wrote: »
    The ability to access the FLASH as USB storage is obviously a matter for a bootloader and available memory/storage... A nice feature, but not pertinent to getting python running on current propeller boards. Though, you should always be looking forward.
    That's going to be tricky on present P2 designs, and on native P2 there is no USB Boot choice, so USB file copy model is not really going to be practical.
    However, there is already a Python-coded P2 downloader, which looks easy to add behind a Microbit (flash) type menu button.

    The harder bit is getting the Python engine running on a P2.

  • jmg wrote: »
    dgately wrote: »
    The ability to access the FLASH as USB storage is obviously a matter for a bootloader and available memory/storage... A nice feature, but not pertinent to getting python running on current propeller boards. Though, you should always be looking forward.
    That's going to be tricky on present P2 designs, and on native P2 there is no USB Boot choice, so USB file copy model is not really going to be practical.
    Ah, yes... I didn't notice that the ATSAMD51 32-bit Cortex M4 on the NeoPixel board has built-in USB support. But does that rule out some future P2-based board that would support access to on-board FLASH via USB?
    The harder bit is getting the Python engine running on a P2.
    Agreed!

    dgately

  • jmgjmg Posts: 15,140
    edited 2018-11-27 01:39
    dgately wrote: »
    Ah, yes... I didn't notice that the ATSAMD51 32-bit Cortex M4 on the NeoPixel board has built-in USB support. But does that rule out some future P2-based board that would support access to on-board FLASH via USB?
    Yes, future USB-Flash support is always possible. Maybe even P2-inbuilt USB will prove solid enough, with some write protected flash ? Tho that's getting more costly in cogs/$/USB
    Otherwise, a small MCU with USB that can connect to flash would be possible.
    If Peter gets EFM8BB1 working nicely, ( & looks like he will) there is an EFM8UB3 sibling, with 40k flash, that comes in QFN20, QFN24 - that could likely manage both MSD and CDC links

    Addit: there may be avenues other than flash-drive emulation too.
    I see the UB3 includes a factory installed HID bootloader, and google finds comments here about getting that working, I think from Python.

    There is a MCU pin (P2.0 / C2D) that determines Stay/Exit the loader, (seems to use 1023+512 bytes for loader), perhaps that could be morphed/tweaked to support both UB3 firmware update, and P2 flash update ?
    However, unclear is the speed this has been designed for.

    I suspect it will be simpler to use the P2-UART and P2 bootloader, as a flash pathway, and to focus on faster serial links for that.
    The P2 Python loader worked quite well, FWIR.

    Digging, I find my notes of FT232H testing, py_p2loader_speed.py - ie the loader is now speed-tuned, and SW/Python, is not the bottleneck (even 12MBd determines the speed)
    # Change to SendAry = b64encode(bytes(buff1)) 
    # Len: 699060 Time to construct s:  0.014075  << inbuilt from base64 import b64encode MUCH Faster 
    # Time to send at Baud  12000000  s  0.58262
    # lastchs =  85 85 85 81 61 61 = appends two ==
    # SendAry = b64encode(bytes(buff1)).rstrip(b'=') 
    # Len: 699058 Time to construct s:  0.007120    -2 on Len as dropped the two ==
    # Time to send at Baud  12000000  s  0.58249
    
Sign In or Register to comment.