Python for P2

13

Comments

  • Also, it could even be possible to do this using Dave Hein's p2gcc or Eric's C compiler once he finishes it.
  • Ken GraceyKen Gracey Posts: 6,359
    edited December 1 Vote Up0Vote Down
    Genetix wrote: »
    Ken,

    Do you intend MicroPython to be "self-hosted" like on the first microcomputers that started with a BASIC prompt, or compiled and edited on another device like SimpleIDE and BlockyProp, or both?

    I suppose it depends on:

    - the cost of developing, or whether there are volunteers
    - the memory used
    - the speed at which the code executes
    - the access to multicore features
    - the long-term maintenance
    - the compatibility with already-popular open-source editors that compile on Mac, Windows and Linux (we don't want to create these systems)

    My understanding is that MicroPython is an interpreter, so there's no compiling on download, but bytecodes (sorta like the BASIC Stamp). Am I close in my explanation here, David?

    Ken Gracey
  • Ken Gracey wrote: »
    [My understanding is that MicroPython is an interpreter, so there's no compiling on download, but bytecodes (sorta like the BASIC Stamp). Am I close in my explanation here, David?
    Yes and no. Yes, it seems MicroPython has a byte code interpreter. The difference is that it can also support a self-hosted compiler to translate source code into byte codes. I believe this is optional though and can be left out for a smaller memory footprint. If you do that, you have to generate the byte codes on a PC like Spin does. I think we can probably fit the self-hosted version on the P2.
  • My understanding is that MicroPython is an interpreter, so there's no compiling on download, but bytecodes (sorta like the BASIC Stamp). Am I close in my explanation here, David?

    There is both an REPL prompt that runs on the host microcontroller serial port, and the ability to send over compiled code from an editor.

    There are bytecodes, but there are also 2 native ARM code emitters that can create native code directly - one that is pure Python and one that is a 'typed' version of Python that emits even faster code. Having P2 support for LLVM seems like it might enable having native P2 emitters, in addition to the bytecode?

    The video that was linked earlier in this thread has a great overview of the different modes and how they can generate more and more optimized code.

    I would think that if the implementation is at least 'MicroPythoic' in nature and follows guidelines laid out by the MicroPython team, it would work fine with Mu.......

    I might pick up one of the official pyboards to play with to wrap my mind around some of this.

  • Ken Gracey wrote: »
    - could a P2 MicroPython be used with a tool like the Mu editor, if we made some modifications to do what we needed it to?

    I think any programming-type text editor could work for micropython... If you mean an editor that does language specific formatting & coloring of source, there are quite a few that format Python well.

    Or, did you mean more of an IDE/editor that includes the above and allows compiling (if necessary), loading, & running the code with access in a terminal to the board that runs micropython? Normally, micropython compiles live (right?) though it can pre-compile to bytecode and native code for efficiency & speed. In most cases the editor would just load/save the source to the micropython-running board.

    That should all be separate from getting micropython running on P2. I wouldn't worry too much as once there is micropython available, the need for an editor will surely push a solution to the top:-)

    Also, the interpretive REPL mode of micropython should be available in any implementation of micropython. So, programming live into REPL should give an environment similar to BASIC on microcomputers of the past.

    dgately
    Livermore, CA (50 miles SE of San Francisco)
  • Also, the interpretive REPL mode of micropython should be available in any implementation of micropython. So, programming live into REPL should give an environment similar to BASIC on microcomputers of the past.

    That's actually a great prototyping / learning mode, one of the reasons I thought highly of Micropython.
    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • Ken Gracey wrote: »
    - could a P2 MicroPython be used with a tool like the Mu editor, if we made some modifications to do what we needed it to?

    Sure, see the other comments I made about mu editor and the existing MCU systems it supports.
    The very simplest interface is like Adafruit, which uses external copy/download, and opens a terminal window.
    For testing, it looks like that could be used almost as-is, and there is a Python P2 Download, that was speed optimized & I checked to 12MBd
    Next is micro:bit that includes the download button.
    Most comprehensive is the native Python support, that has Single Step/Break/Var inspect
  • Ken,

    In the video he shows using the VM (Interpreter) but also running native (assembly) code.
  • And that assembly wrapped up in a Python like syntax. That implies no big programs, just snippets here and there, where a big difference can be made.

    To get the goods out of P2, and into Micropython, it's going to be important to be able to do that, package features up into methods, much like the direction Spin 2 is headed.

    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • I don't mean to hijack the thread at all, but I was actually just thinking about learning some Python and read this thread. I saw this book at the local B&N yesterday: Python for Microcontrollers: Getting Started with MicroPython, but it gets only mediocre reviews. Can anyone here suggest a better one or tutorial to learn Python as it would relate to robotics and what we might envision in the P2 eventually? Thanks!
  • The way I see Python is that it needs to work how Python developers expect.

    You have textual .py files that are compiled to .pyc files at runtime (or statically before runtime if you want).

    The bytecode interpreter then runs the .pyc bytecode.

    It's sort of like SPIN, there is a textual language compiled to bytecode, then the SPIN interpreter runs it.

    What's different is that MP is either just the .pyc part or the .py-.pyc part too.

    If you type "python", you get a prompt and can execute code interactively. Programmers expect this for a variety of reasons.

    If the human interpreter can be a module that is loaded at runtime, that's great, because once you deploy a product, the .pyc files are all you need on an SD card to do stuff. That frees up memory and resources for the .pyc interpreter.

    The GCC vs LLVM thing is pretty much the Linux vs BSD thing. If you're in the BSD camp, then you're predisposed to LLVM, if you're in the Linux camp, GCC is your compiler.

    While LLVM is retargetable, you've already spent resources on getting GCC working. More than just the compiler proper, there's the runtime to consider porting too.

    IMHO, stick with the horse you rode in on.

    Get the MP pure-C parts working on P2.

    Work on creating interfaces to all the P2 pseudo-peripherals.

    Then you can circle back and focus on a pure-ASM .pyc interpreter.

    If you go by the "surveys" that sites release, Python isn't the most popular language...however what you don't see is that the popular languages are dominated by what companies can outsource and what's "kewl".

    Python is mature and well supported across a large swath of infrastructure and there are many developers who know it.

    Python is not the train wreck that Java is, nor the flavor du jour (framework of the week) that Javascript is.

    Python is to C what Visual Basic was to BASIC, in fact, the syntax of SPIN is so close to Python, that many people will be able to transition without having to bend their brains too much.
  • pedward wrote: »
    ...

    Python is to C what Visual Basic was to BASIC, in fact, the syntax of SPIN is so close to Python, that many people will be able to transition without having to bend their brains too much.

    Which raises an important question for Ken :
    Should the next Spin (for P2), be made even more Python like, given they are already close ?
  • No.

    The next SPIN should be what Chip intends it to be. They will be two very different beasts.

    Do not taunt Happy Fun Ball! @opengeekorg ---> Be Excellent To One Another SKYPE = acuity_doug
    Parallax colors simplified: https://forums.parallax.com/discussion/123709/commented-graphics-demo-spin<br>
  • DrPop wrote: »
    I don't mean to hijack the thread at all, but I was actually just thinking about learning some Python and read this thread. I saw this book at the local B&N yesterday: Python for Microcontrollers: Getting Started with MicroPython, but it gets only mediocre reviews. Can anyone here suggest a better one or tutorial to learn Python as it would relate to robotics and what we might envision in the P2 eventually? Thanks!

    I've read the book you linked as well as Programming with MicroPython: Embedded Programming with Microcontrollers & Python and I wasn't all that impressed with either of them. Both provide a fairly wide ranging overview of the hardware facing libraries and provide basic examples of how to use them, but they are by no means a language or library textbook/reference. Also, as far as I can recall (it's been a while since I read these), neither provides any details about how micropython works under the hood or what is required to port MicroPython to a different microcontroller.

    It seems to me that the best approach for learning MicroPython to just learn Python and then get familiar with the MicroPython specific libraries / microcontroller interface features using the MicroPython documentation. The machine module is of particular interest as it is what is actually used to interact with the hardware (e.g. digital pin manipulation, SPI, I2C). From there, you refer to the quick reference for the board you are using to get the hardware identifiers needed when instantiating the hardware facing classes or making hardware facing function calls.

    I don't actually know Python (learning it is on my TODO list) so I can't recommend any particular resource for learning the language.
  • It seems that to get MicroPython working on P2 we first need a C compiler that is up to the job. It would be interesting to try p2gcc since it is currently the only complete C compiler that targets the P2. Whether that will work will probably depend largely on how much of the C standard library MicroPython needs and how much of it works under p2gcc.
  • p2gcc can probably compile MicroPython, but it's unlikely that the current library is sufficient for it to link it. p2gcc's current library was written by me, and contains only the most common functions used by C programs. There is no floating point support yet. I have written a number of the floating point library routines, but I haven't included them in the library yet, nor have I checked them into GitHub.

    I have attempted to compile the P1 ProgGCC's library source, and much of it does compile OK. Quite a bit of P1's library contains P1 assembly code, which I haven't attempted to convert yet. Ultimately, P1 and P2 should share the same library source code with some #ifdef's to handle the differences.

    I've been busy with other things for the past few months, but once I get my hands on a real P2 chip I'm looking forward to working on p2gcc again.
  • DavidZemonDavidZemon Posts: 2,559
    edited December 1 Vote Up0Vote Down
    I'll throw my $0.02 in since everyone else is...

    If p2gcc proves capable of compiling (and linking :wink:) MicroPython, then I'll happily retract most of this post.

    I'm in the "we're jumping the gun way too early" camp on this one. I'm not sure there's a lot of point to most of the discussion here until we have a working C compiler for the P2. Once we have that, porting MicroPython should be quite simple, at least the basic stuff. Then once we have the basic stuff - a REPL running on a P2 - we can dive deeper and see about exposing the P2's RAM/flash/SD card/etc as USB drive for the host computer and after that, maybe even emitting native P2 assembly from Python source.

    But I feel strongly that all of this energy should be put toward getting a C/C++ compiler working. That will open the doors to C/C++, MicroPython, Lua, JavaScript, Arduino, and a million other things that folks are dying for. With a standard toolchain like GCC or LLVM, you also open the doors to powerful tools that work well those toolchains (CMake, Eclipse, CLion).

    As for editors... there are a million. This isn't like the C/C++ editor wars where there's only a handful and they all suck :smile:. You, of course, have my favorite: JetBrains PyCharm. It even has a MicroPython plugin to be extra helpful. I'd love to work with JetBrains to add P2-specific support to that plugin as well. PyCharm is far from lightweight though (it's Java) and it's not a web IDE either, so we'll want something else as an alternative probably. But none of it matters, because Python is much easier to deal with.... anyone can use whatever editor they want. When there's no "build system" to worry about, your choice of editor/IDE becomes far less important because it's much easier to switch from one to another.
    David
    PropWare: C++ HAL (Hardware Abstraction Layer) for PropGCC; Robust build system using CMake; Integrated Simple Library, libpropeller, and libPropelleruino (Arduino port); Instructions for Eclipse and JetBrain's CLion; Example projects; Doxygen documentation
    CI Server: http://david.zemon.name:8111/?guest=1
  • Yeah, getting a working C/C++ compiler should be the first step. However, there are at least two projects now that are working in that direction: p2gcc from Dave Hein and FastSpin/C from Eric Smith. Porting GCC is going to be a big job and I don't think anyone wants to even start it unless it's pretty clear that it's going to be the C/C++ solution going forward and not one of these other things. I at least don't want to put a lot of volunteer effort into something that may be left on the sideline if someone else's solution ends up being what is official promoted by Parallax. People keep saying that the more tools and languages we get the better but I'm not sure that is really true. I think concentrating on a few necessary tools and not going in ten directions at once is more likely to produce a good result.

    So, my interest in MicroPython is based on the fact that working on it is fun and so even if my solution (if I ever have one) doesn't end up getting used, it will still be an enjoyable experience. Working with GCC is not really fun. It's something you do because you have to to create a toolchain to do other fun stuff.
  • It seems like GCC is the obvious choice for the P2. GCC is used on the P1, and by using it on the P2 we can take advantage of all the hard work that has already been done for the P1. p2gcc is, and was intended to be a way to get quickly get C support for the P2 until GCC was fully ported for the P2. Eric's fastspin compiler is great for integrating Basic, Spin and C code, but I'm not sure it will ever as complete or optimized as GCC would be.

    It seems like it is in Parallax's best interest to fund a project to develop GCC based tools for the P2, and in my opinion they should have done it a long time ago. Now they're in the situation where the P2 chip will be available to the public in a few months, and they have no software support for it. That was my opinion 1.5 years ago when I started working on p2gcc out of frustration with Parallax. Parallax's internal development effort has been on Spin2, but it is unlikely that Spin2 will be used by new customers. It is more likely that new customers will use C, C++ or some other language that depends on C/C++ to build it.

    According to a post by Ken, Parallax has spend $5.8 million on developing the P2. It only makes sense that they invest some small fraction of $5.8 million to pay people to develop software tools for the P2.
  • Dave Hein wrote: »
    Eric's fastspin compiler is great for integrating Basic, Spin and C code, but I'm not sure it will ever as complete or optimized as GCC would be.
    I said something similar a while back and Eric's reply was that FastSpin/C would likely be faster than GCC for some things. Also, Parallax is currently only using C and not C++ for their education efforts so FastSpin/C might be entirely adequate for that purpose.
  • There're are lots of things that I could bring up about linkers, debuggers, libraries, optimizers, industry standards, project development tools, etc., but I'm just too tired of rehashing the same old arguments.
  • David BetzDavid Betz Posts: 12,932
    edited December 1 Vote Up0Vote Down
    I'm not trying to argue anything. I'm just saying that we should probably wait to see what Parallax actually wants before putting time into things like GCC or LLVM. Your p2gcc project seems like a great bridge tool until one of those or maybe Eric's compiler is ready. I posted my comments about GCC in response to someone's suggestion that it would be better to be working on a C compiler than MicroPython. That's all.
  • Sorry, my "argument" comment wasn't directed at you. However, I do agree that it would be better to develop the C compiler first, and then worry about Python and other derivates later.
  • Dave Hein wrote: »
    Sorry, my "argument" comment wasn't directed at you. However, I do agree that it would be better to develop the C compiler first, and then worry about Python and other derivates later.
    Then I guess we need Parallax to decide between GCC, LLVM, or Eric's compiler. You've said your p2gcc was never intended to be a long-term solution. As some have suggested, GCC might be a better choice than LLVM simply because we can probably revive the P2 code generator that we started a while ago. GCC would certainly make the most sense if we are going to be using some of the same people who worked on the P1 version of GCC. On the other hand, if there are others who want to jump in who might have more experience with other toolchains that would be worth considering as well. It seems like the quickest path to C would be Eric's compiler though and it has the advantage of supporting Spin and BASIC as well.
  • GCC has been around for a very long time so one can only assume that it's not going to disappear.
  • Genetix wrote: »
    GCC has been around for a very long time so one can only assume that it's not going to disappear.
    Yeah, probably a good idea to take a look at what would be required to get GCC to target P2. One big part of the job is to port binutils but that includes creating an assembler and there are already a number of those. It would be too bad not to be able to make use of one that already exists.
  • Dave HeinDave Hein Posts: 5,642
    edited December 1 Vote Up0Vote Down
    It would be a good idea to review the discussion in the Status of P2 C/C++ Compiler thread. Some of the comments from that thread are being repeated here. The current P1 compiler already generates code that almost works on the P2. In p2gcc I decided to use the P1 COG model to generate P2 code. I wrote a utility call s2pasm that converts the P1 assembly to P2. For the most part, it handles the arbitrary changes that have been made to PASM, such as renaming MAX and MIN, and converting "WC, WZ" to WCZ.

    I also have to handle the fact that the COG model puts constants and addresses in cog memory, but in the P2 I put these in hub RAM instead. Looking back, it may be better to use the P1 LMM model, and convert the LMM pseudo-ops instead. Anyhow, PropGCC already has the hooks in it to generate P2 code.

    I've looked at the ELF files generated by the P1 compiler, and I think I could modify p2asm to generate the same format. p2asm already understands some of the GAS directives, so I think it could be used as the basis for the P2 GCC assembler.

    The GCC linker is a mystery to me. We would need somebody that understands how that works.


  • marsman2020marsman2020 Posts: 38
    edited December 1 Vote Up0Vote Down
    I'm mostly interested in the idea that since it's a new language for P2, we'd hopefully end up with a single coherent MicroPython implementation for P2.

    However since it depends on C, it seems like we'll be back to the old 'what to use' arguments all over again.

    The tools part of this is the most scary for me. It's sad to see all the PASM/Spin objects in OBEX, while everything else is the Parallax website is all C now, with no clear way for normal users to be able to use any of that code that people contributed.... I understand at least some of the reasons behind why this happened but it's a real bummer, and it has to give people pause when they look at using P1 for projects.
  • I thought we came to the conclusion, in another thread, that the amount of work to update propgcc for P2 was likely a lot less than building a whole new LLVM set for P2.
    Also, I didn't notice anyone stepping up with LLVM experience, while a few of you that worked on propgcc are still around.

    Seems like we should get propgcc updated to target P2, and get it's libraries updated for P2 (while being sure to get everything needed for compiling MicroPython). Then we can get MicroPython building and add the needed stuff for P2 (python libs/extensions for P2 hardware features).
  • Amen.

    You have to learn to crawl before you can walk, getting GCC working on the P2 is the crawling stage, but it's much more than that.

    C and C++ are universal high level languages. Every other language (including C) is written in C/C++, with few minor exceptions.

    You can tune out the below if you don't like hypothetical soapboxes.

    If I were "project managing" this effort, I'd touch base with all the developers who worked on the P1 GCC port.

    I would work to identify objectives for implementing GCC on P2.

    Then those objectives would be broken down into finegrained tasks with scheduling and milestones.

    Here's an example:

    I built up an old PC I had and put a VGA card and Hercules card in it. I want to create a demo to show what you can do with 2 monitors on an old PC.

    The initial objectives are:

    To have a "4 color" Hercules 160x100 modified text mode
    To have a "16 color" CGA 160x100 text mode
    To have interrupt driven page flipping/double buffering on Hercules
    To have interrupt driven page flipping/double buffering on VGA
    To write it in C++ (based on some code written 23 years ago by me)
    To run it under DOS and build it with Borland C++
    To have something that doesn't crash ;)

    Demo objectives are:

    Come up with some interesting effects and animations
    Add some sort of sound support to play a sound track

    --
    I've been working on just getting VGA working with a C++ interface and double buffering.

    * I've begun crawling by refactoring the 23yo code, fixing some newb C++ programmer mistakes :smiley:
    * Then I added multiple video mode support in a single class, this involved creating a struct array of video modes and their attributes, then a function to setup the buffers when a mode change occurred
    * I got VGA double buffering working, this required an offscreen buffer to be allocated and a far memcpy from off screen to screen
    * Then I needed to get some block transfer/copyto working, this required copying from one part of the os buffer to another part of the buffer, which requires clipping and other linear addressing issues
    * Then I animated moving a block of pixels across the screen, which required fixing the copyto to use a temporary buffer, because copying from one location to another can result in overwriting what you are trying to copy from, but it was slow
    * So I had to speed up the double buffering by writing some inline assembly (16000 double word moves)
    * Then I got a bouncing box animation working, which required bounding coordinates, etc
    * Then I made the box change color when it changed direction, which required refactoring the original box draw into a routine that accepted a color, dimensions, and location

    At this point I've not touched the Hercules mode, but I've worked on a framework that supports Text 03h, VGA 13h and VGA 11h modes in one class. Adding Hercules is going to be much easier now that I just need to copy a few bits of code from the old herc.cpp class.
    --

    Anyway, my point is that if your objective is to have "Python for P2", you've still got all the foundational steps and tests to prove that each of those steps is working without bugs.
Sign In or Register to comment.