MicroPython zoom meeting...

Great presentation guys!

A couple of things that came out of todays meetup...

The debug cog displaying on VGA - could you release the code please. I expect we can expand upon this nicely :)

The pasm editor - again could you release the code please. We can make a pasm/spin version too :)

«1

Comments

  • That was a great presentation!
  • +1 what @Cluso99 said, the Editor looks absolutely cool and it would be nice if it can get extracted to use with spin/fastspin.

    Since you are currently thinking about a filesystem in flash, maybe later SD, how about adopting @ersmith approach to smooth out the boundaries and allow SD access and HOST access seamlessly with his interesting host file system provided by the P2load Terminal, basically the serial stream gets double used to provide file access.

    I have not looked at the protocol itself, but hell yes this is absolutely cool to access the SD card or the HOST file system with almost the same code.

    Now add your flash file system. Or, start with that and keep in mind to add SD and Plan9 later. Just try to use the opportunity to make it compatible, since you are building it now new anyways...

    I am very impressed with what you have done so far and I will give you now all a physically thumps down so it will be a thump up down under.

    Thanks,

    Mike
  • roglohrogloh Posts: 2,482
    edited 2020-07-24 - 23:37:20
    I just watched the recorded live stream on Youtube. One question came up about interfacing between PASM2 COG code and MicroPython where Brian mentioned it gets tricky with sharing hub resources etc.

    It is already indeed possible to achieve some basic integration via the method that Eric first implemented and I had used way back in January for spawning COGs from encoded hex strings than contain the PASM code. That part wasn't shown today in Brian's demos, but back then I managed to get my own PASM video driver COG spawned from MicroPython using this approach and then controlled via Python variables that are mapped to it's display and region list area. This is nice as it is fully interactive from Python and you can reconfigure video display interactively on the fly. That's at least one way it can be done already, but you need to code your PASM in a way that makes use of structures that Python can negotiate via its startup parameter area.

    Here's the thread link to the way it was done. That particular work was initially tested by using Eric's RISC-V version, but the COG spawning capability was also later ported into the native version after that. It works out okay, though it does use some extra heap overhead for initial string manipulation IIRC, hopefully it should be garbage collected later at some point.

    https://forums.parallax.com/discussion/comment/1486686/#Comment_1486686
    (there was another SVGA example posted later on too)

    Ideally we probably need some way to get objects stored in frozen code areas or SPI flash regions like Brian talked about. Also HyperFlash/HyperRAM offer some potential to be exploited by MicroPython down the track for extending capabilities now my driver is working, but that needs more consideration as to how to further integrate...too much to do!
  • My thoughts on MicroPython for P2 priorities:

    1. Module import capability with Mu File System
    2. Resolve board type (from ESP to a P2) for auto-detect
    3. Release the interpreter so people may experiment and work towards creating modules. Ask Parallax to contibute to the function call nomenclature.

    And maybe most importantly:

    4. Define the initial extent of features around which MicroPython for P2 will function and encapsulate those features in a release. This includes the extent of interaction between MicroPython and Spin2/PASM running in other cogs.

    MicroPython needn't do everything a P2 can do with Spin/PASM.

    There's more, for certain, but these are my notes from yesterday. TeamOz is doing excellent work for the community!

    Ken Gracey
  • Christof Eb.Christof Eb. Posts: 360
    edited 2020-07-23 - 19:56:52
    Sorry, I still do not get the point at all. First invest 12 man years to invent a new processor which has the unique feature of having 8 cores.
    And then make a software tool python top priority, which is 200 times slower than C and which does only support 1 core of 8. So only 1/(200*8) of the power of P2 is accessible with this tool. Even if you don't need the speed of C, you will be happy if your battery is running very much longer, if you can slow down the clock rate.

    If Micropython is a must for education, then there are other hardware solutions already available, which fit better.

    Sorry to be honest, but this decision of priority seems to be completely wrong!

    Kind regards Christof
  • Sorry, I still get the point at all. First invest 12 man years to invent a new processor which has the unique feature of having 8 cores.
    And then make a software tool python top priority, which is 200 times slower than C and which does only support 1 core of 8. So only 1/(200*8) of the power of P2 is accessible with this tool. Even if you don't need the speed of C, you will be happy if your battery is running very much longer, if you can slow down the clock rate.

    If Micropython is a must for education, then there are other hardware solutions already available, which fit better.

    Sorry to be honest, but this decision of priority seems to be completely wrong!

    Kind regards Christof

    Did you see the presentation yesterday?

    Ken Gracey
  • Christof Eb.Christof Eb. Posts: 360
    edited 2020-07-23 - 19:52:10
    Yes, I have seen it today on YT. Before I wrote my post.
  • I understand @Christof Eb. maybe better as an engineer's viewpoint.

    Educational customers are a very different audience. We would communicate the capabilities of MicroPython within the P2 as a system, rather than the P2's extensive feature set. Then, we work within these boundaries to deliver tutorials and examples. As it appears, the combination would be the most powerful MicroPython port once we develop modules to leverage the combination. It isn't common for customers in this realm to get wrapped up in clock cycles, code efficiency and what's missing based on the architecture of the processor.

    The bigger question in my mind is what we'd do with the P2 in education. I don't see strapping it on a rolling robot chassis as all that interesting, but making an educational program (remember the "Understanding Signals" tutorial?) that looks inside of communication protocols, analog and digital signals and parallel processing could be useful for engineers and education. The only problem with this approach is that the market is smaller than, say, robotics.

    All thoughts are welcome!

    Ken Gracey
  • Christof,
    Not sure where you got the idea that python (or micropython) is 200 times slower than C... maybe on some specific mcu/platform? Certainly not in general.

    Ken,
    I think putting a P2 on a rolling(or otherwise moving) robot is a very exciting idea. The extra memory, performance, and capabilities of the P2 alone will make for a much more capable and interesting robot.

  • Python is the “big” language these days!

    In particular, It’s being used for big data analysis and AI (artificial intelligence).

    I use it every day at work. Yes, it’s slow. But the work I do doesn’t require it to be fast.

    Basic was slow, then along came Visual Basic and it was slow too. When it gained enough users who complained about it being slow, but still continued to use it, MS built a compiler (the same base compiler as used in Visual C). I expect to see the same path with python, so probably micropython too at some stage.

    Currently (at least for now anyway) micropython can only utilise one cog/core. The other cogs can be pasm (for speed) or spin, or any other language given the compiler. Fastspin can do a multitude of languages already!

    I see the main program running on P2 likely to be micropython or spin2. On P1 that was spin. The main program in almost all circumstances does not need to be fast. It handles the main program, dedicating other cogs/cores to other time critical pasm code, and other cogs/cores to the slower things in spin2.

    So, IMHO, micropython will be a major player for education, and that then flows to commercial products. Micropython on the P2 will open up so many opportunities that other micros cannot possibly hope to do. Here the P2 is unique!!!
  • @Cluso99 I concur about the flow from educatoin to commercial. If a customer wants to design a product that can be run within MicroPython and the P2's combined feature set, why not? You may have heard from Tubular that he's doing exactly that - he's putting the P2 into their machines which get deployed to locations where customers will interact with the MicroPython interpreter to customize settings. I thought this was an interesting possibility - it's an idea that was very commonly used with BASIC Stamps.

    Ken Gracey
  • I finally had a chance to watch the MicroPython video. Congratulations! You guys did a fabulous job. The additions to allow self-hosting are very clever. So, that brings me to my question. It seems to me that a lot of the work that you have done isn't really MicroPython-specific at all and could be used as a framework for hosting other languages natively on the P2. I'm thinking of your method of redirecting the 63/62 serial traffic and the PASM editor as well as the memory viewer. Is that support code available yet for others to use? I'd like to try to use it with my BASIC interpreter. Right now I can support typing programs in on the P2 and running them but I'm using a clunky line editor. It would be wonderful to be able to use your screen editor instead. Where can I find the code?

    Thanks again for all of your hard work on this!
  • Ken,
    I never used the Basic Stamp so I’m in the dark here.

    I was sooo impressed with the micropython work the guys are doing. With an extension to access hub it can interact much better with other cogs/cores running other languages via a hub mailbox.

    Now that we have 0.5MB of hub RAM (not flash), plus hubexec, we have an extremely powerful micro that is completely different to any other out there. Large soft-loadable programs run nicely on this mix where flash based micros come unstuck.

    In just one (or two) cogs we can have a full-blown terminal using VGA or HDMI and keyboard (mouse optional) that can interact with any other cog(s). If you want a second terminal for debugging, no problem. This makes self-hosting a breeze.

    I think it would be great to get the spin/pasm (pnut) compiler section (ie not the editor) written in micropython (python initially as that would be the fastest way to develop and test on a PC). Python in many ways is more like spin that any other language. And you can compile python to an .exe. I think Chip would love python.
  • @Christof Eb.

    Here's a video from the MicroPython creator (Damien George) at PyCon AU demonstrating 7 different ways to blink a led within the MP environment

    The blink rates vary from ~58 kHz (naive, simple code) through 27,000 kHz using Viper mode, across the 7 examples. Thats a 500x speedup

    We don't yet have a Viper equivalent for P2 within that first MP cog, partly because we have 7 other cogs that can run pasm2, but we can head in that direction over time.

    https://www.youtube.com/watch?v=hHec4qL00x0&feature=youtu.be
  • Cluso99 wrote: »
    Basic was slow, then along came Visual Basic and it was slow too. When it gained enough users who complained about it being slow, but still continued to use it, MS built a compiler (the same base compiler as used in Visual C). I expect to see the same path with python, so probably micropython too at some stage.

    Ah, but there's a problem - the reason why there isn't and won't be really fast AOT compilers for full python,ruby,javascript and similar languages is because of their dynamic type systems - if you're not sure what type some value is, you waste valueable time figuring out:
    (Incidentally, I have to qualify "really fast AOT compilers for [language]" because there are AOT compilers for restricted subsets of dynamic languages - see RPython,mypyc,etc and of course there's JIT compilers for every major dynamic language.)
  • David Betz wrote: »
    I finally had a chance to watch the MicroPython video. Congratulations! You guys did a fabulous job. The additions to allow self-hosting are very clever. So, that brings me to my question. It seems to me that a lot of the work that you have done isn't really MicroPython-specific at all and could be used as a framework for hosting other languages natively on the P2. I'm thinking of your method of redirecting the 63/62 serial traffic and the PASM editor as well as the memory viewer. Is that support code available yet for others to use? I'd like to try to use it with my BASIC interpreter. Right now I can support typing programs in on the P2 and running them but I'm using a clunky line editor. It would be wonderful to be able to use your screen editor instead. Where can I find the code?

    Thanks again for all of your hard work on this!

    I talked about this with Brian after the presentation yesterday. We both think the thing to do is release a 'working set' combning all these components soon after MP 1.13 comes out.

    From there people can either use the combined system, or modify it, or extract individual components such as the editor or whole-of-hub memory persistence display

    This also gives us the chance to bake in a couple of necessary extras, including saving code to flash w/ wear levelling. It keeps our ongoing development quick in the mean time.

    I don't think it will be too long before MP 1.13 is released
  • Tubular wrote: »
    David Betz wrote: »
    I finally had a chance to watch the MicroPython video. Congratulations! You guys did a fabulous job. The additions to allow self-hosting are very clever. So, that brings me to my question. It seems to me that a lot of the work that you have done isn't really MicroPython-specific at all and could be used as a framework for hosting other languages natively on the P2. I'm thinking of your method of redirecting the 63/62 serial traffic and the PASM editor as well as the memory viewer. Is that support code available yet for others to use? I'd like to try to use it with my BASIC interpreter. Right now I can support typing programs in on the P2 and running them but I'm using a clunky line editor. It would be wonderful to be able to use your screen editor instead. Where can I find the code?

    Thanks again for all of your hard work on this!

    I talked about this with Brian after the presentation yesterday. We both think the thing to do is release a 'working set' combning all these components soon after MP 1.13 comes out.

    From there people can either use the combined system, or modify it, or extract individual components such as the editor or whole-of-hub memory persistence display

    This also gives us the chance to bake in a couple of necessary extras, including saving code to flash w/ wear levelling. It keeps our ongoing development quick in the mean time.

    I don't think it will be too long before MP 1.13 is released
    Sounds good. Thanks!

    I didn't hear anyone ask on the video but are you still using p2gcc to compile MicroPython? Do you have any plans to change that at some point since p2gcc isn't currently supported as far as I know?

  • TubularTubular Posts: 3,994
    edited 2020-07-23 - 23:30:39
    Yep, p2gcc while it continues to do the job. It may not be everything but we're grateful for what it has done
  • Hi,
    Speed and not to be able to deal with 7 of 8 cores are already mentioned as disadvantages of python on P2. The next item is code size of the interpreter. Because there is no linker usually very many routines are built in. On P2 all this occupies Hub Ram. The video showed 128k heap left over. Here you have to store all code and data for Micropython and the other cores.

    How large is micropython for P2 at the moment?

    Python has some advantages:
    * You can interactively use large libraries like numpy, written in C like Matlab.
    * Portability between systems. But this only applies, if you have same libraries on different hardware like CC on arduino.

    If Python has to be teached, I would use a system with very high clock speed and big memory. Or a system, where the micropython can stay in flash. Raspi is of course the competition here. Cheap, fast, available, well documented, powerful, lots of memory.

    I do think, that at some time the energy consumption of computers will matter. Python can then only survive, if it has a compiler. Because of this dynamic types it will even then be inefficient.
    It seems to be strange to me, to educate kids today with a language which is so inefficient. For battery powered things python is a problem.

    If Parallax wants to sell P2 they have to provide software which is known (not a special language like spin2, sorry) and which does support the specific strengths of P2.

    Best Regards Christof
  • roglohrogloh Posts: 2,482
    edited 2020-07-24 - 07:39:40
    One possible idea to increase the heap space is to try to make use of the HyperRAM. We'd need to see how easy that would be, because a lot of the MicroPython variable access code might need to change in order to do this. It may turn out not to be practical to change and could slow things down in some places given individual single item accesses from HyperRAM via the mailbox are probably typically limited to about ~ 1 million per second when the clock is around 250MHz, but if this was achievable it could massively increase the Python program size into the megabytes from 128kB.

    The good thing is a lot of the variable access code is abstracted anyway and there already is quite a lot of inherent overhead in referencing the variables to begin with, so the added read/write latency of HypeRAM on top may not appear to be quite so bad, at least in some applications where the performance can be traded for program size...TBD.

    Another approach would be to somehow separate code from data, and keep Python variables in the heap but the code in external memory. Again not sure if/how this can be done. Probably more of a question needed to be asked of the creator of MicroPython (Damien George) as to its feasibility.
  • koehlerkoehler Posts: 594
    edited 2020-07-24 - 06:39:00
    Christof,

    While Python is definitely not a language one would expect to fit well in the uC space, it is exactly what Parallax needs to make it successful in the EDU market. Python is the modern day Basic. I didn't watch the replay yet, however for EDU I don't think Python's inefficiency is going to be a problem. What Parallax will need is some useful, small Python libraries.


    For the enthusiast/pro market, I think Parallax has made a real mistake with pushing Spin over C, however its Ken and Chip's money being spent. My concerns watching the sausage being made in this forum is that the P2 is now so complicated, unless some compiler is going to hide most of the internals.
  • An easy way to increase the micropython heap size is to use the riscvp2 compiler instead of p2gcc. The compressed RISC-V instructions are significantly smaller than P2 code. The performance difference between riscvp2 and p2gcc is pretty small (I think it was 10% or so?), and we've already established that micropython's performance is not a priority. So the tradeoff would be worth it IMHO. My last micropython release built with riscvp2 had 256K heap.

    riscvp2, since it already has a code cache built in, would also easily lend itself to running code from external memory such as flash, which would increase the heap size even more.

    I don't want to pursue any of this unless there's interest, though. My version of micropython has been shunted aside (for better or for worse) so there's not much motivation to work on it further.
  • ersmith wrote: »
    An easy way to increase the micropython heap size is to use the riscvp2 compiler instead of p2gcc. The compressed RISC-V instructions are significantly smaller than P2 code. The performance difference between riscvp2 and p2gcc is pretty small (I think it was 10% or so?), and we've already established that micropython's performance is not a priority. So the tradeoff would be worth it IMHO. My last micropython release built with riscvp2 had 256K heap.

    riscvp2, since it already has a code cache built in, would also easily lend itself to running code from external memory such as flash, which would increase the heap size even more.

    I don't want to pursue any of this unless there's interest, though. My version of micropython has been shunted aside (for better or for worse) so there's not much motivation to work on it further.
    It seems to me that it shouldn't be hard to make your riscvp2 version of MicroPython work with the rest of the framework that Team Oz has created. It should be possible to use it with the PASM editor as well as the hub memory viewer and the 63/62 redirect facility. I'm anxious for Team Oz to release the source for those support pieces.

  • Is there a chance of a very much stripped down subset version of python, which can be used as a scripting part of a flexc?

    For example can flexc compile
    http://www.tinypy.org/
    Or
    https://wiki.python.org/moin/PyMite

    The idea is to have C as the main thing.
  • Is there a chance of a very much stripped down subset version of python, which can be used as a scripting part of a flexc?

    For example can flexc compile
    http://www.tinypy.org/
    Or
    https://wiki.python.org/moin/PyMite

    I don't know how hard it would be to port those to flexc. Why don't you give it a try and let us know?

  • It seems to be strange to me, to educate kids today with a language which is so inefficient. For battery powered things python is a problem.

    +1
    When I discovered the Prop 1, I went straight to PropBasic; Easiest language to learn, spits out pure PASM. It pains me to have something work harder than it needs to.
  • Christof Eb.Christof Eb. Posts: 360
    edited 2020-07-25 - 14:21:31
    "I don't know how hard it would be to port those to flexc. Why don't you give it a try and let us know?"

    No. - Edit: Tinypy gives with FlexC error: inline struct expressions not supported yet. Catalina reports a different problem.

    Because I have no P2. Unfortunately it is not yet offered for a reasonable price as a board and in Germany.
    And because I don't think, it makes any sense to use Python exept perhaps for education, if some teacher think this is the only way. It's Parallax, who could have the benefit, to bring these 2 very different goals together.

    The libraries and their documentation (!!!!) has to be done twice, if things are independent. Flexc could be used as a some sort of linker to bring in only the needed libraries. Your compiler with a CMM and a tiny little scripting language, which has a subset of the syntax of Python seems perhaps to be a solution. PBasic is not at all a full blown MS-Basic too.

    Well, I don't think, that you have to do this for free. Other producers have to invest in software too. The great libraries of teensy or of esp32 did not come from alone. As a customer I do pay for software via a higher hardware price. Teensy sells the bootloader chip for about 7 $. https://www.pjrc.com/store/ic_mkl02.html So I assume, there is about 2$ for software per sold system? As only Parallax sell their P2, they have to put the price onto P2.
    Parallax does spend such money for spin2 and for pasm and for this debug()-function. The problem is, that they cannot spend the money for spin2+flexgui+catalina+micropython+tychyon+? So the question is, how can things be brought together? These tools have to be professional and to be very reliable. It is no fun at all, if you are fighting a bug and don't know, if is your bug or the compilers bug. What will Parallax do, if you do not continue your work?

    I see my role as a (potential) customer of the "makertype". I am not a software professional. I have tried quiet some boards in the past and different languages. Including spin1, forth and python. Using spin1 I was forced to use pasm1.
    I do think, I have some overview over the competition of makerboards and try to bring the point of view of makers, who know the arduino invironment, in here. As I work in a bigger company, I cannot imagine at all that a company with more than 5 employees would use a P2 (these days without a official datasheet) and with spin2.

    I did buy some p1s some years ago. I left p1 programming, because it was more easy to use arduino-cc and their libraries. P1 ran out of memory, sometimes the tools have not been relieable. Assembler was not needed any more using the very powerful Arduino-CC.

    I tried to contribute to the Parallax forums in the past by posting hopefully interesting projects, but Parallax seems not to care very much of this kind of showroom for their products. Contributions of the first forum have been lost. Long time ago lots of fun: https://forums.parallax.com/discussion/148487/drops-rain-and-sleep-and-the-prop-using-propbasic#latest

    Well, I do try to contribute to this development writing this point of view and in this case a suggestion, which is not so bad...

    @Mickster, yes PropBasic was a breakthrough for me too with P1!

    Best regards, Christof
  • Python's got a lot more utility than just education. Sometimes you need to prototype something very quickly, without worrying about optimization (because you need the big micro for other reasons, or just because man hours cost more) and want code that's very simple to modify on the fly. Sometimes you need to drag application coders into a micro project.

    There are plenty of edge case reasons why micropython makes sense. Is it a main-line solution? The future of microcontroller development? Nah, but I still have an angle grinder even though I shouldn't cut pizza with it. :)
  • Riley, welcome back!
Sign In or Register to comment.