Micropython for P2

12467

Comments

  • jmgjmg Posts: 13,932
    ersmith wrote: »
    jmg wrote: »
    ersmith wrote: »
    and a USB keyboard (via @garryj's single COG USB driver, ported to C by me)..
    Nice.
    How does the C version compare with the Spin and p2asm versions in garryj's thread.
    Should the C version be included in that thread's first post, with notes on Size/speed/ when to choose each version ?

    The C version is a mechanical translation using spin2cpp. It's not necessary (at all) with fastspin, but I needed it because I wanted to compile the code with the Risc-V version of gcc. So it's probably not generally useful, although I suppose it might with some tweaking be usable under PropGCC or Catalina.
    Ah, ok. Does that mean your version runs the USB code, via the Risc-V emulator code ? I'd have expected that to be too different in timing ?

  • jmg wrote: »
    ersmith wrote: »
    jmg wrote: »
    ersmith wrote: »
    and a USB keyboard (via @garryj's single COG USB driver, ported to C by me)..
    Nice.
    How does the C version compare with the Spin and p2asm versions in garryj's thread.
    Should the C version be included in that thread's first post, with notes on Size/speed/ when to choose each version ?

    The C version is a mechanical translation using spin2cpp. It's not necessary (at all) with fastspin, but I needed it because I wanted to compile the code with the Risc-V version of gcc. So it's probably not generally useful, although I suppose it might with some tweaking be usable under PropGCC or Catalina.
    Ah, ok. Does that mean your version runs the USB code, via the Risc-V emulator code ? I'd have expected that to be too different in timing ?
    The Spin/C part runs in Risc-V emulation. The PASM part runs in another COG, natively.


  • Hi Eric
    II'm experiencing an issue with your self hosted upython when using paste mode.
    After pasting code and hitting ctrl-D it crashes and video is lost too.
    I have a test cog blinking an led and its still running aftr the crash.
    I can paste the same code into my self hosted upython and all is well.

    BTW I have found that the 230400 baud rate is too fast for upython pasting and requires inserting breaks to avoid lost characters. (upython reporst syntax errors)
    Melbourne, Australia
  • ozpropdev wrote: »
    Hi Eric
    II'm experiencing an issue with your self hosted upython when using paste mode.
    After pasting code and hitting ctrl-D it crashes and video is lost too.
    I have a test cog blinking an led and its still running aftr the crash.
    I can paste the same code into my self hosted upython and all is well.
    Are you sure you didn't hit ctrl-D twice? I think that shuts down upython. I wasn't able to reproduce this issue, although I've also changed the serial driver a bit (see below).
    BTW I have found that the 230400 baud rate is too fast for upython pasting and requires inserting breaks to avoid lost characters. (upython reports syntax errors)

    I've added a buffered serial driver that should do better at keeping up with pasted data (at least for the first 512 characters). The updated upython (v10) is in the first post, as usual.

    Thanks,
    Eric
  • Hi Eric
    Sadly it's not a double ctrl-D isuue with V9.
    Repeated tests gave same outcome, screen and upython crashes.

    V10 behaves differently again and crashes during download.
    It appears to be stuck in a buffer loop cycling the same characters.

    Here's a test sample that works on my V8 based self hosted upython and on my other micropython boards.

    Cheers
    Brian
    #boyer-moore algorithm test
    
    bad_table = []
    
    def setup(x):
        global bad_table
        bad_table = [len(x)]*128
        for i in range(len(x)):
            q = len(x) - i - 1
            if q < 1:
                q = 1
            w = ord(x[i])
            print(i,q,w)
            bad_table[int(w)] = q
    
    m="the quick brown fox jumps over the lazy dog"
    
    def find(s):
        setup(s)
        t=0
        z=0
        while True:
            print(m)
            print(" "*t,end="")
            print(s)
            psx = True
            for u in range(len(s)-1,-1,-1):
                z += 1
                if s[u] != m[t+u]:
                    t += bad_table[ord(m[t+u])]
                    psx = False
                    break
            if t + len(s) > len(m):
                print("No match found!")
                break
            if psx:
                print(z," comparisons made")
                break
    
    find("over")
    


    Melbourne, Australia
  • Sorry Brian! There was a typo in the serial receive code that could cause it to get stuck repeating characters. I've uploaded v11 which does run your example correctly. Thanks for the repo case.

    Eric
  • Thanks Eric!
    V11 looks good, nice work! :cool:

    Melbourne, Australia
  • I've just posted v12 (see the first post in this thread). This one has preliminary support for microSD cards on the P2 Eval board (i.e. it expects the SD card to be SPI driven on pins 58-61) and FAT file systems. I haven't done a lot of testing yet, but it's worked with three different cards I've tried with.

    execfile, open, close, read, and write all seemed to do the right thing. So now with a VGA monitor, SD card, and USB keyboard you can do Python development entirely on your P2 Eval board, and even save your work.

    Again, I haven't done a lot of testing so be *very* careful (don't try it with any SD Cards containing data you care about!).

    One of the cards wasn't automatically detected, so I had to manually mount it with:
    import pyb
    import os
    sd=pyb.SDCard()
    sd.power(1)
    os.mount(sd, '/sd')
    
    but once that was done it seemed to work OK.

    The updated source code is posted to my github micropython repository.

    Eric
  • rosco_pcrosco_pc Posts: 338
    edited 2019-06-25 - 05:23:32
    V12 is working nicely :)

    Trying to get mu-editor up and running and succeeded to some extend:
    * can open a serial terminal (REPL)
    (grr, can not upload a screenshot :()

    what is not working is
    * running a script or
    * accessing the SD card on the P2-eval board.

    The last requires raw_repl to be working, which does not seem to be activated in the current P2 micropython build.

    Question: how can we build micropython, ie what other tools are needed?

    Edit:
    ersmith wrote: »
    I haven't had time yet to organize the micropython build environment, but it's basically a standard micropython build using an rv32im RISC-V toolchain and linked with the RISC-V emulator from my github repository (https://github.com/totalspectrum/riscvemu).

    Let's see if I can get that working :)
    1211 x 890 - 79K
    Stockholm, Sweden
    I am only an egg -- Stranger in a Strange land, Robert A. Heinlein
  • I haven't used Mu yet; I had a lot of trouble getting it installed under Linux. I didn't realize that the raw repl would be useful, but it makes sense. I'm preparing a new release now, I'll add it in.

    I built the RISC-V toolchain myself from the official repository, but with the latest micropython github you can probably use any common rv32 toolchain, since the emulator (the p2trace one, at least) now supports compressed instructions. You'll also need fastspin to build the JIT compiler from RISC-V to P2 instructions.

  • I've updated the first post with v13, which now includes:

    - raw repl support
    - automatically runs "main.py" from the SD card at startup
    - better ANSI terminal support in the VGA code (so ANSI escape codes for things like underline and blinking work, as well as the standard 256 ANSI colors)
    - a faster RISC-V JIT engine, which also supports compressed code. Even with all the additions (like FAT file system, SD card support, floating point support, raw repl, etc.) we still have 200K of space left for user code.
    - debugging hooks added to the RISC-V emulator to make it easy to hook into the I/O
  • jmgjmg Posts: 13,932
    ersmith wrote: »
    I've updated the first post with v13, which now includes:

    - raw repl support
    - automatically runs "main.py" from the SD card at startup
    - better ANSI terminal support in the VGA code (so ANSI escape codes for things like underline and blinking work, as well as the standard 256 ANSI colors)
    - a faster RISC-V JIT engine, which also supports compressed code. Even with all the additions (like FAT file system, SD card support, floating point support, raw repl, etc.) we still have 200K of space left for user code.
    - debugging hooks added to the RISC-V emulator to make it easy to hook into the I/O

    Wow, I'm sure this impressive progress will have Ken drooling :)

  • Cluso99Cluso99 Posts: 15,410
    edited 2019-06-23 - 23:06:27
    Time to jump in feet first :smiley:

    Self hosting is a deal breaker :smiley: :smiley: :smiley:
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • I should point out that while the current version (v13) is self hosting in theory, and even somewhat in practice (I have saved some text to the SD card and then executed it) it's not user friendly at all yet. We still need a port/write a micropython editor to run on the device; or perhaps @ozpropdev 's P2 assembly editor can be made to work with it.
  • rosco_pcrosco_pc Posts: 338
    edited 2019-06-25 - 04:55:09
    ersmith wrote: »
    I haven't used Mu yet; I had a lot of trouble getting it installed under Linux. I didn't realize that the raw repl would be useful, but it makes sense. I'm preparing a new release now, I'll add it in.
    I only needed a 'git clone'. After that it was working. I added a P2 mode (basically a copy of the ESP mode) and to connect it to the P2-Eval board it needed to have the baudrate fixed in the code as mu-editor has it fixed to 115200. I opened an issue for that: https://github.com/mu-editor/mu/issues/865
    ersmith wrote: »
    I built the RISC-V toolchain myself from the official repository, but with the latest micropython github you can probably use any common rv32 toolchain, since the emulator (the p2trace one, at least) now supports compressed instructions. You'll also need fastspin to build the JIT compiler from RISC-V to P2 instructions.
    I'll try later today.
    ersmith wrote: »
    I've updated the first post with v13, which now includes:

    - raw repl support
    - automatically runs "main.py" from the SD card at startup
    - better ANSI terminal support in the VGA code (so ANSI escape codes for things like underline and blinking work, as well as the standard 256 ANSI colors)
    - a faster RISC-V JIT engine, which also supports compressed code. Even with all the additions (like FAT file system, SD card support, floating point support, raw repl, etc.) we still have 200K of space left for user code.
    - debugging hooks added to the RISC-V emulator to make it easy to hook into the I/O

    Thanks, you're incredibly responsive and fast.

    Update: success :) Can run code from mu. Still not working is file access but I think that is related to how mu handles the filesystem (started digging there now)
    scoaNVo
    ersmith wrote: »
    I should point out that while the current version (v13) is self hosting in theory, and even somewhat in practice (I have saved some text to the SD card and then executed it) it's not user friendly at all yet. We still need a port/write a micropython editor to run on the device; or perhaps @ozpropdev 's P2 assembly editor can be made to work with it.

    There is this one here: https://github.com/robert-hh/Micropython-Editor . I can not test it is working fro some time as I need to get a VGA and USB connector board soldered up (did not get the accessory boards :( )

    Stockholm, Sweden
    I am only an egg -- Stranger in a Strange land, Robert A. Heinlein
  • Here's some pics of my self hosted micropython effort.
    I'm using one of Tubular's compact dual vga boards with usb.
    The monitor on the left is displaying live hub activity.
    The Parallax analog joystick controls the zoom window.
    The monitor on the right is showing the colorized full editor(written in PASM2).
    It's very much a work in progress but has proved the concept very nicely!



    1394 x 1035 - 576K
    1394 x 1035 - 717K
    1394 x 1035 - 432K
    Melbourne, Australia
  • Ozpropdev, looks like you are well on your way to having a self-hosted system.
  • This is great stuff, kudos to you Eric!
  • OK, I thought I had mentioned it on this thread but apparently I didn't: the P2 source code is at

    https://github.com/totalspectrum/micropython

    The port is in the ports/riscv-p2 directory. Directions for obtaining the toolchain are at https://github.com/totalspectrum/riscvp2.

    Sorry about that, I had a distinct memory of posting that link -- it must have been in some other thread.

  • Oh, and the first post is updated to include an updated build with a few additional builtin modules (like regex parsing) which the pye.py editor use. pye.py runs now, although using it with the USB keyboard is a bit awkward because the cursor keys don't send the escape sequences that pye.py expects (instead they send the EMACS control keys that micropython uses). I suspect that it would not be hard to fix pye.py to use the EMACS sequences instead, but I'll leave that as an exercise for the reader :).
  • Thanks Eric. I hadn't realised the P2 micropython source was available. I'll have to check out the Pye editor

    I remember OzProp saying he had to do something with scancodes, but think it was function keys rather than cursor keys
  • Thanks Eric.
    I too wasn't aware of the availability of the source.
    I've been hacking your binary (v8) for all my micropython efforts.
    Things were getting complex.

    With regard to keycodes I ended up coming up with my own custom codes to cover cursor and editing/selection functions for my editor.
    Melbourne, Australia
  • I ended up creating a table to convert scancodes (which Garry' s USB driver provides) to ANSI escape sequences, like ESC [ A for cursor up. micropython understands those just fine, as does pye, so it seems like that's probably a better option than what I was doing before (mapping the key presses directly to the EMACS keys that micropython also understands, but which pye handles differently). That hasn't made it to the binary release but it's in github.

    There's still a problem with my VGA text driver not handling the ANSI sequences for scrolling regions, so screen redraws can get messed up sometimes in pye. I hope to have that fixed soon.
  • ersmithersmith Posts: 3,403
    edited 2019-07-14 - 22:55:31
    I've fixed the ANSI escape sequences in the VGA driver, so there's an updated binary release in the first post now. With v15 pye.py works correctly; just put pye.py (included in the zip and in the source repository) on an SD card and do:
    from pye import pye
    pye('myfile.py')
    
    It works best with a USB keyboard and VGA monitor plugged in, and/or with a terminal emulator that supports ANSI escape sequences and is set up with a 100x40 character window (the same as the VGA display).

    Now that it can use a full screen editor which is itself written in python I think "self-hosting" is very much enabled :).

  • WooHoo. Well done Eric!

    Self-hosting will be a great bonus for when the real silicon arrives :smiley:
    My Prop boards: P8XBlade2, RamBlade, CpuBlade, TriBlade
    Prop OS (also see Sphinx, PropDos, PropCmd, Spinix)
    Website: www.clusos.com
    Prop Tools (Index) , Emulators (Index) , ZiCog (Z80)
  • ersmith wrote: »
    I've fixed the ANSI escape sequences in the VGA driver, so there's an updated binary release in the first post now. With v15 pye.py works correctly; just put pye.py (included in the zip and in the source repository) on an SD card and do:
    from pye import pye
    pye('myfile.py')
    
    It works best with a USB keyboard and VGA monitor plugged in, and/or with a terminal emulator that supports ANSI escape sequences and is set up with a 100x40 character window (the same as the VGA display).

    Now that it can use a full screen editor which is itself written in python I think "self-hosting" is very much enabled :).

    Sounds pretty amazing, Eric.

    I imagine the editor was composed in a PC-hosted environment before being run on the P2. Does the conversion to bytecodes happen on the P2, or in the PC environment?
  • This is looking nice Eric. I had to take out some lines that resulted in "OS not supported" error message, then things work

    For those who want to check it out this is what I found I needed to do

    ## turn off the P123
    ## plug the VGA monitor into P123 vga board attached to P48~55
    ## plug USB keyboard to lower connector of P123 "serial host" board attached to P16~23
    ## rename Eric's v15 .binary to _BOOT_P2.BIX on an SD card
    ## edit/strip the last function (~15 lines) off pye.py and copy the remainder to pye2.py in root directory of the SD card
    ## (my pye2.py with lines already stripped off attached to this post, in zip file)
    ## turn off P123 flash switch (in case of difficulties)
    ## power up and wait for micropython prompt to appear on VGA screen (few seconds)
    ## type the following commands into the micropython repl
    import pyb
    sd=pyb.SDCard()
    sd.power(1)
    ## repl will echo 'True'

    import os
    os.mount(sd,'/sd')
    os.chdir('/sd')
    execfile("pye2.py")
    from pye import pye
    pye('pye2.py')
    ## that should bring up the editor and move around the ~970 line pye2 source.
    ## control-k-s to save back to SD card
    ## control-k-q to quit back to repl

  • cgracey wrote: »
    ersmith wrote: »
    Now that it can use a full screen editor which is itself written in python I think "self-hosting" is very much enabled :).

    Sounds pretty amazing, Eric.

    I imagine the editor was composed in a PC-hosted environment before being run on the P2. Does the conversion to bytecodes happen on the P2, or in the PC environment?

    The editor is one that someone else wrote for micropython. But you can edit and change it on the P2. The micropython binary I posted has the bytecode compiler built in, so all Python code is compiled on the P2. I think it's possible to pre-compile bytecodes, but I haven't really tried that and there's not much need (the compiler is fast enough). I guess it might be interesting for a really stripped down version of micropython, but then you'd lose the ability to type commands interactively.

  • Lachlan:

    I hadn't thought of booting micropython from the SD-card, thanks for trying that out!

    Interesting that you had to remove that bit at the end that tests for the OS. I actually hadn't tried "execfile", I'd just been using "from pye import pye"; I guess the OS check only happens in execfile. But thanks for the fix, I'll update my version of pye.py accordingly.

    It's a bummer that you have to manually mount the SD card. Any ideas of a good way to detect the presence of a card? I've been checking for pin 59 low, but it appears not all SD cards make this happen.
  • An SD card itself has a pull-up on its CS so check P60 for a high = card or else floating.

    Tachyon Forth - compact, fast, forthwright and interactive
    useforthlogo-s.png
    --->CLICK THE LOGO for more links<---
    P2 +++++ TAQOZ INTRO & LINKS +++++ P2 SHORTFORM DATASHEET
    P1 +++++ Latest Tachyon includes EASYFILE +++++ Tachyon Forth News Blog
    Brisbane, Australia
Sign In or Register to comment.