While trying to get nanogui working (maybe with Parallax SSD1331 based Oled display), found a rather horrible bug in SPI implementation....
SPI has one function (transfer) for both read and write or both read and write at same time.
Guess I was assuming that when doing write without read, they would provide some safe buffer to write to.
Well, this is not right and if buff==NULL, you should not write what you read to the buf.
Doing so causes some rather bad crashing...
Fixed now. Guess I need to update the binaries at some point.
Guess didn't notice when writing just a few bytes, but writing a lot makes it go off the rails..
I'm also seeing that inside python code that you import, if it tries to import something with a long filename, it won't work.
It works from inside REPL, but not when running code... Going to have to live with that I guess, 'cause I've got no idea how to fix that...
Had to add a couple more modules to mpconfigureport.h, UCTYPES and FRAMEBUF.
Binary size was getting questionable, but remembered how to adjust heap size with #define USER_MEMORY 200*1024, so all good now.
Actually, I should probably increase that a bit before posting new binaries with SPI bug and this fixed...
@rayman Very nice work getting this going again! I downloaded your source code and built it. It crashed when I tried to run but I think that was due to running out of memory. In main.c there are some lines:
I think those values are too big; when I reduced to 200*1024 the crash went away.
Those defines are also the explanation for why the binary size increases when you remove VGA, the user memory pool increases in that case (I guess by more than the size of the VGA code).
Unfortunately I can't seem to get the SD card to work -- it can't find the module "pyb", even though it looks like it should be linked. There is a "machine" module, but it doesn't have an SDCard method.
Because I can't debug directly, I'm stabbing in the dark here, but I'm guessing the pye editor crash might be related to either the memory running out (see above) or else the new memset addition to asm.templ in riscvp2. If it's the latter then changing memset to memset_dont_use everywhere in that file should fix it. But I'd suspect the memory first.
Sorry I haven't been able to support you more with MicroPython, I haven't had much free time lately due to a combination of work and personal issues .
Some more good news!
With the newest version of riscvp2 (fixed by disabling memset), the -HIMEM-flash option combined with "LDFILE=riscvp2_flash.ld" in the makefile, now has a working PYE.
This is great. Should be able to dramatically increase code size now, right?
Seems the only nanogui demo that works on 96x64 OLED is the color96 demo.
Kind of disappointing, but good news is that this demo shows how to do a lot of stuff.
@ersmith Playing around to see how much stuff we can to Micropython with the -HIMEM=flash option enabled so that code is stored in flash instead of HUB RAM...
with user memory increased 50% to:
"#define USER_MEMORY 300*1024"
It appears to work and have:
text data bss dec hex filename
294540 16936 320040 631516 9a2dc build/firmware.elf
This seems to still have plenty of margin. But, with user memory at 400*1024, get:
region `RAM' overflowed by 36312 bytes
Does work at 350*1024 too:
text data bss dec hex filename
294540 16936 371240 682716 a6adc build/firmware.elf
Was wondering how much moving code to flash would slow down the system.
Also curious how much code compression slows things downs.
So, dug up this pystone_lomem test from here: https://forum.micropython.org/viewtopic.php?f=2&t=2659
Results are strange:
compressed? HIMEM?
y y 194,205,194
n y 85,89,88
n y 134,141,132
y n 140,140,140
Seems speed is actually faster when code is compressed and stored in flash...
Speed is nothing to write home about though, compared to the others...
But, might be doing something wrong...
Trouble in paradise...
Was enjoying the relatively enormous free RAM and code space with the HIMEM=flash option.
Thought it was all fixed with the latest version of riscvp2.
But, my previous test for "working" was being able to load up the pye.py text editor.
Unfortunately, discovered today that while loads fine, it locks up after 6 or so edits.
So, not quite ready it seems...
Going to assume that @ersmith will get HIMEM=flash working one day and have switched back to regular loader.
Just have to reduce RAM a bit to get LFS2 included. Maybe doesn't matter, amount of free RAM is still a lot...
LFS2 appears to work as a ramdrive. So, should be able to mount the upper part of the boot flash chip as a drive.
There is code around to do this kind of thing, just need to adapt it...
Also noticed that when switched things from pyb to p2, left off a couple things like CPU and CSR.
Added CPU in already, so can launch cogs with assembly code.
Just need to bring CSR back. Appears that riscvp2 has been improved so that it can now run arbitrary P2 assembly instructions. Think that could mean that something like inline assembly is possible...
@Rayman said:
@ersmith With HIMEM, seems there's enough room to add in LittleFS...
Is it LFS2 that should match FlexProp or LFS1?
LFS_VERSION is defined as 0x002002, so I guess that's LFS2.
Trouble in paradise...
Was enjoying the relatively enormous free RAM and code space with the HIMEM=flash option.
Thought it was all fixed with the latest version of riscvp2.
But, my previous test for "working" was being able to load up the pye.py text editor.
Unfortunately, discovered today that while loads fine, it locks up after 6 or so edits.
So, not quite ready it seems...
That sounds like running out of HUB RAM. Even if the program technically "fits" in the RAM, the stack could still overflow down into the data area. I would test with the user memory define in main.c set to something ridiculously low (like 128K) and only once everything is known to be working and stable gradually increase it. I don't know how much stack, exactly, is required for micropython these days, but it's probably wise to leave at least 32K free.
@ersmith Thought of that already... Tried with #define USER_MEMORY 100*1024 to same result...
It's doing a lot of things right to get as far as it does. It's a strange bug...
Also went back to when pye first started working with HIMEM and it has same symptoms too.
Guess need a more rigorous test plan...
@ersmith you probably knew this already, but have figured out that the easiest way to add an optional feature is to look at its source code to see what needs to be defined in mpconfigureport.h
Feeling like close to having everything we need now though
Another nice feature learned is that some modules, like asyncio, can just live as folders on the filesystem. Doesn't have to be included in the binary...
For now, moving on from improving Micropython to actually using it for something.
First big challenge is to launch an assembly driver for a 2.4" TFT display into a new cog and then interact with it via Micropython.
Somewhat hampered by not knowing Python that well (but learning )...
But, made big progress today and can now initialize the LCD and clear the screen!
This assembly driver is compiled and then renamed to ili934x.binary and put on uSD along with Eval2p4.py
One thing did learn is that Notepad is no good for editing python scripts.
Seems it messed the indentation up when mixing tabs and spaces.
Reminds me of Spin...
Anyway, switched to VIsual Studio Professional and all better.
The microgui example on Github is pretty good. But, the problem there is using HUB for framebuffer. 320x240 at 16bpp is a lot (150kB).
Looks like their solution is to use a 4bpp buffer (38 kB).
The LCD already has a buffer though, so not really needed. With this 8 bit interface, can redraw the screen pretty quickly.
microgui looks to be using SPI, so that would be a lot slower...
This assembly driver has routines for lines and circles and text.
Will first look at what one can do with that...
Just found a new environment to work with micropython inside Visual Studio Community...
Has been a pain to edit python code, move the uSD to PC, copy the files, move uSD back to P2 and then go to FlexProp and run the binary.
So, just added this to the @ersmith "shell.c" demo, in the main routine, before going interactive:
//RJA Adding file copy here
r = mount("/sd", _vfs_open_sdcard());
if (r == 0) {
printf("mounted uSD /sd\n");
char* fn = "Eval2p4.py";
char fn1[80] = "/host/";
char fn2[80] = "/sd/";
strcat(fn1, fn);
strcat(fn2, fn);
do_copy(fn1, fn2);
printf("RJA: Copy of file %s from %s to %s complete!\n",fn,fn1,fn2);
}
printf("Run firmware.bin? (Y)");
char c = getchar();
if ((c=='Y')||(c == 'y'))
_execve("/sd/firmware.bin", 0, 0);
So now, when I hit the play button, it copies the .py file being worked on over to P2's uSD and then launches Micropython from uSD.
What also found is that the terminal window doing it this way is in color. Think that should mean work better for things like pye...
@ersmith Seems that using _execve() is fine, except that uSD doesn't mount. Is there any way to fix that?
Also seeing that need to copy your pyb.SDCard() over to p2 so can mount after the fact...
Actually, looks like SDCard is supposed to be part of machine now...
This looks like a fair amount of work to implement... Think going to skip for now as uSD already works in the usual case.
I got SDCard to show up under P2, but it doesn't work. Just going to punt on this for now...
Comments
These kind of options moved to platform.h
While trying to get nanogui working (maybe with Parallax SSD1331 based Oled display), found a rather horrible bug in SPI implementation....
SPI has one function (transfer) for both read and write or both read and write at same time.
Guess I was assuming that when doing write without read, they would provide some safe buffer to write to.
Well, this is not right and if buff==NULL, you should not write what you read to the buf.
Doing so causes some rather bad crashing...
Fixed now. Guess I need to update the binaries at some point.
Guess didn't notice when writing just a few bytes, but writing a lot makes it go off the rails..
I'm also seeing that inside python code that you import, if it tries to import something with a long filename, it won't work.
It works from inside REPL, but not when running code... Going to have to live with that I guess, 'cause I've got no idea how to fix that...
IGNORE THE ABOVE. Seems was operator error...
Got a nanogui demo going on Parallax Oled module.
Had to add a couple more modules to mpconfigureport.h, UCTYPES and FRAMEBUF.
Binary size was getting questionable, but remembered how to adjust heap size with #define USER_MEMORY 200*1024, so all good now.
Actually, I should probably increase that a bit before posting new binaries with SPI bug and this fixed...
This nanogui demo (colors96) was a lot more boring that it was supposed to be...
Had to fix os.urandom() to make it right.
all better now...
Next obstacle is don't have asyncio. Hoping can add that in on the filesystem though...
The tbox.py demo for nanogui needs it as is...
@rayman Very nice work getting this going again! I downloaded your source code and built it. It crashed when I tried to run
but I think that was due to running out of memory. In main.c there are some lines:
I think those values are too big; when I reduced to 200*1024 the crash went away.
Those defines are also the explanation for why the binary size increases when you remove VGA, the user memory pool increases in that case (I guess by more than the size of the VGA code).
Unfortunately I can't seem to get the SD card to work -- it can't find the module "pyb", even though it looks like it should be linked. There is a "machine" module, but it doesn't have an SDCard method.
Because I can't debug directly, I'm stabbing in the dark here, but I'm guessing the pye editor crash might be related to either the memory running out (see above) or else the new memset addition to
asm.templin riscvp2. If it's the latter then changingmemsettomemset_dont_useeverywhere in that file should fix it. But I'd suspect the memory first.Sorry I haven't been able to support you more with MicroPython, I haven't had much free time lately due to a combination of work and personal issues
.
Regards,
Eric
@ersmith make sure you have Eval board selected in platform.h
It might be set to SimpleP2 with uSD on other pins
Haven’t seen it crash, but I also reduced heap a few posts back…
I’ll try using flash for code again, thanks
There is definitely a problem with the latest asm.templ file in the riscvp2 master files. I’ll try your memset idea
@ersmith also the pyb stuff was renamed to p2. Not sure how you feel about that, but I like it better…
Didn't like having that bad SPI version hanging out there too long, so posted the latest version here:
https://www.rayslogic.com/Propeller2/upy/micropython.html
This one has the SPI fix and the added extras to run the colors96.py demo correctly.
Some more good news!
With the newest version of riscvp2 (fixed by disabling memset), the -HIMEM-flash option combined with "LDFILE=riscvp2_flash.ld" in the makefile, now has a working PYE.
This is great. Should be able to dramatically increase code size now, right?
Adding asyncio folder to uSD and then adding this to mpconfigport.h:
#define MICROPY_PY_SELECT (1)the tbox demo runs. But, it errors out because the screen is too tiny.
Tried to make it work, but no luck.
Bad news is that SELECT makes the binary bigger.
But, not so worried about that now as the new riscvp2 seems to work with HIMEM=flash now.
Seems the only nanogui demo that works on 96x64 OLED is the color96 demo.
Kind of disappointing, but good news is that this demo shows how to do a lot of stuff.
@ersmith Playing around to see how much stuff we can to Micropython with the -HIMEM=flash option enabled so that code is stored in flash instead of HUB RAM...
with user memory increased 50% to:
"#define USER_MEMORY 300*1024"
It appears to work and have:
text data bss dec hex filename
294540 16936 320040 631516 9a2dc build/firmware.elf
This seems to still have plenty of margin. But, with user memory at 400*1024, get:
region `RAM' overflowed by 36312 bytes
Does work at 350*1024 too:
text data bss dec hex filename
294540 16936 371240 682716 a6adc build/firmware.elf
Was wondering how much moving code to flash would slow down the system.
Also curious how much code compression slows things downs.
So, dug up this pystone_lomem test from here:
https://forum.micropython.org/viewtopic.php?f=2&t=2659
Results are strange:
Seems speed is actually faster when code is compressed and stored in flash...
Speed is nothing to write home about though, compared to the others...
But, might be doing something wrong...
@ersmith With HIMEM, seems there's enough room to add in LittleFS...
Think have it enough to show up anyway:
Is it LFS2 that should match FlexProp or LFS1?
Trouble in paradise...
Was enjoying the relatively enormous free RAM and code space with the HIMEM=flash option.
Thought it was all fixed with the latest version of riscvp2.
But, my previous test for "working" was being able to load up the pye.py text editor.
Unfortunately, discovered today that while loads fine, it locks up after 6 or so edits.
So, not quite ready it seems...
Going to assume that @ersmith will get HIMEM=flash working one day and have switched back to regular loader.
Just have to reduce RAM a bit to get LFS2 included. Maybe doesn't matter, amount of free RAM is still a lot...
LFS2 appears to work as a ramdrive. So, should be able to mount the upper part of the boot flash chip as a drive.
There is code around to do this kind of thing, just need to adapt it...
Also noticed that when switched things from pyb to p2, left off a couple things like CPU and CSR.
Added CPU in already, so can launch cogs with assembly code.
Just need to bring CSR back. Appears that riscvp2 has been improved so that it can now run arbitrary P2 assembly instructions. Think that could mean that something like inline assembly is possible...
LFS_VERSIONis defined as0x002002, so I guess that's LFS2.That sounds like running out of HUB RAM. Even if the program technically "fits" in the RAM, the stack could still overflow down into the data area. I would test with the user memory define in main.c set to something ridiculously low (like 128K) and only once everything is known to be working and stable gradually increase it. I don't know how much stack, exactly, is required for micropython these days, but it's probably wise to leave at least 32K free.
@ersmith Thought of that already... Tried with
#define USER_MEMORY 100*1024to same result...It's doing a lot of things right to get as far as it does. It's a strange bug...
Also went back to when pye first started working with HIMEM and it has same symptoms too.
Guess need a more rigorous test plan...
@ersmith you probably knew this already, but have figured out that the easiest way to add an optional feature is to look at its source code to see what needs to be defined in mpconfigureport.h
Feeling like close to having everything we need now though
Another nice feature learned is that some modules, like asyncio, can just live as folders on the filesystem. Doesn't have to be included in the binary...
For now, moving on from improving Micropython to actually using it for something.
)...
First big challenge is to launch an assembly driver for a 2.4" TFT display into a new cog and then interact with it via Micropython.
Somewhat hampered by not knowing Python that well (but learning
But, made big progress today and can now initialize the LCD and clear the screen!
This assembly driver is compiled and then renamed to ili934x.binary and put on uSD along with Eval2p4.py
One thing did learn is that Notepad is no good for editing python scripts.
Seems it messed the indentation up when mixing tabs and spaces.
Reminds me of Spin...
Anyway, switched to VIsual Studio Professional and all better.
The microgui example on Github is pretty good. But, the problem there is using HUB for framebuffer. 320x240 at 16bpp is a lot (150kB).
Looks like their solution is to use a 4bpp buffer (38 kB).
The LCD already has a buffer though, so not really needed. With this 8 bit interface, can redraw the screen pretty quickly.
microgui looks to be using SPI, so that would be a lot slower...
This assembly driver has routines for lines and circles and text.
Will first look at what one can do with that...
Just saw mention of ulab for Micropython, a lightweight version of numpy.
Seems to be a C module though, so would eat up RAM:
https://github.com/v923z/micropython-ulab?tab=readme-ov-file
Just found a new environment to work with micropython inside Visual Studio Community...
Has been a pain to edit python code, move the uSD to PC, copy the files, move uSD back to P2 and then go to FlexProp and run the binary.
So, just added this to the @ersmith "shell.c" demo, in the main routine, before going interactive:
//RJA Adding file copy here r = mount("/sd", _vfs_open_sdcard()); if (r == 0) { printf("mounted uSD /sd\n"); char* fn = "Eval2p4.py"; char fn1[80] = "/host/"; char fn2[80] = "/sd/"; strcat(fn1, fn); strcat(fn2, fn); do_copy(fn1, fn2); printf("RJA: Copy of file %s from %s to %s complete!\n",fn,fn1,fn2); } printf("Run firmware.bin? (Y)"); char c = getchar(); if ((c=='Y')||(c == 'y')) _execve("/sd/firmware.bin", 0, 0);So now, when I hit the play button, it copies the .py file being worked on over to P2's uSD and then launches Micropython from uSD.
What also found is that the terminal window doing it this way is in color. Think that should mean work better for things like pye...
@ersmith Seems that using _execve() is fine, except that uSD doesn't mount. Is there any way to fix that?
Also seeing that need to copy your pyb.SDCard() over to p2 so can mount after the fact...
Actually, looks like SDCard is supposed to be part of machine now...
This looks like a fair amount of work to implement... Think going to skip for now as uSD already works in the usual case.
I got SDCard to show up under P2, but it doesn't work. Just going to punt on this for now...