Shop OBEX P1 Docs P2 Docs Learn Events
Lua for P2? — Parallax Forums

Lua for P2?

RaymanRayman Posts: 13,797
edited 2017-03-19 03:38 in Propeller 2
Stumbled across this language Lua while googling the Roblox game my son is addicted to..
Says it's also what Angry Birds is programmed in...

https://en.wikipedia.org/wiki/Lua_(programming_language)

https://www.lua.org/about.html

What's nice about it to me is that it has a C-ish looking syntax but also something like Spin and that the bytecode interpreter comes in at 100 kB on a PC. If it is also 100 kB on P2, this sounds really nice.

If P2 does get that full 512kB, using a100 kB for the interpreter doesn't sound too bad. I think P1 used 16 kB out of 64 kB (RAM+ROM) for the interpreter.
«134

Comments

  • RaymanRayman Posts: 13,797
    Wonder if the bytecode format is a performance issue though...

    Looking here:
    http://luaforge.net/docman/83/98/ANoFrillsIntroToLua51VMInstructions.pdf

    There are three instruction types and 38 opcodes (numbered 0 through 37) are currently in use as of Lua 5.1. The instruction types are enumerated as iABC, iABx, iAsBx, and may be visually represented as follows:

    The bytecodes aren't really bytes, they are 6 bits.
    Also, the operands are 8,9 or 18 bits.
    How big a problem is this?
    637 x 162 - 6K
  • Heater.Heater. Posts: 21,230
    Should be quite doable. Lua runs on those little ESP WIFI modules. I think there was even a Lua running on the P 1 with an external RAM, compiled with Catalina.
  • jmgjmg Posts: 15,140
    Rayman wrote: »

    Says this :
    "If you need even more speed, try LuaJIT, an independent implementation of Lua using a just-in-time compiler."

    "Lua is small

    Adding Lua to an application does not bloat it. The tarball for Lua 5.3.4, which contains source code and documentation, takes 297K compressed and 1.1M uncompressed. The source contains around 24000 lines of C. Under 64-bit Linux, the Lua interpreter built with all standard Lua libraries takes 246K and the Lua library takes 421K"


    I guess as soon as P2 has C compiler support, a size-check Lua can be built ?
  • jmgjmg Posts: 15,140
    Rayman wrote: »
    Wonder if the bytecode format is a performance issue though...
    The bytecodes aren't really bytes, they are 6 bits.
    Also, the operands are 8,9 or 18 bits.
    How big a problem is this?

    ? I read that as this

    "Lua instructions have a fixed size, using a 32 bit unsigned integer data type by default. "

    So the byte-code engine, needs to fetch always 32b (not bytes)

    The 6b is the operand portion, so it is quite like P2 in that aspect. Most MCUs have a smaller actual opcode slice.

    I think P2 can extract the 9:9:8:6 and 18:8:6 splits they show readily enough ?

  • RaymanRayman Posts: 13,797
    edited 2017-03-18 20:57
    I think for an education market, there's something good about being able to program in the exact same language as ROBLOX and Angry Birds.

    I think my son would be interested in being able to code for ROBLOX anyway...
  • Lua fascinates me, too. It's one of the languages that runs on the little Canon Elph camera that I acquired awhile back. It seems like a nice fit for the Prop chips. 'Not sure what gymnastics are necessary to make it multiprocessor-aware, though.

    -Phil
  • RaymanRayman Posts: 13,797
    That's an interesting point...

    It has coroutines, but they aren't real multithreading...

    There is an add on library that adds real multithreading called "lanes".
    Not ideal...
  • Heater.Heater. Posts: 21,230
    I think the best idea would be to run multiple Lua instances, one per COG, and provide a library that allows for message passing between them.
  • RaymanRayman Posts: 13,797
    Was looking some more and it was looking good until I saw the
    Lua symbol for "not equal" is ~=

    This just kills me... Why use a C-like syntax and then do that...
  • Cluso99Cluso99 Posts: 18,066
    edited 2017-03-19 03:27
    The P1 spin interpreter is 2KB (496 longs). It fully loads from hub rom and runs within a cog.
  • RaymanRayman Posts: 13,797
    Yes, you're right...

    Maybe there's no way to compete with an interpreter that can fit inside a cog...
  • jmgjmg Posts: 15,140
    Rayman wrote: »
    Yes, you're right...

    Maybe there's no way to compete with an interpreter that can fit inside a cog...

    In P1, there was little choice, the Spin Byte-code token engine had to fit into a COG.

    In P2, most of the Lua 6b token engine can fit inside a COG, with some elements ok if using HubEXEC.
    The speed hit in P2 of COG -> HUB is tolerable, and the least-used tokens can call routines in HUB, as needed.

  • kwinnkwinn Posts: 8,697
    jmg wrote: »
    Rayman wrote: »
    Yes, you're right...

    Maybe there's no way to compete with an interpreter that can fit inside a cog...

    In P1, there was little choice, the Spin Byte-code token engine had to fit into a COG.

    In P2, most of the Lua 6b token engine can fit inside a COG, with some elements ok if using HubEXEC.
    The speed hit in P2 of COG -> HUB is tolerable, and the least-used tokens can call routines in HUB, as needed.

    Don't forget that P2 cogs also have the lut, which can also be used for code.
  • jmgjmg Posts: 15,140
    kwinn wrote: »
    Don't forget that P2 cogs also have the lut, which can also be used for code.
    Of course, and it also has 16 COGs.

    I'm wondering about a dual-COG engine, where one reads Lua codes, and does some prep work (almost JIT Compile) to feed to the second COG that does the table-playback.

    This depends on how much 'prep vs playback' goes on in the Lua handling, but it would give a compelling demonstration of the use of co-operating COGs in P2.

  • kwinnkwinn Posts: 8,697
    jmg wrote: »
    kwinn wrote: »
    Don't forget that P2 cogs also have the lut, which can also be used for code.
    Of course, and it also has 16 COGs.

    I'm wondering about a dual-COG engine, where one reads Lua codes, and does some prep work (almost JIT Compile) to feed to the second COG that does the table-playback.

    This depends on how much 'prep vs playback' goes on in the Lua handling, but it would give a compelling demonstration of the use of co-operating COGs in P2.

    Something along the lines of a software version of the fetch/decode/execute hardware pipeline in software. Certainly worth looking at for an interpreted/bytecode language
  • Keep in mind that there might be more than one instance of Lua. With Spin we can have 8 different Spin programs running at the same time on P1 and most likely 16 on P2. Do we want to limit Lua to running only 8 on P2 because we use two COGs apiece?
  • jmgjmg Posts: 15,140
    David Betz wrote: »
    Keep in mind that there might be more than one instance of Lua. With Spin we can have 8 different Spin programs running at the same time on P1 and most likely 16 on P2. Do we want to limit Lua to running only 8 on P2 because we use two COGs apiece?
    Of course, there can be both a Speed optimized version (which is the one that gets most attention) and also a Size optimized one, less 'sizzle' but probably more widely used :)

    I was thinking of ways to get the MCU worlds attention, and a Prep/Playback 'JIT' seems like it could give a performance gain.

  • kwinnkwinn Posts: 8,697
    David Betz wrote: »
    Keep in mind that there might be more than one instance of Lua. With Spin we can have 8 different Spin programs running at the same time on P1 and most likely 16 on P2. Do we want to limit Lua to running only 8 on P2 because we use two COGs apiece?

    No reason to limit yourself to one or the other since it is done in software. Use a dual cog version where the speed is required and a single cog version when it is not. That kind of flexibility is one of the great features of the Propeller.
  • I realize I'm chiming in from left field here, but as a bit of food for thought on this from someone with a lot of app dev experience and relatively little Propeller experience:

    Lua is great language choice for when you want to add dynamic pieces in an environment where things are statically compiled. It's been used a lot in, e.g. games where the game is written in C++ and Lua is used to define a level, or a behavior or a scene - something self contained that can be loaded separately by the statically compiled program. It also works well as a config file language. It's great for that sort of thing. I can see it having good use on P2 in cases where your app really is a lot more sophisticated than what you can do currently on P1 - for example a fairly sophisticated web server which provides a web interface for a bunch of sensors is a realistic scenario - using Lua as configuration and/or to script additional behavior into the device could be seriously awesome.

    That said, IMO, writing large programs in Lua can be (and usually is) cumbersome and gets counter productive after your program grows to a certain size (I could make this argument about any loosely typed language but it definitely applies here). Which makes me think that it might be useful to implement in a way that integrates well with the Propeller GCC environment - as opposed to having something that is more like SPIN where it has it's own interpreter and doesn't talk very well with your C environment.

    For example, if a command were provided that compiles a .lua file to runnable code, it could essentially be a thin wrapper around something like:
    	char program_bytecode[] = {1,2,3/* raw lua bytecode, NOT source */};
    
    	int main(void)
    	{
    	    lua_State *L;
    
    	    L = luaL_newstate();                        /* Create Lua state variable */
    	    luaL_openlibs(L);                           /* Load Lua libraries */
    
    	    luaL_loadstring(L, program_bytecode);       /* Load the Lua script */
    
    	    lua_pcall(L, 0, 0, 0);                      /* Run the loaded Lua script */
    
    	}
    

    Part of the idea here could be that the compiler (Lua source -> Lua bytecode) could also be available to be called from Propeller C code, or not (and just stripped by the linker and not included). Although from what I can tell that would definitely require some mods to the existing Lua code base. But the idea is if you wanted to be able to upload a script to a running Prop and have it compile and execute it - that would be possible. But programs that are 100% Lua compiled using some sort of "propluac" command, the source -> bytecode would happen beforehand and that code could be omitted from the resulting binary.

    You'd probably need to fork the Lua code base and make whatever changes so it works well with the Prop architecture; but my point is that it may work to still have it as regular C code that can be called from C or C++ and can integrate with other larger code - and make use of existing libraries, etc.

    Maybe all of the craziness with memory models and what PropGCC has to deal with to make things work would not make this the greatest approach - maybe more optimization would be needed to make it small or fast enough - but I wanted to throw it out there just in case it could work.

    It if it did work, this also may be a possible approach to allow support for other languages, e.g. JS using http://duktape.org/

    Food for thought.
  • Heater.Heater. Posts: 21,230
    Sometimes I think there is some confusion over this word "embedded".

    Lua is described as a great embedded language.

    However that does not mean it's great for embedded systems as we all know and love. That is tiny computers embedded into systems that are not themselves computers.

    Rather it means an great scripting language to embed in some big application, like games, for normal computers. As bgp say above.

    The guys that were building the Electric Imp WIFI enabled SD card started out using Lua. The interpreter is very small after all. Weeks before product launch they switched to a different language, "Squirrel" because they realized the run time memory overheads of Lua were far to great. Each variable taking 40 bytes or some such.

    https://www.sparkfun.com/products/11395

    As bgp says, Javascript support is not far away. duktape is an example. Jerryscript is another. Or the fully featured Espruino.

    Who would do this? I don't know.
  • jmgjmg Posts: 15,140
    Heater. wrote: »
    The guys that were building the Electric Imp WIFI enabled SD card started out using Lua. The interpreter is very small after all. Weeks before product launch they switched to a different language, "Squirrel" because they realized the run time memory overheads of Lua were far to great. Each variable taking 40 bytes or some such.

    https://www.sparkfun.com/products/11395
    That's quite cool until you get to the caveat :
    Note: Although, the Electric Imp comes in an SD form factor, it isn’t compatible with standard SD devices. Development boards are available, though, in the related items below!

    SD form factor is nifty, but surely the whole point of that form factor, is to be SD Compatible ?
    Otherwise, all you have done is limit yourself to just 6io ? (and added a shipload of field confusion, as SD sockets are not what they seem...)
  • Heater. wrote: »
    Sometimes I think there is some confusion over this word "embedded".

    Lua is described as a great embedded language.

    However that does not mean it's great for embedded systems as we all know and love. That is tiny computers embedded into systems that are not themselves computers.

    Rather it means an great scripting language to embed in some big application, like games, for normal computers. As bgp say above.

    The guys that were building the Electric Imp WIFI enabled SD card started out using Lua. The interpreter is very small after all. Weeks before product launch they switched to a different language, "Squirrel" because they realized the run time memory overheads of Lua were far to great. Each variable taking 40 bytes or some such.

    https://www.sparkfun.com/products/11395

    As bgp says, Javascript support is not far away. duktape is an example. Jerryscript is another. Or the fully featured Espruino.

    Who would do this? I don't know.
    Do you have a link to Squirrel? Or is it proprietary? Any chance it would fit on P1?

  • KeithEKeithE Posts: 957
    edited 2017-03-22 21:09
    The SD card form factor (imp001) is intended for platform evaluation, and maybe early prototyping. That particular version is pretty old as well. See here for more options:

    https://electricimp.com/docs/gettingstarted/devkits/

    Another interesting data point might be how many choose Lua for ESP8266 development, and how many choose C, Micropython, Basic,...
  • jmgjmg Posts: 15,140
    edited 2017-03-22 21:16
    David Betz wrote: »
    Do you have a link to Squirrel? Or is it proprietary? Any chance it would fit on P1?
    http://www.squirrel-lang.org/

    More chance of fitting on P2, if it currently fits on Cortex parts....

    Addit : yes, link says this :
    both compiler and virtual machine fit together in about 7k lines of C++ code and add only around 100kb-150kb the executable size.
  • Heater.Heater. Posts: 21,230
    jmg,

    Good point. I never looked at the Electric Imp so closely. Only heard the developers yakking about it on the Amphour.

    So basically it's a forerunner to the ESP modules. Which, strangely enough, can also run Lua!

    It's hard to keep up with these developments....

    I get the idea that the product is not the IoT hardware exactly but the cloud service behind it.

    It's not for me.
  • Heater. wrote: »
    Weeks before product launch they switched to a different language, "Squirrel" because they realized the run time memory overheads of Lua were far to great. Each variable taking 40 bytes or some such.

    I believe that the memory usage was related to Lua tables.

    There's some information about the memory consumption of Lua tables here:

    http://wowwiki.wikia.com/wiki/Lua_object_memory_sizes

    "A new, empty table will consume 40 bytes.
    If a table is indexed by sequential integers, each index will take 16 bytes (not including any memory allocated for the value). If the becomes a hash, the index size will jump to 40 bytes each."
  • One thing to point out about the way that Imp does things (not that it has to be done this way):

    "There's no low level assembler access on the imp - sorry. User code only runs within the VM"

    That's one hassle with their platform. Everything needs to be ported to squirrel, and you live with the capabilities that they give you.
  • Heater.Heater. Posts: 21,230
    Understandable.

    If you could program the thing in assembler, or some level below their VM, then you could trash the whole device and subvert it for your own purposes.

    Other than supporting their cloud "IoT"service.

    Which is not the business model they have in mind.

    I love the IoT idea. I hate that many players are trying to box us into their services forever.
  • Ah - I wasn't even thinking about that. I was thinking about them preventing their potentially unsophisticated customers from unintentionally trashing devices and requiring truck rolls.
  • KeithE wrote: »
    Heater. wrote: »
    Weeks before product launch they switched to a different language, "Squirrel" because they realized the run time memory overheads of Lua were far to great. Each variable taking 40 bytes or some such.

    I believe that the memory usage was related to Lua tables.

    There's some information about the memory consumption of Lua tables here:

    http://wowwiki.wikia.com/wiki/Lua_object_memory_sizes

    "A new, empty table will consume 40 bytes.
    If a table is indexed by sequential integers, each index will take 16 bytes (not including any memory allocated for the value). If the becomes a hash, the index size will jump to 40 bytes each."

    Not attacking anyone, just commenting on their decision...

    I'm not a fan of bloat, but totally not a fan to fracture into yet another language to deal with a non-existent problem.

    Why does this matter given the minimum many kilobytes of RAM nowadays and for what 95% of the people will use the language for in these IOT applications?

    For example, I completed a volunteer IOT project in February to monitor a Curling club ice compressor. It uses exactly one input to the real world, one pushbutton. Interfaces to a tiny I2C display and to the cellphone network to send out text message alarms.

    The program wasn't very big to say the least and the OLED display was unnecessary but had a coolness factor as its always nicer to "know what's going on". Since ice takes hours to start melting, the only performance I needed was handled by a 3G cellphone modem onboard (some U-Blox thing). The server at the other end distributes the messages to those wanting to receive them.

    But it's bound to save so much gasoline and time to not keep having to voluntarily check up on something. The building has no internet access, and a monthly internet bill would be way overkill for an alarm that occurs between 0 to 3 times a month.
Sign In or Register to comment.