Spin Simulator

Dave Hein · 2010-12-15 19:26

This thread started out as a request for information on simulators that execute spin code. I then decided to write my own spin simulator which is call spinsim. The latest version of spinsim is included in this post.

The original message is given below.

Are there any simulators that run Spin bytecodes? I read about Gear and pPropellerSim, which are machine-level simulators that run PASM code. Has anybody run the Spin interpreter in these simulators?

A Spin simulator would allow Spin programs to run on other platforms. It would open up the use of the Spin programming language for other processors in addition to the Propeller. Certain functions, such as console I/O and file I/O could be handle by special objects that interface to the platform's OS.

Ravenkallen · 2010-12-15 19:38

I have wondered that to. It would be neat if someone could make a spin simulator.

Sapieha · 2010-12-15 19:52

Hi Dave.

Gear RUN entire Programs that are written for Propeller with some Visualization possibility's.

Dave Hein wrote: »

Are there any simulators that run Spin bytecodes? I read about Gear and pPropellerSim, which are machine-level simulators that run PASM code. Has anybody run the Spin interpreter in these simulators?

A Spin simulator would allow Spin programs to run on other platforms. It would open up the use of the Spin programming language for other processors in addition to the Propeller. Certain functions, such as console I/O and file I/O could be handle by special objects that interface to the platform's OS.

Dave Hein · 2010-12-20 13:36

I spent a few days writing a Spin simulator. The attached zip file contains a first cut at the simulator. I've implemented most of the Spin bytecodes, except for the cogxxx and lockxxx ones. Also the value for cnt is always zero. I'll transate the system time into a cnt value at some point. Pin-related operations don't really do anything. I also haven't implemented the bit read and write functions.

The simulator runs in a DOS window on a Windows machine. Extract the zip file into a convenient directory, such as the desktop. Start up a DOS window, and go into the spinsim001 directory. Run "spinsim hello.binary" and it should print "Hello World". Run "spinsim demo.binary" and it will execute a demo program that can print files in ASCII or hex. Try the "type hello.spin" and "dump hello.spin" commands in the demo program.

You can also get a listing of the bytecodes as they are executed. This is done with the "-l" option. The simulator can run the first N instructions by specifying the number of instructions, such as -20 for the first 20 instructions. Try running "spinsim -l hello.binary" to see all the instructions executed. "spinsim -l -20 hello.binary" will run the first 20 instructions and print them out.

Console and file I/O is implemented through rendezvous locations at $7ff8 and $7ffc. The simulator looks at these locations after executing each instruction, and it performs the specified command.

I used the same mnemonics that I used with the spasm spin bytecode assembler. Most of the mnemonics are fairly intuitive, except for maybe the memory access ones. There are four types - ld..., st..., la... and ex... for load, store, load-address and execute. Memory addressing is done relative to the object base, variable base, local stack base or an absolute addess -- these are specified with the letter o, v, l or a. Memory access sizes are either byte, word or long, which is specified with the letter b, w or l. As an example, a word can be loaded from the local stack area with a "ldwl" instruction (LoaD Word Local). A byte can be stored in the variable area with a "stbv" instruction (STore Byte Variable). The indexed mode can be specified by appending an "x" to the instruction, such as "ldwlx" and "stbvx". The first 8 longs in the local stack and variable areas can be accessed with a compact opcode by appending a "c" character, such as "ldllc" and "stlvc" (LoaD Long Local Compact, and STore Long Variable Compact). I'll write up a more complete description of the mnemonics later on.

Let me know if you have any questions or comments.

Dave

jazzed · 2010-12-20 14:36

Cool. Seems to work fine on Windows cmd shell.
Any chance to test this with wine on linux? I tried, but console IO fails.
When you have cognew and friends, I'll test it with my SPIN debugger

When it can run Sphinx, we'll have an open source solution (if you publish the source).

Dr_Acula · 2010-12-20 15:04

Memory addressing is done relative to the object base, variable base, local stack base or an absolute addess

Intriguing. Maybe this is a step towards Big Spin? It may well be quicker to build it first as a simulation, then port it to the real chip with external memory. So you don't have to worry initially about the nuts and bolts of the memory access code.

It seems that in order to write your simulation you have had to consider how memory works. That is fantastic.

Simple things first - define a byte in memory at a location higher than 32k. Can you read and write a byte to a location?
Second, define an array at a high memory location. This ought to be an extension of the above.
Third, and this is more complicated, integrate code from high memory.

I get confused thinking this through, but as a simple example, say you have a PUB and it is at the end of a large program and the program is over 32k. This PUB is different, because already it is going to be slower than PUBs near the beginning of the program. So the programmer has to think about that and put the fast PUBs near the beginning of the code.

What things does this PUB need to do?
Access bytes at fixed locations in hub ram and external ram. That should be easy.
Access bytes at variable locations in hub (and external) ram. This is more complicated, because the variable Myvariable might exist anywhere and only the compiler knows where. This is where your simulator could be the key. If you know where Myvariable ended up after compilation, then any PUB can access it, even PUBs that are in high memory.
What else? Call PUBs elsewhere in the program. Again, only after compilation will you know where they are.

The most complicated thing is how you run this PUB from external memory. You can run it as a LMM simulation in pasm. But maybe there is another way. If you move that PUB into hub ram, and if external PUBs were never greater than a certain size, then you could run that PUB as if it came from hub ram.

Or you might bring in external spin mnemonics one at a time, complete with references to locations of MyVariable which have been allocated as part of the compilation.

The part I have always got stuck on is what happens if that external PUB references a variable, eg Myvariable, which might be a common variable to the program? Only a compiler (or simulator) knows where Myvariable is.

Maybe there are other solutions too. How cool would it be to start off with a simulator with megabytes of memory! You can do that sort of thing in a simulator, then think about how it ports over to a real chip.

I'd be very interested to hear your thoughts on how you might solve these problems in a simulator.

Dave Hein · 2010-12-20 16:03

jazzed, I haven't tried it under linux, but I plan on posting the source code once I have cleaned it up a bit. Once I sweep all the cog-specific variables into a struct it should be pretty easy to implement 8 instances of the simulator. That will allow the capability of doing a cognew. It should be possible to run Sphinx under the simulator once I've added all the file support.

Dr_Acula, the simulator doesn't do any memory checking, so it is possible to execute out of a flat memory space that is much larger than 32K. The memory pointers in the Spin interpreter are all 32-bit longs. These are pbase, dbase, vbase, pcurr and dcurr. However, the interpreter currently uses 16-bit words for in the method table and the stack frame. These would have to be increased in size to make it work for larger hub RAM, like the Prop 2 will have.

With external memory, every memory access will have to be checked to see if it's referencing hub RAM or external RAM. Most operations use the stack, so it would be reasonable to require that the stack is always in hub RAM. Memory accesses could be localized to a few small routines, so it would be fairly easy to control which RAM is being accessed. The tough part is fitting it in a single cog. I think SpinLMM could be extended further so that external memory routines could be added.

stevenmess2004 · 2010-12-20 22:40

Dave Hein wrote: »

The memory pointers in the Spin interpreter are all 32-bit longs. These are pbase, dbase, vbase, pcurr and dcurr. However, the interpreter currently uses 16-bit words for in the method table and the stack frame. These would have to be increased in size to make it work for larger hub RAM, like the Prop 2 will have.

But most the addresses in an object are jumps not absolute addresses so there is still a reasonable chance it will work.

Dave Hein · 2010-12-23 20:14

stevenmess2004 wrote: »

But most the addresses in an object are jumps not absolute addresses so there is still a reasonable chance it will work.

Jumps use relative addressing, so they shouldn't have to change for a larger memory. They are limited to a +/- 16K range, but that should be OK since jumps are only done within the boundaries of a method. pbase, dbase, vbase and dcurr are all long addresses, which means that the two least significant bits are zero. These 2 bits could be used to extend the addressing to 256K. pcurr is a byte address, and it would require a full 18 bits for 256K.

I've implemented all of the Spin operators and I moved the cog variables into a struct. This allowed me to create 8 instances of the variables. The simulator can now run all 8 cogs. I added another test program call test.spin. It starts up a new cog once per second. I also added "dir" and "run" commands to demo.spin. You can run hello.binary or test.binary from demo.binary.

The attached zip file contains all the source and binary files. The simulator is implemented in the three C files -- spinsim.c, debug.c and interp.c. There are very few comments. I'll add that later.

jazzed, I tried to port Sphinx to the simulator. I was able to get sphinx, lex and codegen to run separately. However, I couldn't get it to compile conio.spn because it didn't like something in the syntax. The Sphinx compiler implements a limited syntax, so there must be something in conio.spn that it doesn't support.

Dave

mpark · 2010-12-23 21:54

Dave Hein wrote: »

However, I couldn't get it to compile conio.spn because it didn't like something in the syntax.

What error message did Sphinx give?

jazzed · 2010-12-23 22:06

Cool. I dinked around with the files and have something that builds/works on Linux to a point. I had to add a unixy kbhit() and getch() in conion.c/h to make it work. Need to port directory handling for command == SYS_FILE_READDIR - it's commented out for linux in the attachment. Here's what I have if you want to incorporate changes so I don't have to hack it again. Added a simple Makefile. Also note that I added check for char == 10 in demo.spin::gets.

I take it there is no PASM interpreter? There are a few floating around that can be added ....

Thanks and Merry Christmas.
--Steve

Dave Hein · 2010-12-24 08:25

Michael, codegen generated the messages shown below. I'll post the modified Sphinx source once I clean it up. I hacked it up a bit to get it to work.

codegen 100225
reading timestamp: 12
stack space: 2300
table space: 6000
work space: 8004
Opening CONIO.TOK
Pass 1
Pass 2
Syntax error (unexpected)
13,25 '(EOL)'

Steve, I'll look at the changes you made for linux, and I'll make sure I keep them in. Yes, there is no PASM interpreter in spinsim. That was intentional, but I may have to add one in at some point.

jazzed · 2010-12-24 09:13

Dave Hein wrote: »

Steve, I'll look at the changes you made for linux

Oops, I broke your readdir fragment - sorry. It's fixed here. Now dir works in linux.

Dave Hein · 2010-12-28 20:53

I did a bit more work on the spin simulator, and I was able to get sphinx to run on it. I found several bugs in the math ops in spinsim and fixed them. The updated files are in the attached zip files. The sphinx directory contains three subdirectories like Michael Park had. sphinx0 contains the *.spin and *.binary files I generated from the Prop Tool. These files implement sphinx command line program and the various portions of the sphinx Spin compiler -- lex, codegen and link. I added a program called loader that is used to load programs. loader is loaded at the end of RAM, and it then loads another program at the beginning of RAM.

sphinx2 contains the same files as sphinx0, but they are name *.spn and *.bin instead. The *.bin files are generated by the sphinx compiler, except for codegen.bin. I seem to run out of memory when linking this file.

sphinx1 contains various applications, such as dir, copy and del. Run "spinsim sphinx.bin", and then run the applications from sphinx by typing dir, copy or del.

Steve, I didn't get around to merging your linux changes. I do that in the next version.

Dave

bsnut · 2010-12-28 23:38

Dave,

I love your idea. But, I have one big question for you. Can you run the simulations without downloading to the Propeller itself? If you can do this, it will be cool. Test your before you download it to the Propeller.

Dave Hein · 2010-12-29 06:18

bsnut wrote: »

Can you run the simulations without downloading to the Propeller itself? If you can this, it will be cool. Test your before you download it to the Propeller.

Yes, the code can be simulated before it is downloaded to a propeller. The simulator is limited to Spin code only -- it does not support PASM code. Also, the basic I/O drivers are different. Instead of using FullDuplexSerial and FSRW the simulator uses conio and fileio. These drivers provide the same basic methods such as rx, tx, popen, pread, etc.

You can use the Parallax Prop Tool, BST or homespun to compile the spin code and generate a binary file. spinsim loads the binary file and executes it.

Dave

bsnut · 2010-12-29 06:24

Dave,

Thanks, I will download the today simulator and give a try.

Dave Hein · 2010-12-29 18:36

I integrated Steve's (jazzed) linux changes into the previous version. I haven't tried it under linux, but I think I added the changes correctly. I did change the line for readdir in spinsim.c that gets the buffer address. Steve, let me know if this works OK. I'll try it on a linux machine at work if I get a chance.

This version should be identical to the previous version for Windows machines.

jazzed · 2010-12-29 18:42

Dave Hein wrote: »

Steve, let me know if this works OK.

Thanks Dave. Works good so far. I'll test with sphinx later.
--Steve

mpark · 2010-12-30 22:25

Dave Hein wrote: »
Michael, codegen generated the messages shown below. I'll post the modified Sphinx source once I clean it up. I hacked it up a bit to get it to work.
codegen 100225
reading timestamp: 12
stack space: 2300
table space: 6000
work space: 8004
Opening CONIO.TOK
Pass 1
Pass 2
Syntax error (unexpected)
13,25 '(EOL)'

I don't see anything in conio that Sphinx shouldn't be able to handle, but I don't have a real Sphinx setup to verify. Is it real Sphinx or simulated Sphinx that's generating the error? I'm so confused.
What did you have to do to get it to work?

Humanoido · 2010-12-31 00:27

Dave, is this under the MIT open source licensing?

Dave Hein · 2010-12-31 06:03

mpark wrote: »

I don't see anything in conio that Sphinx shouldn't be able to handle, but I don't have a real Sphinx setup to verify. Is it real Sphinx or simulated Sphinx that's generating the error? I'm so confused.
What did you have to do to get it to work?

It's simulated sphinx that was generating the error. The problem was due to some bugs in spinsim. I fixed the bugs and I'm able to build everything except codegen.bin. I suspect that my drivers are using more space than yours, and the final codegen.bin image is too big to fit into memory.

For the most part the main components of lex, codegen and link are unchanged. I did have to modify bintree.spn to use the Spin version of one of the methods instead of the PASM version. Most of the changes are in the drivers -- fdserial, isxfs, isxtv, sxfile, sxfs, sxkb and sxtv. I replaced them with versions that use conio and fileio.

I implemented the Execute method by loading a program called loader.bin in high memory, which then loads the target program in low memory. It was a little tricky because I had to save the name of the last file that was opened, and then pass the file name to loader.bin.

Dave

Dave Hein · 2010-12-31 06:12

Humanoido wrote: »

Dave, is this under the MIT open source licensing?

Everything is open source under the MIT open source license. I'll add the notice in the next update.

mpark · 2010-12-31 10:09

Dave Hein wrote: »

It's simulated sphinx that was generating the error.

That's a relief. Thanks, Dave. You've had a pretty good year, Propeller-wise, haven't you? Spinix, Spin+LMM, SpinSim—what else? I'm sure I'm forgetting something.

Dave Hein · 2010-12-31 14:12

Michael, I have one more project to post before the end of the year.

Check out my next post.

BTW, I'm very impressed by Sphinx. The compiler is much more complete than I first thought. It was an amazing feat to shoe-horn it into the prop's 32K of RAM.

rosco_pc · 2011-01-04 18:56

Just had some time to play around with this, only to find out that the simulator is not 64bit safe (I'm running a 64bit Archlinux at home) as it "segfaulted".

The bits/types.h defines the following:
char - 8 bits
short - 16 bits
int - 32 bits
long - 64 bits

So changing a few long to int took care if the issue. Here is a diff that does just that:spinsim-64bit.txt

Dave Hein · 2011-01-04 20:19

Thanks for the info on 64 bit implementations. Have you tried file I/O? There may be a problem there also because the file pointer is passed to the spin code as a 32-bit value. gcc has a -m32 option that forces the 32-bit mode. I tried this on a linux machine at work, but it didn't work because a 32-bit include file was missing.

jazzed · 2011-01-04 20:19

I found using stdint.h solves the size problem best for linux. Don't know how many other
compiler packages have the header, but it's easy to include simple typedefs in the source.

To use stdint.h all 32 bit int and long would become int32_t (uint32_t for unsigned).
Can you guess what unsigned short would be?

rosco_pc · 2011-01-04 23:17

Would including "sys/types.h" not be easier then?

And sofar I only tried the some simple programs. Will try the file I/O next and see what happens

bsnut · 2011-01-05 01:46

I know that everyone is getting the simulator to work. I normally don't work with the command prompt and I am having a little brain fart:blank:. I am wondering if someone point me in right why with the command prompt commands and know its like ridding a bike,but I seem to have fallen off.

rosco_pc · 2011-01-05 03:18

Dave Hein wrote: »

Thanks for the info on 64 bit implementations. Have you tried file I/O? There may be a problem there also because the file pointer is passed to the spin code as a 32-bit value. gcc has a -m32 option that forces the 32-bit mode. I tried this on a linux machine at work, but it didn't work because a 32-bit include file was missing.

File I/O works for me.

Spin Simulator

Comments