JVM for prop
Peter Verkaik
Posts: 3,956
Having done the smallc and examined what would be required to add longs and floats to smallc,
I decided that it is not worth the effort. Instead I tried some things with the Javelin IDE program.
It turns out, the compiler used to compile javelin code does support longs, floats and doubles
and multidimensional arrays. It is the javelin linker that throws errors when using these types.
See picture for some demo code using all types. The linker has been made public some time
ago and is part of the JavelinDirect package and you can find that package here:
http://tech.groups.yahoo.com/group/JavelinCode/files/JavelinDirect/
That linker generates a (up to) 32KB eeprom image. I think I can easily change that
to create a spin file with a DAT area holding the byte codes.·Which means we only need
a JVM (and some native routines) to run java code (java 1.1).
I also attached a spin file with all byte code values defined, and commented out
types currently not supported. The JVM could be done in spin for now, as I did for smallc.
Here are the types and their memory use:
· primitive· storage/value· supported
· boolean···· 2bytes/1bit····· yes···
· byte······· 2bytes/1byte···· yes····· byte[noparse][[/noparse]n] storage is 2 ints + n bytes
· char······· 2bytes/1byte···· yes····· char[noparse][[/noparse]n] storage is 2 ints + n bytes
· short······ 2bytes/2bytes··· yes···
· int········ 2bytes/2bytes··· yes···
· long······· 4bytes/4bytes··· later·
· float······ 4bytes/4bytes··· later·
· double····· 8bytes/8bytes··· later·
······································· other arrays: 2 ints + n*storage bytes
The types currently not supported generate an error, and I just need to change the linker to generate
the appropiate byte codes. For now I would like to get a working JVM for the supported types.
I will start working on the linker. If anyone wishes to participate by setting up a JVM in spin,
you're welcome. Using java, we will have true object passing, classes, unlimited include path's
so we can use multiple·folders to maintain applications.
regards peter
I decided that it is not worth the effort. Instead I tried some things with the Javelin IDE program.
It turns out, the compiler used to compile javelin code does support longs, floats and doubles
and multidimensional arrays. It is the javelin linker that throws errors when using these types.
See picture for some demo code using all types. The linker has been made public some time
ago and is part of the JavelinDirect package and you can find that package here:
http://tech.groups.yahoo.com/group/JavelinCode/files/JavelinDirect/
That linker generates a (up to) 32KB eeprom image. I think I can easily change that
to create a spin file with a DAT area holding the byte codes.·Which means we only need
a JVM (and some native routines) to run java code (java 1.1).
I also attached a spin file with all byte code values defined, and commented out
types currently not supported. The JVM could be done in spin for now, as I did for smallc.
Here are the types and their memory use:
· primitive· storage/value· supported
· boolean···· 2bytes/1bit····· yes···
· byte······· 2bytes/1byte···· yes····· byte[noparse][[/noparse]n] storage is 2 ints + n bytes
· char······· 2bytes/1byte···· yes····· char[noparse][[/noparse]n] storage is 2 ints + n bytes
· short······ 2bytes/2bytes··· yes···
· int········ 2bytes/2bytes··· yes···
· long······· 4bytes/4bytes··· later·
· float······ 4bytes/4bytes··· later·
· double····· 8bytes/8bytes··· later·
······································· other arrays: 2 ints + n*storage bytes
The types currently not supported generate an error, and I just need to change the linker to generate
the appropiate byte codes. For now I would like to get a working JVM for the supported types.
I will start working on the linker. If anyone wishes to participate by setting up a JVM in spin,
you're welcome. Using java, we will have true object passing, classes, unlimited include path's
so we can use multiple·folders to maintain applications.
regards peter
648 x 532 - 54K
Comments
I haven't done any Java in years but did recently return to look at the JVM specification. It certainly looks possible; it's well defined and there are others who have done similar. Plenty of books available and the source code for the JVM itself, from Sun and third parties.
java.sun.com/docs/books/jvms/second_edition/html/VMSpecTOC.doc.html
regards peter
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Cheers,
Simon
www.norfolkhelicopterclub.co.uk
You'll always have as many take-offs as landings, the trick is to be sure you can take-off again ;-)
BTW: I type as I'm thinking, so please don't take any offense at my writing style
AIUI there are a number of things which make up for what people call "Java" -
A compiler which supports the core Java language and turns source into bytecode classes.
A set of provided classes which are like C-libraries/include files which provide pre-written functionality. Those can be tailored to suit what the propeller supports and provide the interface to the underlying hardware as well as more general purpose routines which are hardware independent.
A linker which turns bytecode classes into an application. This can also perform static linking and so on to minimise what the JVM has to support / do.
A JVM which runs on the target hardware which loads the bytecode, executes it and handles the interactions with hardware.
Developing Java for the Propeller is therefore a case of bringing all that together as a complete whole. This is what makes for that confusing set of Java implementations ( J2SE, J2ME, KVM, CDLC ) for the unfamiliar. It's the linking stage which I don't really have any understanding of and how that affects the JVM.
Looking at the bytecode, it seems the first 170 opcodes or so are nothing special, simply stuff that manipulates the stack, those above are the more complex ones. I'm out of my depth there.
I was enthused enough to look around to see what is available and already done and there are some quite small implementations out there; TinyVM and LeJOS already mentioned, plus NanoVM for the AVR ATmega8 with an 8KB footprint.
www.harbaum.org/till/nanovm/index.shtml
I don't think it's a case of simply implementing a JVM, but also working from the other direction as well, the class files and linking in particular, and I'm not sure it's possible to develop the JVM in isolation.
I'd like to get involved but where I'm struggling is in understanding exactly what must be done, and how we would go about doing it.
I adapted the JavelinDirect sources. It now generates a spin file, which· holds
the binary image in the DAT area, of what would be loaded into the Javelin eeprom.
I also added a switch to the commandline to enable the debug output.
It turns out, the linker creates an image using only 86 bytecodes, called jemcodes.
I attached the java source for which I generated the spin file, the spin file itself,
an updated jvm file with jemcodes and the debug output.
The java class files are staticly linked and are the exact program as it runs on the javelin.
So besides a jvm we need some native routines that are listed at the end of the debug file.
(testJVM.txt). You will notice that at some point the entries in these tables take two words
whereas they start with one word. This is due to the fact that the generated jemcodes
where to run on a SX48, and the flashmemory (wordsize=12bits) of the sx48 is paged. For the prop we
can make all entries 1 word.
regards peter
I'm a Java lover and have·written some applications professionally.·The Javelin Java variant left me wanting to meet the garbage collector [noparse]:)[/noparse] Having a modern programming language on Prop·could have benefits.
So, this is an interesting idea, and who knows maybe we can fix the memory management issue. I spent some time porting the JavelinDirectSrc1.0 code MSC++ to play around today. WIN32 is a horrible pre-processor symbol don't you think [noparse]:)[/noparse] I managed to get a testJVM debug·listing similar to your's on jem.out, but didn't see the 'address' method in your file .... There were a ton of other differences having to do mostly with the dec->hex printing. You have a .cvs repository somewhere ?
Seems that there is enough power here to do things in a decent way. Doing a JVM in spin might be a little slow but functional. Work done in the spin JVM can be leveraged in providing easily ported "pseudo-code" for more serious work when a good C-compiler is available.
Do you think the JVM might be able to step in when a Task class is used·and loading up and run on a·cog·? (not sure·I remember "Task" correctly from my brief Javelin experiments). Not sure what is possible in the JVM, but there appear to be hooks for the "native" code for which a subset, I imagine, can be implemented in the JVM. Looks like there is a good deal of work left to do in the JVM. This does seem like it can be a bit of a chicken/egg problem though considering the size of jem.out. Seems that one may need a debugger.
I'm no java expert, but I can make contributions here.·This forum of provides a fair medium for project workload distribution and status. Peter may be finished coding before I post this message [noparse]:)[/noparse] The project seems to be big enough to stress even his prolific producivity though.
Looking forward to the next installment ....
Given the way the JavelinDirect works ( and looking at the .txt ) I don't see any major problems. I'm presuming the native methods are those from the stamp.* class files and they'd simply be replaced by whatever native methods propeller.* class files use ?
I've got the JavelinStamp software ( including Jikes ) and the JavelinDirect source code now so feeling quite confident that you're on the right track.
Is there a pre-compiled JavelinDirect.exe available - doesn't have to be the latest with floats and longs added ?
I think garbage collection can be added without
sacrificing realtime behaviour by using a seperate COG
to do the garbage collection. The same applies to
adding threads.
For now, I am just trying to get a working jvm that
lets me execute javelin code.
The Javelin IDE has a single step debugger that
could be converted for the prop I suppose. As I
maintain the Javelin IDE right now, I could even
add an option to let the Javelin IDE beta generate
the spin file, as the linker integrated into the IDE
is the same as JavelinDirect (as that·was extracted
from the IDE).
Regarding the native methods, at the moment that
is a fixed table as the javelin does not support user
defined native methods. For the prop that could be
changed by letting the linker add entries to the table
for any userdefined native method.
@Hippy,
I added the native methods to the jvm.spin file.
All Virtual Peripherals (background tasks, not threads!)
have a java wrapper class that calls·native methods.
These VP's··have a 16byte register area that is setup
by using registerRead and registerWrite native methods.
Fortunately, all those registers are defined in the
wrapper classes so I can extract the memory map from
those classes.
I attached an executable PropDirect that is an adapted
JavelinDirect.
To use it:
propdirect --path "c:\javelin\lib;c:\javelin" --port 1 --link --debug testJVM
It then generates a jemfile, a spinfile, and debug files
jem.bin, jem.hex and jem.out (this is testJVM.txt)
Note that path can be different, depending on where
the Javelin IDE was installed.
regards peter
and have put in some code. I would like some prop expert to look
at that code, especially for the pin manipulations (input/output/low/high).
A few native methods have no code yet, but these are much like
some BS2 functions I guess. Perhaps we can use that BS2 code.
regards peter
Needed the RTL60.BPL/VCL60.BPL installing but I've now got testJVM.java compiling and linked to produce what's expected.
I've been thinking about how best to handle coding the JVM and my preference would be to write it in PASM and hand off bytecode opcodes not handled in PASM to a Spin routine, let Spin do its stuff and pass control back to PASM. That allows development in Spin with all its debugging abilities and eases porting the Spin into PASM as it progresses. I've got the framework done but there are still a few details to iron out.
One thing I found; we don't really need testVM.spin, just use
and the bytecode is automatically there for the reading ( byte[noparse][[/noparse] @JvmBytecodeStart+N ] ).
Time to go and re-learn Java and get to understand the JVM specification ...
Edit: I just checked it and it does. Good idea.
regards peter
Post Edited (Peter Verkaik) : 1/23/2008 11:02:00 AM GMT
The Javelin IDE beta always generates a jemfile when you do·Project->Link
for the application program (the one with static void main in it).
Combine that with the FILE keyword, and you don't need the commandline
tool PropDirect.
I attached the IDE beta for you.
Unzip it in your Javelin IDE installation folder and run it (javelin.exe) instead
of "Javelin Stamp IDE.exe".
After first startup, do Project->Global Options and click ok.
This ensures the registry settings are updated.
regard peter
I couldn't make head nor tail of TSXcomm in the JavelinDirect sources but presume the Javalin download protocol is documented / known / can be reverse engineered, in which case ( as when I emulated the BS1 / PICAXE ) when the JVM is done, it can be one-off programmed into a Propeller and then the Javelin IDE itself can be used to develop, download and debug Java programs.
With a JVM.eeprom and a third party Propeller downloader ( or built into the Javelin IDE ) a Java programmer wouldn't need to know anything about Spin or the Prop Tool to use the propeller.
from the TSXComm file that you mentioned. For the prop that is entirely irrelevant.
What I can do, because I maintain the IDE beta, is add an option to the Javelin IDE beta,
to select wether to download to a prop or javelin. For that I need a comprehensive
description of the prop download protocol and a way to generate the prop file that
must be downloaded. At the moment only the jemfile is generated.
Are there commandline tools (required/available) to automate that task?
It would be nice if we could program the prop directly from the IDE, especially
since the IDE has a single-step debug facility builtin. That is actually targeted
towards the sx chip SX-key debug capability, but I know what commands are sent,
and maybe we can use that for the prop also.
regards peter
Can you please post the RTL60.BPL/VCL60.BPL files or links ?
Thanks for the garbage collector carrot [noparse]:)[/noparse] How many cogs do you think is necessary for the JVM? I have a project that uses all cogs including 1 for main spin, 1 for debug, 2 for floating point libraries, and 4 for hardware driver objects. We are resource constrained. Maybe a little cog planning is in order.
Are you able to use javadoc to create an "API manual" from the .spin file ?
Presuming the memory requirements for the JVM + native methods is about
equal to the memory in the SX48 chip, which is 4KB, I expect 2 cogs are
needed for the jvm. But I am no PASM expert so perhaps it can be done
in 1. I do think alot of the Virtual Peripherals could share a COG. ADC, DAC PWM
and Timer are all very small routines. The Uarts are probably more difficult to
share a COG but then the native Uart routines are smaller than say FullDuplexSerial
because a lot of the Uart code is encoded in bytecodes.
The javadocs for the java classes available can be viewed in the IDE help.
regards peter
One advantage of a roll-your-own JVM is that there's complete control over everything, so there's no reason not to hold a 32KB bytecode image in I2C Eeprom; the ProtoBoard is handy for that. Execution would be slower but program size can be larger. The JVM could dynamically choose the best option depending on size of bytecode.
One thing I'm very pleased about; the Beta Javelin IDE and all the tools run under 98SE.
its entire 32KB eeprom space anyway. The javelin only writes the required bytes,
the remaining eeprom is untouched and can be used for persistent application data.
Having a 'fixed' jvm would also allow for that on the prop (read spin stamp).
But to allow userdefined native methods, like·new Virtual Peripherals, it must also
be possible to download a complete new image. VP's usually require some assembly
and that must be programmed with the prop tool.
regards peter
If one can use the JIDE debugger with Prop's JVM, it can be a big win.
Getting a VM to work on Prop is a prerequisite to any of it though [noparse]:)[/noparse]
Exposing Prop capabilities and other enhancements would be useful later, and
may be done in java classes with a little help from the VM. One could have a
CogThread for the user programmer to get access to free cogs. Having access
to vga or tv devices is a Prop "staple" and at some point should be available.
@Peter you wrote:
"The javadocs for the java classes available can be viewed in the IDE help."
Actually I was asking about your use of javadoc/doxygen markup in .spin files
not .java files. If one surrounds the comment in double braces and adds a
function signature, the comments will show up in the prop "Document" view.
It seems that having collaborative written specifications would be useful
and can be made easy using a tool like www.writeboard.com . This may
seem less important now in the prototyping phase, but defining features,
basic decisions, and documenting results of experiments early makes it
easier to produce good engineering output. Teams especially benefit.
-Phil
How do we get the length from the "jem.bin" file ? There must be a simple "small" solution.
According to the .out file the second field is "a" length, but a byte is of course too small to
hold the file length. Some alternatives: use your original embedding method with a length
constant or use jem.hex which results in a 50% bigger byte code file and needs to be parsed.
Aw heck. never mind. This works:
DAT
· jvmBytecodeStart FILE "jem.bin"
· jvmBytecodeEnd
PRI getJemLen : len
· len := @jvmBytecodeEnd-@jvmBytecodeStart
Post Edited (jazzed) : 1/23/2008 11:44:23 PM GMT
· .jem File Format: (For version 0.1)
·+
+
·| Flag········· | Run/Debug flag.
·+
+
·+
+
·| Length······· | Length of the file in 256 byte multiples.
·+
+
·+
+
·| Version······ | Version stored as BCD major.minor version #.
·+
+
·+
+
·| NumStatic···· | Space required for static variables (measured in bytes).
·+
+
·| MainOffsetMSB | Offset to the code for the `main' method.
·| MainOffsetLSB |
·+
+
·|ObjectOffsetMSB| Offset to the java.lang.Object class for instantiating arrays.
·|ObjectOffsetLSB|
·+
+
·|StringOffsetMSB| Offset to the java.lang.String class for instantiating strings.
·|StringOffsetLSB|
·+
+
·| ThrowOffsetMSB| Offset to the java.lang.Throwable.throwVMException() method
·| ThrowOffsetLSB| for throwing exceptions.
·+
+
·| STOffsetMSB·· | Offset to the string table.
·| STOffsetLSB·· |
·+
+
·| CBOffsetMSB·· | Offset to the callback method.
·| CBOffsetLSB·· |
·+
+
·| OOMOffsetMSB· | Offset to the java.lang.OutOfMemoryError class for JVM errors.
·| OOMOffsetLSB· |
·+
+
·| StackBaseMSB· | Base of the stack in memory.
·| StackBaseLSB· |
·+
+
·| ArrayClassMSB | Offset to the dummy array class.
·| ArrayClassLSB |
·+
+
·+
+ First Class:
·| SuperOffsetMSB| Offset of the superclass from the start of the file.
·| SuperOffsetLSB|
·+
+
·| NumFields···· | Number of fields in objects of the class.
·+-+
+
·| NumParams···· | Number of method parameters
·+-+
+
·| Meth1ETabMSB· | Offset to the exception table for this method.····
·| Meth1ETabLSB· | Offsets measured from start of file.
·+
+
·| NumLocals···· | Number of local variables (excluding parameters)
·+
+
·| Flags········ | Native method flag.
·+
+
·| Meth1OffsetMSB| Offset for the first method of this class (and superclass)
·| Meth1OffsetLSB| Offsets measured from start of file.
·+
+
·|E|NumParams··· | Number of method parameters
·+
+
·| Meth1ETabMSB· | Offset to the exception table for this method.
·| Meth1ETabLSB· | Offsets measured from start of file.
·+
+
·| NumLocals···· | Number of local variables (excluding parameters)
·+
+
·| Flags········ | Native method flag.
·+
+
·| Meth2OffsetMSB| Second method
·| Meth2OffsetLSB|
·+
+
·| ...·········· |
·+
+
·+
+ Second Class:
·| SuperOffsetMSB| Offset of the superclass from the start of the file.
·| SuperOffsetLSB|
·+
+
·| NumFields···· |
·+
+
·|E|NumParams··· |
·+
+
·| NumLocals···· |
·+
+
·| Method1 offset|
·| ...·········· |
·| code ...····· |
·+
+ Method exception table.
·| NumHandlers·· |
·+
+
·| fromMSB······ | PC to catch from.
·| fromLSB······ |
·+
+
·| toMSB········ | PC to catch to.
·| toLSB········ |
·+
+
·| handlerMSB··· | Start of the catch handler.
·| handlerLSB··· |
·+
+
·| TypeMSB······ | Offset of the class to catch.
·| TypeLSB······ |
·+
+
·|...··········· |
·+
+ String offsets table.
·|Offset0MSB···· | Offset to string 0, measured from the start of the file.
·|Offset0LSB···· |
·+
+
·|Offset1MSB···· | Offset to string 1, measured from the start of the file.
·|Offset1LSB···· |
·+
+
·|...··········· |
·+
+ String data.
·|...··········· |
·If the high bit of the high byte of the method offset is set then the method is native. The low byte
·is then an offset into the native method dispatch table. Thus there can be only 256 native methods.
In the code itself we find
· binary[noparse][[/noparse]kLengthOffset] = (binaryLength/256)+1;
· if ( binary[noparse][[/noparse]kLengthOffset] < 20 )
··· binary[noparse][[/noparse]kLengthOffset] = 20;
This means
binaryLength = 256*(binary[noparse][[/noparse]kLengthOffset]-1) where binary[noparse][[/noparse]kLengthOffset] >= 20
According to the above description you need to use
binaryLength = 256*binary[noparse][[/noparse]kLengthOffset]
so there is some waste at the end but it ensures the heap starts at a 256byte boundary
in respect to the start address which would be 0 for the javelin but for the prop it is not 0.
The jvm must account for this offset.
regards peter
I now included code for all native methods except 8bit sigmadelta ADC and 8bit DAC.
If anyone has some code for this please provide a link.
regards peter
Made some small additions to your jvm.spin file ... added lines surrounded by {<jazzed>} tags, and
added an init for the pc in the run method. Created a small JvmMain.spin mainly as an entry point
and a way to specify the clock setup that you can use or toss away.
You can toss JvmMonDebug.spin too if you like [noparse]:)[/noparse] I intend to add some methods that will make
debugging the JVM sections easier without requiring interference on the serial port.
I was thinking next to start into the jem_* case code. I hope I have more answers than questions [noparse]:)[/noparse]
The heap.spin object will be useful here ... alas I doubt I have the lastest code to support heap.
Is that right ? In the .out I'm getting it seems to be the msb of 'flag' set, high byte is $00, but the entire offset appears available for use.
I've attached my first attempt at doing the JVM in PASM, doesn't run yet, I haven't folded in the native methods nor calls down to the Spin code.
I've also created a page on the Wiki. That can be split into programming / development of JVM as we progress.
propeller.wikispaces.com/Programming+in+Java
Just one suggestion: Please use "jem.bin" as the jem file so any class can be used
without modifying Prop code. This output of PropDirect is exactly the same as testJVM.jem.
I can see where one may like to use a more specific file name for JVM development to
quickly change test files, but it is likely in the end the generic reference will be necessary.
Since VGA_Text and TV_Text have mostly the same API, I was able to change just a few lines
to use my VGA display with the PropProtoboard ... OBJ tv:"VGA_Text" and CON TV_PIN = 16
Maybe that preprocessor would be useful....
BTW, I'm finding the nanovm source code to be a valuable "guide".
Thanks for the NanoVM pointer. Sun's JVM Spec is probably brilliant once you've "been there, done that, started an 'I write JVMs' T-shirt printing business", but there's got to be a better way to check what "DUP2_X2" actually does with the stack etc. Nothing better than being led by example I find.
There's also TaurusVM ( solved my DUP queries in a jiffy ) and SimpleRTJ.
You are right, it is bit7 of Flags that is set for a native method. The text as supplied was
directly copied from the sources (and these are thus in error).
@Jazzed,
I have also been looking at Tinyvm (no garbage collection but threads are implemented
for a single processor). I shall look into nanovm also. Perhaps we should select one
as our guide that·we can reference.
The propdirect.exe outputs fixed names for debug output: jem.bin, jem.hex and jem.out
I could change that to be <filename>.bin, <filename>.hex and <filename>.out
if that is more convenient.
regards peter
Whatever the "*.bin" file name is would be included in the VM code I suppose.
I'm content with "jem.bin" and it seems like extra work to go beyond that.
Doubt we have to choose one VM as a guide, the number of algorithmic solutions is not finite,
although the context does dictates requirements.
@Hippy,
I'm not sure how to integrate all this, but my since you have chosen TV for debugging PASM,
I think I can continue using VGA for .spin debugging code unless there is some conflict using
both simultaneously.
Is it possible to glue the two modules together without requiring constant diff updates?
Peter has done a great deal of work in native functions ... someone show the way.
In the jvm.spin, I had put "pc" initialization in run(), in retrospect that appears to
be a bad idea. I've moved it to init(). Also calling run(length) is probably a debug thing.
I suppose a VM should handle as many codes as it wants to run [noparse]:)[/noparse]
The JvmMonDebug VGA output is shaping up like below so far will post more later ...
·
·
· VmMonDebug· pc $0000·· sp $0000
· heap $0000 use $0000 free $0000
· Frame$0000
· OpCode <opcode name>
· <operand dump>
·
· -- debug section --
· -- error section --
·