JVM for prop

Peter Verkaik · 2008-01-28 15:38

@Hippy,
I found this info in the sources.

/* Stack frame format:

· +

+
· | Local0······· | <- JVM_FP
· | ...·········· |
· | LocalN······· |
· +

+
· | Method MSB··· | <- JVM_FP + numLocals
· | Method LSB··· |
· +

+
· | Old FP MSB··· |
· | Old FP LSB··· |
· +

+
· | JVM_PC MSB··· |
· | JVM_PC LSB··· |
· +

+
· | Stack0······· |
· | ....········· |
· +

+
···················· <- JVM_SP

*/

const int kMPOffset = 0;
const int kFPOffset = 2;
const int kPCOffset = 4;

#define STACK_WORD(x) (stack->Items(x+1)|(stack->Items(x)<<8))
#define UNWIND_MP STACK_WORD(fp + kMPOffset + numLocals)
#define UNWIND_FP (STACK_WORD(fp + kFPOffset + numLocals)-stackBase)
#define UNWIND_PC STACK_WORD(fp + kPCOffset + numLocals)

There are a number of commands sent to the Javelin that aid in debugging (file TSXComm.cpp):
··// Ask the SX its status.
void __fastcall TSXComm::QueryStatus(int *pc, int *activity, int *bytecode)
· ·// Ask the SX the size of its stack/heap and its program memory.
void __fastcall TSXComm::QueryConfig(int *memSize, int *progSize, int *heapBase)
· ·// Retrieve a portion of the JVM heap.
void __fastcall TSXComm::QueryHeap( int rangeStart, int rangeLength, unsigned char *data, int *hp )
··· // Download a list of breakpoints to the SX.
void __fastcall TSXComm:[noparse]:D[/noparse]ownloadBreakpoints( int *bpList, int count, int stepBP )
···// Ask the SX to single step.
void __fastcall TSXComm::Step()
···// Ask the SX to run.
void __fastcall TSXComm::Run()
···// Ask the SX to stop.
void __fastcall TSXComm::Stop()

These commands are handled identical to the download command (bytestuffed packets)
so I will add these to the command handler state machine. Since there is no command
to retrieve the values for static variables, we must assume the static variables are the
first to be allocated from the heap. Since there is no garbage collection, we only need
to remember the address from which to allocate next (single word pointer).
Also note that arrays are allocated from the heap, but references are on the stack if
locally declared.

regards peter

Peter Verkaik · 2008-01-28 20:56

The file memory.java has a method freeMemory():

· /**
·· * Find the number of bytes of memory SRAM.
·· *
·· * @return the number of bytes of free SRAM.
·· */
· public static int freeMemory() {
··· int stackPointer = (CPU.readRegister(JVM_STACK_POINTER+1)<<8)|CPU.readRegister(JVM_STACK_POINTER);
··· int heapPointer = (CPU.readRegister(JVM_HEAP_POINTER+1)<<8)|CPU.readRegister(JVM_HEAP_POINTER);

··· return heapPointer - stackPointer;
· }

The result is positive, and combined with the stack frame from the previous post,
that yields following memory map:

······ Memory map layout
$0000· +

+
······ | jemcodes····· |
······ | ...·········· |
$XXX0· +

+ stackbase
······ | Local0······· | <- JVM_FP
······ | ...·········· |
······ | LocalN······· |
······ +

+
······ | Method MSB··· | <- JVM_FP + numLocals
······ | Method LSB··· |
······ +

+
······ | Old FP MSB··· |
······ | Old FP LSB··· |
······ +

+········ stack frame
······ | JVM_PC MSB··· |
······ | JVM_PC LSB··· |
······ +

+
······ | Stack0······· |
······ | ....········· |
······ +

+
······ |·············· | <- JVM_SP
······ |···· free····· |
······ |·············· |

······ |·············· |
······ |···· free····· |
······ |·············· |
······ +

+
······ |·············· | <- JVM_HP
······ |·· allocated·· |
······ |·············· |
······ +

+
$7FFF··················· heapbase

Obviously, for the prop heapbase is the last byte from the reserved DAT area for java code
and it will be less than $7FFF. $0000 corresponds with the first byte of the reserved DAT area.

regards peter

jazzed · 2008-01-29 00:09

Hi. I've made some progress with Jem_NEW, Jem_GETFIELD, Jem_PUTFIELD, and Jem_LDC ...
Hippy started Jem_LDC code. I've added some get heap, copy constant, and push to stack code;
some analysis of what else needs to be done for LDC ... I've added string handling for now to get
past failing LDC in case such as below. I'll post spin code for review later today.
String s = new String("Hello World");
System.out.println(s.toString());"

public class mytest {
  static void main() {
    String s = new String("Hello Java");
    System.out.println(s.toString());
    while (true) {
    }
  }
}

Post Edited (jazzed) : 1/29/2008 1:59:50 AM GMT

Peter Verkaik · 2008-01-29 17:50

I am close to get the Javelin IDE working with the prop. The IDE
recognizes a prop as javelin (correct version and echo, only GotStatus is still
not working because it not quite clear what the IDE expects).
I have put all known addresses and register names in my test file.
I defined 2 areas: javaProg that holds the jembytes,statics,stack and heap.
jvmRegs that holds all the jvm and native method registers.
This reflects how the SX jvm is used.
(currently jvmRegs is 256 bytes but I think that can be reduced).

I simulate a reset by setting the variable reset from a cog that is only used
to check the ATN pin (spin stamp pin 3).
I would like the serial driver to check that pin (saves a cog) and set the reset variable,
but I have no idea how to do that, so if anyone can figure that out, please.

The serial driver is an adapted fullduplexserial that has the native methods
message() and getByte() implemented. This makes it easy to send debug messages
directly to the Javelin IDE message window. Having a spin stamp really benefits here,
as the SOUT and SIN pins do not interfere with the prop TX/RX pins for downloading
new JVM firmware.

regards peter

jazzed · 2008-01-29 19:12

@Peter, Good progress on downloader.

@Hippy, I'm down in ARRAY land now.
In at least one case "newarray char", the operand for NEWARRAY is the "type".
The length comes off the stack. I'm finding the java bytecode wiki instructive.
http://en.wikipedia.org/wiki/Java_bytecode

I'll post code after I've answered more questions that are bothering me.
BTW: I've gotten quite used to your latest design and am fairly impressed.
Please don't re-architect unless absolutely necessary [noparse]:)[/noparse]
I'm putting·what I can discern into the spec i've started.
·

jazzed · 2008-01-31 21:43

Hi guys. My frustration meter has been pegged for about 1.5 days now.
Time to pass the baton to the next runner. I don't really want to hold my
breath waiting for the widely anticipated C compiler, but that is attractive
right now. I haven't spent this much time looking at·a stack in 20 years.

My frustration usually means I've misinterpreted something. I've interpreted
the JVM code so far as object reference and parameter passing on stack and
"constant table" entries as pointers on the object reference. So a "block"
diagram looks something like this:

vertical columns represent stack-wise manipulation

horizontal rows are object reference/fields storage

main ->

class init -> "constant table"
           -> "function table"

           -> "class field array"

           ... instructions with stack manipulation ...

           ... ldc/getfield/putfield with class fields ...

           ... etc ...

           -> class init -> "constant table"
                         -> "function table"

                         ....

           ... other instructions/method calls ...

... other main instructions, etc....

Thing is i've had this mostly working except for one or two stack accounting
problems. The InvokeStatic and InvokeVirtual methods are tough and tougher[noparse]:)[/noparse]
InvokeStatic is a method call as far as I can tell, but the stack gets off by a
16bit word pointer near the beginning. As far as InvokeVirtual goes, well either
we dynamically suck in the function table (not desirable for memory reasons)
or try to keep track of it on an "as needed basis" (worse performance).

At this point, I'll leave a little thought to you with a couple of pointers to
what I believe I've resolved in line below. Maybe you will notice something
completely off base here. Beats me.

PRI doNEW | objectref, data, classAddr, superClass, numFields, type, len
{{
/**
 * see case jem#Jem_NEW comment for basic operation description.
 */
}}     
  if jazzDebug
    tv.Str( String($d,"New " ) )
    tv.Hex( pc-bytecodeStart, 4 )
     
  classAddr := Operand_BytecodeAddress
  
  superClass := byte[noparse][[/noparse]classAddr+0] << 8
  superClass |= byte[noparse][[/noparse]classAddr+1]
  numFields  := byte[noparse][[/noparse]classAddr+2] '>> jemFileCountAdjust
   
  ' allocate heap pointer for class
  ' get enough storage for superClass and numFields
  ' fields are stored in the stack ???
  '
  objectref := GetFromHeap(jem#T_ANY, numFields<<1+1) 
  GetHeapInfo(objectref, @type, @len, @data)
 
  'save this class name
  classAddr -= bytecodeStart

  word[noparse][[/noparse]data] := classAddr
  'push "this"
  Push (objectref)
   
  if jazzDebug
    tv.Str(string($d,"pc "))
    tv.Hex(pc-bytecodeStart, 4 )
    tv.Str(string($d,"this "))
    tv.Hex(objectref, 4 )
    tv.Str(string(" class addr "))
    tv.Hex(classAddr, 4 )
    tv.Str(string(" SC "))
    tv.Hex(superClass, 4 )
    tv.Str(string(" F$"))
    tv.Hex(numFields, 2 )
    tv.Str(string(" t$"))
    tv.Hex(type, 2 )
    tv.Str(string(" l$"))
    tv.Hex(len, 2 )
 
  result := 0
 
 
PRI LoadConstant(constantSizeInBytes, tableIndex) | constAdr, constLen, constOff, ptr, type, len, data
  constAdr := byte[noparse][[/noparse]ct + tableIndex << 1 + 0] << 8
  constAdr |= byte[noparse][[/noparse]ct + tableIndex << 1 + 1] 
  constAdr += bytecodeStart 
  constLen := byte[noparse][[/noparse]constAdr + 8] << 8
  constLen |= byte[noparse][[/noparse]constAdr + 9]
  constOff := constAdr + 10
  
  ' string table holds all constants
  ptr := GetFromHeap(jem#T_ANY, constLen+1) 
  GetHeapInfo(ptr, @type, @len, @data)
  word[noparse][[/noparse]data] := constOff
  PushAddr(ptr)
  if jazzDebug
    tv.Str(string($d,"LDC "))
    tv.Dec(constantSizeInBytes)
    tv.Str(string(" "))
    tv.Hex(tableIndex, 4 )
    tv.Str(string(" co "))
    tv.Hex(constOff-bytecodeStart, 4 )
    tv.Out("=")
    tv.Hex(constOff, 4 )
    tv.Out(" ")  
    result := 0


PRI doGETFIELD | objectref, data, blockAddr, index, type, len, val, xlen
{{
/**
 * case jem#Jem_GETFIELD Get 16bit field from object.
 * Pop object/Push value
 */
}}
  index := Operand_UnsignedWord
  objectref := PopAddr
  GetHeapInfo(objectref, @type, @len, @data)
  if index =< len
    PushAddr(word[noparse][[/noparse]data+(index>>1)])
  else
    if jazzDebug
      tv.out($D)
      tv.out("?")
      tv.dec(index)
      tv.out("<")
      tv.dec(len)
    Crashed(string("GETFIELD index out of bounds"))
    
  result := 0
 
    
PRI doPUTFIELD | objectref, type, len, data, value, index
{{
/**
 * case jem#Jem_PUTFIELD Set 16bit field in object.
 * Pop value/Pop object
 */
}}
  index := Operand_UnSignedWord
  
  value := Pop
  objectref := PopAddr
  GetHeapInfo(objectref, @type, @len, @data)
  if index =< len
    word[noparse][[/noparse]data+(index>>1)] := value
  else
    if jazzDebug
      tv.out($D)
      tv.out("?")
      tv.dec(index)
      tv.out("<")
      tv.dec(len)
    Crashed(string($D,"PUTFIELD index out of bounds"))
        
  result := 0
  
PRI doNEWARRAY | arrayLen, arrayType
{{
/**
 * case jem#Jem_NEWARRAY Create new array of operand type
' .---------------------------------------------------------------------------------------------.
' |     NEWARRAY                                                                            $46 |
' |     Meaning: Allocate new array.                                                            |
' |     Description: A new array of a specific array type, capable of holding size              |
' |     elements, is allocated. The result is a reference to the new object. Allocation         |
' |     of an array large enough to contain size items of the specific array type is            |
' |     attempted and all elements of the array are initialized to 0.                           |
' |     size represents the number of elements in the new array and must be an                  |
' |     integer. The result is stored with an internal code that indicates the type of          |
' |     array to allocate. Possible values for the type of array are as follows:                |
' |     T_BOOLEAN(4), T_chAR(5), T_FLOAT(6), T_DOUBLE(7), T_BYTE(8), T_SHORT(9),                |
' |     T_INT(10), and T_LONG(11).                                                              |
' |     Note                                                                                    |
' |     A NegativeArraySizeException is thrown if size is less than 0. An                       |
' |     OutOfMemoryError is thrown if there is not enough memory to allocate the array.         |
' `---------------------------------------------------------------------------------------------'
  */
}}
  arrayLen := Pop
  'Operand_UnsignedByte ' ***** HACK **** This is $00 so what's it mean ?
  arrayType := Operand_UnsignedByte  ' $00 means *ANY* arrayType
   
  case arrayType
    0:  arrayType := jem#T_INT
    1:  arrayType := jem#T_BYTE
    other: arrayType := jem#T_BYTE
    
  Push( NewArrayRef( arrayType, arrayLen+1 ) )
  result := 0

   
PRI doANEWARRAY | arrayLen, index
{{
/**
 * case jem#Jem_NEWARRAY Create new array of operand type
' .---------------------------------------------------------------------------------------------.
' |     ANEWARRAY                                                                           $47 |
' |     Meaning: Allocate new array of objects.                                                 |
' |     Description: A new array of the indicated class type and capable of holding             |
' |     size elements is allocated. The result is a reference to the new object.                |
' |     Allocation of an array large enough to contain size elements of the given class         |
' |     type is attempted and all elements of the array are initialized to null.                |
' |     size represents the number of elements in the new array and must be an                  |
' |     integer. byte1 and byte2 are used to construct an index into the constant pool          |
' |     of the current class. When the item at that index is resolved, the resulting            |
' |     entry must be a class.                                                                  |
' |     The anewarray instruction is used to create a single-dimension array.                   |
' |     Note                                                                                    |
' |     A NegativeArraySizeException is thrown if size is less than 0. An                       |
' |     OutOfMemoryError is thrown if there is not enough memory to allocate the array.         |
' `---------------------------------------------------------------------------------------------'
 */
}}
  arrayLen := Pop
  index := Operand_UnsignedWord '... the class constant pool index (for array storage ?)
   
  Push(NewArrayRef(jem#T_ANY, arrayLen))
  
  result := 0

Peter Verkaik · 2008-01-31 22:22

Jazzed,
The stack frame should look like
/* Stack frame format:

· +

+
· | Local0······· | <- JVM_FP
· | ...·········· |
· | LocalN······· |
· +

+
· | Method MSB··· | <- JVM_FP + numLocals
· | Method LSB··· |
· +

+
· | Old FP MSB··· |
· | Old FP LSB··· |
· +

+
· | JVM_PC MSB··· |
· | JVM_PC LSB··· |
· +

+
· | Stack0······· |
· | ....········· |
· +

+
···················· <- JVM_SP

*/

I use this as jvm reference, which is more understandable than the Sun reference.
http://www.opensitesolutions.com/books/comp_books/1-57521/088-6/
Chapter 24 is a detailed description of the jvm.

regards peter

jazzed · 2008-02-01 00:35

@Peter, that reference is a·slightly more verbose than the original comments. Only the sun bytecode definitions are more verbose, but as you mentioned are difficult to understand. Some things about the jem.out class are mysterious. The only "constant table" that I can find·is a string table ... maybe it was just my test class snippet ? Have you created a class that caused a numeric constant table to appear?· Somethings bother me·such as·class names and jem.bin ... i see java/lang/Object, etc in the jem.out, but there is no such "string name" that I can tell in the bytecode. One question is "how does one pull class info from the bytecode list?" in the jem bytecode file ... there are implied delimiters but can one guarantee some of the details ?

Looking around I found a very detailed yet flexible description of JVM implementation theory. It has more answers than I have questions (the A's don't match the Q's in most cases) .... It goes into great detail about possible object data designs and stack implementation. If only there was such great detail about what is expected of the "short-cut" jem JVM we're stuck with.

http://www.artima.com/insidejvm/ed2/jvm.html

@Hippy, BTW i'm eating 32 bit words now [noparse]:)[/noparse] This 16 bit stuff is just complicating things. You have some "macros" for dealing with this problem, but it is not always clear where such need to be applied. Have you given any thought to referencing class data -vs- instance data in an object ?

Thanks.· I'm taking a break for a few days. C ya.

hippy · 2008-02-01 05:32

jazzed said...
Looking around I found a very detailed yet flexible description of JVM implementation theory.

http://www.artima.com/insidejvm/ed2/jvm.html

@Hippy, BTW i'm eating 32 bit words now [noparse]:)[/noparse] This 16 bit stuff is just complicating things. You have some "macros" for dealing with this problem, but it is not always clear where such need to be applied. Have you given any thought to referencing class data -vs- instance data in an object ?

Thanks. I'm taking a break for a few days. C ya.

Thanks for the link. I'd found that site, but the pages didn't seem to give anything useful so I missed that entirely !

I tried to abstract the 16/32 issue away but it's still hard getting one's head round what should be what and I admit to running in 32-bit mode. Get it working ( 32-bit ) then improve ( eg 16-bit ) is a reasonable approach. The biggest problem I found was trying to design/implement the small bits without understanding the Big Picture (TM). I ran up against the same wall with classes / class and instance data, it's all in a big cloud marked "huh?"

I think that if we knew what the JVM was, we'd be able to understand how the JEM stuff fits in with that, but looking at the JEM code doesn't reveal the JVM. I'm expecting some 10kW light bulb to suddenly come on and The Scheme of Things to be revealed in crystal clarity, but that hasn't happened yet.

You probably guessed that I'm into my break as well. Sorry I haven't given any feedback / encouragement. I'm still here, still intending to do more, but needed to step back to position myself for a new attack - - - and I was hoping you'd crack it and save me the effort

I look at it this way; we're designing a JVM, in two weeks from having taken off our diapers. It ain't going to happen. But it will all fall into place at some point. It's an up-hill struggle and I think you've done remarkably well. Take that well deserved break.

Peter Verkaik · 2008-02-01 10:23

Let me give the memory layout as I think it is used in the sx jvm.
But first the way the datatypes are handled. In the java specs,
an int is 32bits, (in Jazzed's link that is defined as a machine word),
anything smaller than a wordsize gets promoted to wordsize before placing it on the stack.
There are two types, long and double, that have doubleword size.
In the sx jvm, an int has 16bits meaning wordsize is 16bits. The smaller types boolean,
byte, char, short are all promoted to wordsize. Yet unsupported types long, float and double,
will eventually be doubleword size (float is immediately promoted to double by the linker,
as is short promoted to int). So we are still dealing with wordsize and doublewordsize units.

Assuming we set aside a DAT area that represent the memory space available,
the first part are the jembytes. At entry $0012 there is a word that holds the stackbase value.
At entry $0003 there is a byte that holds the bytesize for the static variables (0 identifies 128 variables).
Each static variable has wordsize (2 bytes in our case).
Static variables are accessed by index number (after getstatic etc). We can either place the statics
at the stackbase, or at the far end of the reserved memory area (which is the end of the heap).
I prefer to place them at the stackbase. That means the stack starts at address stackbase+staticsSize.
Frames are allocated from the stack. For each method call, a new·frame is 'pushed', upon return that
frame gets 'popped'.
Objects are allocated from the heap and never garbage collected (for now). This means we can use a simple
heappointer that points to the last allocated block.·Object references are wordsize. Arrays are objects.
If you use
· int[noparse]/noparse myArray = new int[noparse][[/noparse]12];
two things happen: a wordsize variable myArray is created, and a block·with size (2 +·12*2) bytes is
allocated from the heap. The first 2 bytes (=wordsize) of that block holds the length of the remaining
of the block (myArray.length)

So here is the memory layout as I see it.

········ Memory map layout
· $0000· +

+ @javaProg (reserved DAT area for·java program)
········ |··· header···· |
· $0014· +

+
········ | jemcodes····· |
········ | ...·········· |
· $XXX0· +

+ stackbase
········ |·············· |
········ | static vars·· |
········ |·············· |
········ +

+··························· ---
········ | Local0······· | <- JVM_FP·················· ^
········ | ...·········· |···························· |
········ | LocalN······· |···························· |
········ +

+
········ | Method MSB··· | <- JVM_FP + numLocals
········ | Method LSB··· |
········ +

+
········ | Old FP MSB··· |
········ | Old FP LSB··· |
········ +

+······················ stack frame
········ | JVM_PC MSB··· |
········ | JVM_PC LSB··· |
········ +

+···························· |
········ | Stack0······· |·operand stack·············· |
········ | ....········· |···························· V
········ +

+····························---
········ |·············· | <- [url=mailto:JVM_SP@jvmRegs]JVM_SP[/url]
········ |···· free····· |
········ |·············· |
········ |·············· |
········ |···· free····· |
········ |·············· |
········ +

+
········ |·············· | <- [url=mailto:JVM_HP@jvmRegs]JVM_HP[/url]
········ |·· allocated·· |
········ |·············· |
········ +

+
· JAVASIZE················ heapbase

@Jazzed, you are right about the constant pool only storing the strings. Static final {type}·constants are
treated as #defines and·resolved by the linker so these take no storage space. Static initializers
(eg. static int p = 3[noparse];)[/noparse] are done in code (<clinit>), instead of a·static variables area with preset values.

regards peter

Post Edited (Peter Verkaik) : 2/1/2008 11:09:07 AM GMT

Peter Verkaik · 2008-02-02 17:50

Succesfully downloaded javelin program to prop, directly from Javelin IDE program.
The spin sources are for a spin stamp, as that has the hardware echo on its
SOUT and SIN pins. I use its ATN pin to simulate a reset. This means I can just
download the jvmJIDE to ram, then do Project->Program in Javelin IDE.
After programming, the javelin is reset and the message window comes up.
Still need to tweak it for optimum timing.

regards peter

hippy · 2008-02-02 18:55

Excellent stuff Peter. It would have taken me forever to unravel the mysteries of TSXcomm.

We so far have ...

A Javelin IDE used for end-user development
Direct download from the IDE
The ability to run everything using just the PropTool for JVM debugging
A JVM which can do basic flow control, handle primitive types and call native methods
Core native methods implemented
Access to the tools needed to add / change native methods
Comms back from the JVM to the Javelin IDE
Some more advanced JVM code

That's quite an achievement in such a short time.

Is the next thing worth looking at; how the Javelin IDE does interactive debugging ? Would that be the easiest way to see what was working and what wasn't in the JVM ?

Peter Verkaik · 2008-02-02 19:26

Hippy,
Attached is·the latest jvmJIDE. This one does generate a negative reply if a packet
is received corrupted. In the previous version·I always gave a positive·reply. Turned out
to be a misunderstanding of the checksum. The checksum is the negative sum
of all·leading packetbytes, so adding all packetbytes including checksumbyte
must be 0 for an errorfree packet.
I also added some democode how to display values. I use the Format object
to generate asciiz strings which are then simply displayed in the IDE message
window. Next step is to get·input from the IDE message window. That gives
us the power to do interactive debugging directly from the Javelin IDE.

regards peter

Peter Verkaik · 2008-02-04 13:09

Found out there·is an error in the description of the protocol used
to send messages to the Javelin. The description states the messages
are bytestuffed using values $7E and $7F.·However, the values used
for bytestuffing are $7E and $7D, the same as for·packets.
The fact I·did not discover this before, is that it only matters if you
use characters with ascii code $7D, $7E or $7F, which I·never use.

Attached are two pictures of debug screens·for·a real javelin. These
combined with the jem.out must gives us hints on how to initialize
jvm parameters (the names are the names used in the IDE sources):

jvmPC: current program counter
jvmSP: current stack pointer
jvmFP: current frame pointer
jvmMP: current method pointer
jvmHP: current heap pointer

Further properties of interest are:
MemoryTotal: available·memory space = JAVASIZE (reserved DAT area size for java program)
MemoryCode: bytes occupied by code
MemoryStrings: bytes occupied by strings
MemoryStatic: bytes occupied by static variables
MemoryHeap: bytes allocated from heap
MemoryStack: bytes occupied by stack
MemoryFree = MemoryTotal - sum of memoryblocks

The pictures are valid for no·bytecodes executed (all variables are still uninitialized),
so this gives us info how to initialize the jvm parameters, as the outcome must
be as specified·in the status debug window,·below program source.

regards peter

jazzed · 2008-02-04 20:11

Ok Guys. Got this sucker running down to InvokeVirtual. Will look at that more later.
The code posted by default uses serial io and vga for debug. Start console program
and strike any key to make it run. Most debugging is turned off now. There are two
flags debugging and jazzDebug. Var debugMsDelay is used to control execution speed
or pause for input ... if value < 0, pause for input ... CR single steps, space lets it run.

--jazzed

jazzed · 2008-02-05 00:34

I've updated the writeboard design document with initialization and heap design information. API descriptions are also updated. Also I've registered for a sourceforge.net repository and waiting for approval. This can be a CVS repositiory if you guys are up for it; if you're not up for it, I can host the repository temporarily on one of my linux servers.

Peter Verkaik · 2008-02-05 16:03

Some more debug info.
I captured the communication between a javelin·and the IDE,
after giving the command Project->Debug.
The details are in the pdf.
It turns out that some debug commands sent to javelin, require
that a resetflag ($7E, same as end of packet indicator) is sent
back to the IDE.

It is imperative to use the stack frame layout as described in the attached
jvmJIDE.spin file, because the source level debugger uses that layout.
This spin file, when loaded into the spin stamp, allows us to use
Project->Debug, that downloads the jembytes, then resets the javelin,
and enters debug mode.
The picture shows the succesful completion of the Project->Debug command.

Next, I will try to figure out what the Step Into and Step Over commands
of the debugger do and expect. That should give us more clues on how the
jvm parameters must be updated. If possible, I will try to singlestep bytecodes.

regards peter

CJO · 2008-02-05 18:16

First of all, let me just say that you guys are gods among men.

Moving on to more fruitful topics:

1) With the existence of a TV/VGA screen with input devices, you get to where an O-O language can really shine: UI creation and manipulation. It seems to me that particularly without a GC, something like Swing is too complex, as it relies so heavily on dynamic object creation. However, something more in the Observer pattern might well be possible. Ultimately, I think that this would require some native methods, particularly to deal with keyboard/mouse position.

2) While I haven't near your understanding of Spin to help with the project, I am quite solid with O-O design, and would be happy to help with the 'System' type classes.

3) A GC and dynamic object creation would be really useful

4) "Thread/Runnable" native methods would be really cool

5) If you guys have SF cvs, etc that's great, otherwise, I have a server where I can provide svn/web hosting space (gratis, of course)

~ Christopher

jazzed · 2008-02-05 19:57

Hi,

The included package will run "Hello World" on the Propeller Protoboard.
Have not tested SpinStamp since I don't have that product.
Download, connect with terminal program, hit any key.

Peter, I've added framing information; you may need to check it.
The only native method that should work +for now+ is nm_core_CPU_message.
Source used is in the box below.

--jazzed ... to be meerly a meekly mortal.

public class mytest {
  static final int gBee = 0x1BEE;
  static void main() {
    int p1;
    System.out.println("Hello World");
    while (true);
  }
}

Post Edited (jazzed) : 2/5/2008 8:13:18 PM GMT

Peter Verkaik · 2008-02-05 20:39

I have been playing around with adding/removing static variables from my
test program and I found out the following relationship:

Edit: just discovered heapsize is calculated as 0x8000-jvmHP, instead of heapBase-jvmHP
(for the javelin heapBase = 0x7FFF and this makes the +1 which really is virtual, so no last byte)

When using Project->Debug, the used heapsize as given by the debug memory usage window,
equals static variables size + 1, no matter how·many static variables I declare.
This also means, in contrast to what I thought, that static variables are allocated
from the end of the heap, and·are not located at stackbase.

So here is the updated memory layout.

········ Memory map layout
· $0000· +

+ @javaProg
········ |··· header···· |
· $0014· +

+
········ |·· jemcodes··· |
········ |·· ...········ |
· $XXX0· +

+ stackbase················ ---
········ | Local0······· | <- JVM_FP················· ^
········ | ...·········· |··························· |
········ | LocalN······· |··························· |
········ +

+··························· |
········ | Method MSB··· | <- JVM_FP + numLocals····· |
········ | Method LSB··· |··························· |
········ +

+··························· |
········ | Old FP MSB··· |··························· |
········ | Old FP LSB··· |
········ +

+······················ stack frame
········ | JVM_PC MSB··· |···························
········ | JVM_PC LSB··· |··························· |
········ +

+··························· |
········ | Stack0······· | operand stack············· |
········ | ....········· |··························· V
········ +

+·························· ---
········ |·············· | <- JVM_SP
········ |···· free····· |
········ |·············· |

········ |·············· |
········ |···· free····· |
········ |·············· |
········ +

+
········ |·············· | <- JVM_HP
········ |·· allocated·· |
········ |············ · |
········ +

+··························
· JAVASIZE················ heapbase (on javelin $7FFF)

A note on the jvmdebug_details.pdf I uploaded earlier.
On the last page (QueryHeap) the value for heap[noparse][[/noparse]0] is $1D.
This is the byte[noparse][[/noparse]@javaProg], so QueryHeap is used to retrieve
a block from the entire memory used by the java program, not just
parts of the heap.
Also note that the checksum that follows $1D (=kDebugMagic), is always 0
(and therefore mostly incorrect) and that the IDE does NOT check this checksumbyte.
I can only guess the javelin·did not have enough space left to calc the checksum
for the QueryHeap·command.

regards peter

Post Edited (Peter Verkaik) : 2/5/2008 8:58:31 PM GMT

Peter Verkaik · 2008-02-05 23:04

@jazzed,

I was reading your code and I think your heap code is more (complex) than needed.
As there is no garbage collection, all we need is

PUB allocate(size)[noparse]:p[/noparse]tr
'allocate size bytes from heap
'return address of block or null
· if jvmHP - size =< jvmSP· 'jvmHP cannot go below jvmSP (see memory map)
··· return 0
· jvmHP -= size
· return jvmHP+1 'jvmHP holds address of last free byte

When an object must be stored on the heap

· ptr := allocate(objectsize)
· if ptr == 0
··· activity := kPassive 'QueryStatus command from IDE will read activity
··· exception(OUT_OF_MEMORY) 'this will display the runtime·error OUT_OF_MEMORY in the IDE status window
··· jvmHalt 'halt jvm

regards peter

jazzed · 2008-02-06 00:08

@Peter,

I agree the heap code is complicated. Hippy wrote the heap code; I added wrappers to keep me from breaking the heap [noparse]:)[/noparse]. Your approach is simpler if different types are not required. In defence of the original design, saving a string as a sequence of longs/words whatever is quite wasteful, and memory is tight. However, if one uses only word-size elements the impact is not so bad. Another item is that having the type byte define in-use or free is easier than some of the other first fit heap ideas (using the pointer to next address lsb for example can work but is difficult). If you want a generic malloc, that can be added as a wrapper with little grief. For me a wrapper is the right approach for now since there is so much more left to do. Free and collect should be fairly easy to implement when the time comes.

I'll add a variant of your allocate API function to the next JVM posting. If there are other API you would like, please don't hesitate to ask. We should discuss integration ... I would like to add some of the VGA_text style functions to you jvmSerial.spin so we can use the same file. Having two serial drivers doesn't scale. Another item that is interesting is "how much can the JVM code be a separate object?" Using DAT ... variables, the serial port is easily shared by different .spin files barring reentrancy issues. The big question I have is "can you try to use the JVM demo file?" ... how would you do that? Very little is public today but that could be easily changed of course.

I have all of your native methods getting parameters now. I looked around and it seems the message function is the only one that takes a "string" pointer. ... Please verify .... Also, i've commented most of the jazzDebug stuff and the code lost weight.

@Christopher,

Your comments on implementing swing are very relevant and welcome, it's possible that you have read the living spec I keep on the writeboard feed. Swing would be impossible on Propeller version 1 ... we are tight for space already and the JVM can only today do "Hello World"

Of course anything half serious in this realm must have garbage collection, and it is on the radar. The follow on Propeller with 256KB hub memory is a more likely candidate for AWT or Swing ... I hope that Parallax can afford a bigger memory with same cogs as now at some point; guess that depends on how much more memory could be added.

At this point, havng a thread class is not an option. I understand the desire as I've used threads in many projects. This would be reserved for the follow-on product though.

If you're interested ... there are several test cases in some of the other VM examples that would be nice to follow for functional testing. Let me know if you want to persue this. I am very aware of the often negative developer impulse regarding test, but this would be a great way for you to contribute after the JVM is a little more mature. If you like, I can send you pointers to that code.

Peter Verkaik · 2008-02-06 00:51

@Jazzed,
I do not have indepth knowledge of the heap specifics, but for it to work
with the source level debugger in the IDE, there must be a contiguous block
of memory as specified by the memory map. There is a heap object in the
object exchange (written by me) that is fully selfcontained in a byte array and could·be used
when garbarge collection is implemented, because you can let it start at any address,
and blocks can be allocated and freed in any order. It should only be adapted for
descending addresses, it currently uses ascending addresses.
StringBuffers and arrays of char, byte·and boolean all have byte elements (not words)
plus 4 bytes overhead (a 2byte object reference and a 2byte size field). There is no
need to specify a type field because the linker/compiler already has done the typechecking (and throws
an error if there is a type cast error).
I see the benefits for garbage·collection when more ram·comes available, for 32KB minus
the space required for the JVM and the native methods, having garbarge collection may
in fact leave us with less space than without·it, due to the size of the garbarge
collection code.

I consider the code I am working on as the frontend, downloading jembytes and initializing
the jvm parameters·and restarting to either run the jvm or run the jvm in debug mode.
At some point we must integrate but I think the frontend, the jvm and the garbage collection
can be made independant,·with just the java memory map layout and jvm parameters shared.

There are only three more debug commands I·need to check out and get working. Then I will
compile a new IDE, probably with an option to disable the expected echo. That should be sufficient
to let us download jemfiles using the prop TX/RX pins via the usb connector. The IDE releases
the serial port if it is not downloading or debugging so it can be shared with the proptool.

regards peter

jazzed · 2008-02-06 02:28

@Peter,

I respect the fact that you are a talented programmer. Please don't forget it.

Yes, I've seen your heap code and was wondering when that would come up.
Having been in this trench too long already, however, I would rather leave things as
they are for now since the basic requirements are being met until a more compelling
argument for change is made ... and/or when I have time to attend to the issue.

The front-end should not care much about the back-end implementation except that
the interface API layer be sufficiently defined. There is obviously some whitebox
character necessary in the JVM, but this should be expressed in an API.
I suggest you seriously consider an API layer now and offer suggestions.

As I understand it today the API requirements are for stack and heap inspection, some
stack manipulation, debug step/run control. You know this better than me, but I offer
the list below after looking at your jvmJIDE.spin in an effort as an amicable team player.

Please review and respond.
Cheers.

"Public JVM API"

PUB jvmStep
{{
/**
* This is a debug control command.
* Allow JVM to run one instruction (opcode/operand)
*/
}}

PUB jvmRun
{{
/**
* This is a debug control command.
* Allow JVM to run the bytecode program.
*/
}}

PUB jvmStop
{{
/**
* This is a debug control command.
* Stop the JVM from executing the next instruction and abort the current one.
*/
}}

PUB getJvmPC
{{
/**
* This is a debug status command that returns the value of the relative bytecode offset program counter.
* The absolute address of the program counter can be derived by subtracting the bytecode array base address.
* @returns value of the program counter.
*/
}}

PUB getJvmSP
{{
/**
* This is a debug status command that returns the "absolute" value of the stack pointer.
* The absolute value in this case means the SP value that the JVM keeps (not math absolute value definition).
* @returns value of the stack pointer.
*/
}}

PUB getJvmSB
{{
/**
* This is a debug status command that returns the "absolute" value of the stack base pointer.
* The absolute value in this case means the SB value that the JVM keeps (not math absolute value definition).
* This value is a representation of the JVM's internal BP or base pointer.
* @returns value of the stack base pointer.
*/
}}

PUB getJvmFP
{{
/**
* This is a debug status command that returns the "absolute" value of the Frame Pointer.
* The FP is defined as the beginning of the stack for the currently executing method.
* @returns value of the frame pointer.
*/
}}

PUB getJvmMP
{{
/**
* This is a debug status command that returns the relative bytecode offset value of the currently running
* Method's start address. This value is is the constant table method start address + the bytecode base address.
* The absolute address of the MP start can be derived by subtracting the bytecode array base address.
* @returns value of the method pointer.
*/
}}

PUB getJvmHeapStart
{{
/**
* This is a debug status command that returns the "absolute" current start position of the Heap pointer.
* @returns value of the heap pointer lowest memory position.
*/
}}

PUB getJvmHeapEnd
{{
/**
* This is a debug status command that returns the "absolute" current end position of the Heap pointer.
* @returns value of the heap pointer highest memory location.
*/
}}

Peter Verkaik · 2008-02-06 03:59

@Jazzed,
That's the idea, except that all addresses must be relative to @javaProg, the DAT area
that will hold the jembytes, stack, heap and static variables.
The IDE uses 0 as the start of that area.
Also, there must be a
PUB init
in the jvm object that initialises the parameters without running the jvm.

PUB jvmStep
{{
/**
* This is a debug control command.
* Allow JVM to run until a breakpoint
*/
}}

PUB jvmSingleStepBytecode 'not sure if this is possible
{{
/**
* This is a debug control command.
* Allow JVM to run one bytecode instruction (opcode/operand)
*/
}}

Up to 10 breakpoints can be downloaded into the javelin, a breakpoint
is the address (relative to @javaProg) at which the jvm must pause,
waiting for a new command from the IDE.
So the jvm also needs access to the JIDEComm routine in the frontend,
this must therefore be placed in a seperate object so it can be included
by both jvm and frontend.

When I think of other required methods I will let you know.

regards peter

Peter Verkaik · 2008-02-06 18:03

The attached zip contains 4 files:

testJVM4.java - my testfile
jem.out - the generated jembytes (135 kB)
status_stepinto.out - a detailed capture of communication between IDE debugger and javelin (1.02 MB)
status_singlestepbytecode.out - even more details, singlestepping bytecodes (5.75 MB)

I open the 3 .out files in notepad++ which makes it easy to go from one file to the other.
To see what really happens, observe the following entries:
sendSimpleCommand(3) = Step, lets javelin do next step
QueryStatus, returns jvmPC, activity, bytecode
Also of interest are QueryHeap that returns between 20 and 50 bytes
and QueryStack.

These debugfiles hold all the info we need to figure out how the bytecodes
are executed. The interesting stuff starts on line 130 in the·singlestep·file.
You can trace the program flow by comparing it with jem.out

regards peter

jazzed · 2008-02-07 04:19

@Peter, those are fascinating traces. I'm a regular Notepad++ user. Nice tool.
I've coded the API we discussed plus several more. Added a "debug menu"
to the JVM main for unit testing. All the API functions work. I'm having a little
trouble running jvmInit more than once; it will cause run and continue·to fail.

Now, when the "standalone" jvm boots, you can hit enter to see the debug menu
incase it scrolled by before you got connected. The "menu" describes commands.
I believe you have the public API's in this package necessary to start using it
at your pleasure from the GUI. They are in the doxygen marked ____ PUB ....

I'm including the JavaPropDesign and the Doxygen API specs in this post.

Guess I'll do more test and debug now.

Enjoy.

Peter Verkaik · 2008-02-07 06:16

@Jazzed,
I have also reorganized my code. I now have all reusable objects
that each deal with a very specific part.

jvmMain - top object, monitors reset and does initialization
and allows easy adding of userdefined native functions

jvmData - declares the shared data, using API as you suggested

jvmComm - communication with the IDE via the 'programming' port

jvmSerial - serial driver for the 'programming' port

jvmNative - the default native functions

jvmEngine - the bytecode interpreter

I have declared in jvmMain (but currently commented out):
OBJ
· jvm: "jvmEngine"

When I include that object the compiler throws an error
and I do not understand why, because that object only includes
objects that are already included.
Can you see what is wrong (or is this a compiler bug)?

regards peter

jazzed · 2008-02-07 15:50

Yes, I think you've hit a compiler bug/limit. I've seen other issues in having too many
layers/modules also, so i've generally tried to avoid too much "organizational engineering".

Take the files/structure below. Without including bigtest, the "data" in bigdata can be up to
$7700 or 30464 bytes. Including bigtest, the most "data" can be is $4fd4 or 20436 bytes.
If I include bigtest in test.spin, but leave out bigdata, the "data" can be up to $7700 bytes.
So, the problem appears to be a number of layers issue where bigdata is included, rather
than the number of times it's included issue.

CON
{
test.spin
}

OBJ
big: "bigdata"
inc: "include"
tst: "bigtest"

PUB main
inc.include

CON
{
include.spin
}

OBJ
big: "bigdata"

PUB include
big.get

CON
{
bigtest.spin
}

OBJ
big: "bigdata"
inc: "include"

PUB main
inc.include

CON
{
bigdata.spin
}

DAT
data byte 0 [noparse][[/noparse]$5000]

PUB get
return data

Peter Verkaik · 2008-02-07 18:10

I found a reasonable workaround by putting the large array into
its own object file that is only included by the top object file.
Other objects that must reference the array get the array properties
via their init routine.

@Jazzed,
I added an error object (jvmError) that has all the·jvm errors that are defined
by the IDE. These are runtime errors that can occur, either when the jvm just
runs or runs with·the source level debugger active.

I understand you like to use VGA for your jvm debugging (and this is seperate
from the jvm running with the IDE source level debugger).
I think it is possible to only include the VGA object in the top object file,
by extending my main loop.

· repeat 'keep running
··· jd.init(jprog.getJavaProgram,jprog#JAVASIZE)· 'initialize jvm registers and pass javaProg properties
··· if comm.isReset
····· comm.doReset
····· comm.clearReset
····· repeat
······· c := comm.doDebug
····· until c == -1
··· else
····· 'mainloop
····· repeat while NOT comm.isReset
······· if jd.debugger
········· c := comm.doDebug 'check for debug commands
······· c := doBytecode···· 'execute next bytecode
······· case c
········· < 0: jvmDebugPrint(c) 'print errormessage for jvm debugging
········· 0..47: nf.doNative(c)
········· 'add userdefined native functions here

If we allow doBytecode to return a negative address,
the error messages can be generated from the mainloop.·The positive address
would be an array of parameters used by a generic error print routine,
which only needs to be included in the top object file.
I would include a serial object, where you would include the VGA object.

regards peter

JVM for prop

Comments