Shop OBEX P1 Docs P2 Docs Learn Events
How to understand the hex dump of a Spin/PASM binary file — Parallax Forums

How to understand the hex dump of a Spin/PASM binary file

WossnameWossname Posts: 174
edited 2017-06-14 18:04 in Propeller 1
EDIT:
If anyone has some obscure, rare or self-authored Propeller 1 related documentation please consider adding it to this thread as an attachment so that it may be preserved before the ravages of time steal it from us forever.

I will collate any such files into a zip file for easy access at a later date on the main Propeller Resources thread.




I have created a piece of test code ("test.spin" attachment) in order to try to understand the structure of the eeprom/binary file ("test.eeprom" attachment) created by the Propeller Tool.

I hex-dumped this eeprom file thus...
0000000: 00b4c404 6f2b1000 d000d800 a000dc00  ....o+.......... 'clock config
0000010: c0000200 90000000 00000000 00000000  ................ 'jump to Spin section???
0000020: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog0's PASM
0000030: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog1's PASM
0000040: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog2's PASM
0000050: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog3's PASM
0000060: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog4's PASM
0000070: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog5's PASM
0000080: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog6's PASM
0000090: 010afc0c 00000000 00000000 030a7c0c  ..............|. 'cog7's PASM
00000a0: 3722c780 80352c38 06c77035 2c3805c7  7"...5,8..p5,8..
00000b0: 60352c37 01c75035 2c3721c7 40352c37  `5,7..P5,7!.@5,7
00000c0: 00c73035 2c36c720 352c35c7 10352c32  ..05,6. 5,5..5,2
00000d0: fffff9ff fffff9ff 00000000 00000000  ................ 'marker and end of program...
00000e0: 00000000 00000000 00000000 00000000  ................
00000f0: 00000000 00000000 00000000 00000000  ................
'zeroes ad infinitum...

Basically I'm attempting to run some simplified PASM code on all 8 cogs in sequence.

Am I right in thinking that the byte sequence "c0000200 90000000" is an absolute jump to the beginning of the interpreted Spin language section between bytewise addresses 00000a0 and 00000cf inclusive? If it's not a jump but something else, what is it?

The "c0000200 90000000" sequence is followed by my two debugging NOP instructions and then by each of my cogs' allocated PASM code chunks (the 8 repeated identical lines) then by the spin bytecode).

I've scoured the manual and the datasheet for the P8X32A but can find no detailed information about the spin language section of this data. How might I be able to decode this in order to find out the addresses that each of my 8 cogs will take their PASM code from (for instance if my cogs were all running much larger, arbitrary sized sections of PASM)?

I can see some patterns in the spin language section that appear to correspond with the original test.spin file but there are some anomalous bytes here and there which seem to disrupt those patterns a little. I'd appreciate some guidance on this.

Comments

  • |===========================================================================|
    Compile it using BST {view - compiler listing]
    |===========================================================================|
    Objects : -
    test
    
    Object Address : 0010 : Object Name : test
    
    Binary Image Information :
    PBASE : 0010
    VBASE : 00D0
    DBASE : 00D8
    PCURR : 00A0
    DCURR : 00DC
    |===========================================================================|
    |===========================================================================|
    Object test
    Object Base is 0010
    |===========================================================================|
    Object Constants
    |===========================================================================|
    Constant _clkmode = 00000408 (1032)
    Constant _xinfreq = 004C4B40 (5000000)
    |===========================================================================|
    Object DAT Blocks
    |===========================================================================|
    0018(0000) 00 00 00 00 |           nop 'DEBUGGING ONLY
    001C(0001) 00 00 00 00 |           nop 'DEBUGGING ONLY
    0020(0002)             |                         org     0
    0020(0000)             | ENTRY_POINT_COG_0
    0020(0000) 01 0A FC 0C |                         cogid   c0_cogid
    0024(0001) 00 00 00 00 |                         nop
    0028(0002) 00 00 00 00 |                         nop
    002C(0003) 03 0A 7C 0C |                         cogstop c0_cogid
    0030(0004)             | c0_time                 res     1
    0030(0005)             | c0_cogid                res     1
    0030(0006)             |                         fit
    0030(0006)             | There are 490 ($1EA) Longs left in the cog
    0030(0006)             |                         org     0
    0030(0000)             | ENTRY_POINT_COG_1
    0030(0000) 01 0A FC 0C |                         cogid   c1_cogid
    0034(0001) 00 00 00 00 |                         nop
    0038(0002) 00 00 00 00 |                         nop
    003C(0003) 03 0A 7C 0C |                         cogstop c1_cogid
    0040(0004)             | c1_time                 res     1
    0040(0005)             | c1_cogid                res     1
    0040(0006)             |                         fit
    0040(0006)             | There are 490 ($1EA) Longs left in the cog
    0040(0006)             |                         org     0
    0040(0000)             | ENTRY_POINT_COG_2
    0040(0000) 01 0A FC 0C |                         cogid   c2_cogid
    0044(0001) 00 00 00 00 |                         nop
    0048(0002) 00 00 00 00 |                         nop
    004C(0003) 03 0A 7C 0C |                         cogstop c2_cogid
    0050(0004)             | c2_time                 res     1
    0050(0005)             | c2_cogid                res     1
    0050(0006)             |                         fit
    0050(0006)             | There are 490 ($1EA) Longs left in the cog
    0050(0006)             |                         org     0
    0050(0000)             | ENTRY_POINT_COG_3
    0050(0000) 01 0A FC 0C |                         cogid   c3_cogid
    0054(0001) 00 00 00 00 |                         nop
    0058(0002) 00 00 00 00 |                         nop
    005C(0003) 03 0A 7C 0C |                         cogstop c3_cogid
    0060(0004)             | c3_time                 res     1
    0060(0005)             | c3_cogid                res     1
    0060(0006)             |                         fit
    0060(0006)             | There are 490 ($1EA) Longs left in the cog
    0060(0006)             |                         org     0
    0060(0000)             | ENTRY_POINT_COG_4
    0060(0000) 01 0A FC 0C |                         cogid   c4_cogid
    0064(0001) 00 00 00 00 |                         nop
    0068(0002) 00 00 00 00 |                         nop
    006C(0003) 03 0A 7C 0C |                         cogstop c4_cogid
    0070(0004)             | c4_time                 res     1
    0070(0005)             | c4_cogid                res     1
    0070(0006)             |                         fit
    0070(0006)             | There are 490 ($1EA) Longs left in the cog
    0070(0006)             |                         org     0
    0070(0000)             | ENTRY_POINT_COG_5
    0070(0000) 01 0A FC 0C |                         cogid   c5_cogid
    0074(0001) 00 00 00 00 |                         nop
    0078(0002) 00 00 00 00 |                         nop
    007C(0003) 03 0A 7C 0C |                         cogstop c5_cogid
    0080(0004)             | c5_time                 res     1
    0080(0005)             | c5_cogid                res     1
    0080(0006)             |                         fit
    0080(0006)             | There are 490 ($1EA) Longs left in the cog
    0080(0006)             |                         org     0
    0080(0000)             | ENTRY_POINT_COG_6
    0080(0000) 01 0A FC 0C |                         cogid   c6_cogid
    0084(0001) 00 00 00 00 |                         nop
    0088(0002) 00 00 00 00 |                         nop
    008C(0003) 03 0A 7C 0C |                         cogstop c6_cogid
    0090(0004)             | c6_time                 res     1
    0090(0005)             | c6_cogid                res     1
    0090(0006)             |                         fit
    0090(0006)             | There are 490 ($1EA) Longs left in the cog
    0090(0006)             |                         org     0
    0090(0000)             | ENTRY_POINT_COG_7
    0090(0000) 01 0A FC 0C |                         cogid   c7_cogid
    0094(0001) 00 00 00 00 |                         nop
    0098(0002) 00 00 00 00 |                         nop
    009C(0003) 03 0A 7C 0C |                         cogstop c7_cogid
    00A0(0004)             | c7_time                 res     1
    00A0(0005)             | c7_cogid                res     1
    00A0(0006)             |                         fit
    00A0(0006)             | There are 490 ($1EA) Longs left in the cog
    |===========================================================================|
    |===========================================================================|
    Spin Block EngineTemplateMain with 0 Parameters and 0 Extra Stack Longs. Method 1
    PUB EngineTemplateMain
    
    Local Parameter DBASE:0000 - Result
    |===========================================================================|
    9                        coginit(7, @ENTRY_POINT_COG_7, 0)
    Addr : 00A0:          38 07  : Constant 1 Bytes - 07 - $00000007 7
    Addr : 00A2:       C7 80 80  : Memory Op Long PBASE + ADDRESS Address = 0080
    Addr : 00A5:             35  : Constant 1 $00000000
    Addr : 00A6:             2C  : CogInit(Id, Addr, Ptr)
    10                        coginit(6, @ENTRY_POINT_COG_6, 0)
    Addr : 00A7:          38 06  : Constant 1 Bytes - 06 - $00000006 6
    Addr : 00A9:          C7 70  : Memory Op Long PBASE + ADDRESS Address = 0070
    Addr : 00AB:             35  : Constant 1 $00000000
    Addr : 00AC:             2C  : CogInit(Id, Addr, Ptr)
    11                        coginit(5, @ENTRY_POINT_COG_5, 0)
    Addr : 00AD:          38 05  : Constant 1 Bytes - 05 - $00000005 5
    Addr : 00AF:          C7 60  : Memory Op Long PBASE + ADDRESS Address = 0060
    Addr : 00B1:             35  : Constant 1 $00000000
    Addr : 00B2:             2C  : CogInit(Id, Addr, Ptr)
    12                        coginit(4, @ENTRY_POINT_COG_4, 0)
    Addr : 00B3:          38 04  : Constant 1 Bytes - 04 - $00000004 4
    Addr : 00B5:          C7 50  : Memory Op Long PBASE + ADDRESS Address = 0050
    Addr : 00B7:             35  : Constant 1 $00000000
    Addr : 00B8:             2C  : CogInit(Id, Addr, Ptr)
    13                        coginit(3, @ENTRY_POINT_COG_3, 0)
    Addr : 00B9:          38 03  : Constant 1 Bytes - 03 - $00000003 3
    Addr : 00BB:          C7 40  : Memory Op Long PBASE + ADDRESS Address = 0040
    Addr : 00BD:             35  : Constant 1 $00000000
    Addr : 00BE:             2C  : CogInit(Id, Addr, Ptr)
    14                        coginit(2, @ENTRY_POINT_COG_2, 0)
    Addr : 00BF:          38 02  : Constant 1 Bytes - 02 - $00000002 2
    Addr : 00C1:          C7 30  : Memory Op Long PBASE + ADDRESS Address = 0030
    Addr : 00C3:             35  : Constant 1 $00000000
    Addr : 00C4:             2C  : CogInit(Id, Addr, Ptr)
    15                        coginit(1, @ENTRY_POINT_COG_1, 0)
    Addr : 00C5:             36  : Constant 2 $00000001
    Addr : 00C6:          C7 20  : Memory Op Long PBASE + ADDRESS Address = 0020
    Addr : 00C8:             35  : Constant 1 $00000000
    Addr : 00C9:             2C  : CogInit(Id, Addr, Ptr)
    17                        coginit(0, @ENTRY_POINT_COG_0, 0) '<-- this must be the last SPIN instruction
    Addr : 00CA:             35  : Constant 1 $00000000
    Addr : 00CB:          C7 10  : Memory Op Long PBASE + ADDRESS Address = 0010
    Addr : 00CD:             35  : Constant 1 $00000000
    Addr : 00CE:             2C  : CogInit(Id, Addr, Ptr)
    Addr : 00CF:             32  : Return
    
  • The values "c0000200 90000000" make up the method table for the object. The first word is $00c0, which is the size of the object in bytes. The next two bytes indicate the number of methods+1 and the number of objects. The next two words give the address offset and number of local variables in the first and only method of the object. The starting address is $a0, which is $0090 + $10.

    The fffff9ff longs at the end are actually a stack frame that is placed at the beginning of the stack. When the first method in the top object returns it actually jumps to location $fff9, which is located at the end of the ROM. The code at $fff9 executes a COGID followed by a COGSTOP.
  • @Dave Hein,

    Does there exist some official documentation regarding this data structure layout?
  • This may have what you want. There used to be a Propeller Wik containing this sort of stuffi, but it's been discontinued by the host organization
  • Thank you Mike. I'll have a look at this.

    I did see that the wiki that used to have the "3 bit protocol" information has vanished. I have a copy on my 1st gen Kindle from way back.

    Feels like Propeller is an endangered species. :(
  • Most of that early stuff is here, but so "way back" that it's very hard to find.
  • Wossname wrote: »
    I did see that the wiki that used to have the "3 bit protocol" information has vanished. I have a copy on my 1st gen Kindle from way back.

    Feels like Propeller is an endangered species. :(
    And Gadget Gangster left without handing over a backup of his creations to someone... I sometimes try to dig deep in the WayBackMachine... :-(

    Do you mean this wiki?
    —▶ https://github.com/rosco-pc/propeller-wiki
  • David BetzDavid Betz Posts: 14,516
    edited 2017-06-13 13:33
    Wossname wrote: »
    @Dave Hein,

    Does there exist some official documentation regarding this data structure layout?
    The short answer is no, there is no "official" documentation on this. Various people have figured it out by reverse engineering binary files generated by the Spin compiler but there is no official Parallax document describing the binary file format or the Spin byte code instruction set. Chip has promised that there will be one for the P2 Spin byte code instruction set and binary file format though.
  • WossnameWossname Posts: 174
    edited 2017-06-13 17:35
    Ok. I can use the info in this thread to get me a bit further. I'll probably hard-code some of the address jumping for now and revisit it later on. Doing much more than that, I suspect, would require writing a Spin interpreter in C which I'm not looking forward to if there's no documentation!

    Thank you all for your help :)

    Cheers.

    w.
  • @Peter Jakacki,

    Thanks for the tip about `bstc -ls` for viewing the listing with source. Brad's tools are pretty awesome.
  • Dave HeinDave Hein Posts: 6,347
    edited 2017-06-13 19:15
    Wossname wrote: »
    Doing much more than that, I suspect, would require writing a Spin interpreter in C which I'm not looking forward to if there's no documentation!
    Or you could look at the source code for the Spin interpreter used by spinsim. It's already written in C. Or you could look at the PASM source for the interpreter.

    The inner workings of the Spin interpreter were never meant to be documented. Chip went through some effort to scramble the bits so it would be difficult to reverse engineer. He finally released the source for the interpreter when people were getting close to descrambling the binary code located in the ROM.

  • Here's another "documentation" file
  • This is useful stuff.

    I encourage people to attach any files containing useful data (particularly if it's been lost over the years and gone '404') and I'll go through this thread in a a few weeks' time and wrap all of them up into a tarball we can all download in one lump.

    Here's one from a now '404' wikia site which I cannot remember the name of. It discusses the "3 Bit Protocol" (3PB) used to push program binaries / eeprom files over a serial port (RS-232 signalling) connection...


  • This document contains a description of the spasm assembler that I wrote and describes the Spin byte codes. There are no official opcode names for the Spin bytecodes, so I defined some names and use them in spasm.
  • Cluso99Cluso99 Posts: 18,069
    There is also a doc I put together from various sources including Chip's Interpreter when I wrote my Fast Spin Interpreter. The info should be in the Fast Spin Interpreter thread.
  • Here's a link to the source code for the booter, runner and the interpreter. It appears that the original thread is no longer available, but Jon McPhalen re-posted the source code in the linked thread.
  • It's amazing that there is no official source for this information. Every time someone wants to understand a Spin binary file we go through the same archeology trying to find ancient sources for this information.
  • David Betz wrote: »
    It's amazing that there is no official source for this information. Every time someone wants to understand a Spin binary file we go through the same archeology trying to find ancient sources for this information.

    How about something like github? If we can collect a bunch of all the non-proprietary information that people have gleaned over the years and are willing to submit it, would that be a good way to maintain (I hesitate to use the term "crowd-sourcing") documentation? Any improvements made over time can be integrated using the normal github mechanisms. I don't know how Parallax would feel about such a thing though, we'd be wise to ask their blessing, and we'd need to nominate some people to maintain it. How practical is that? I don't know.

    Not sure how else we'd do it except doing a wiki-style site. But that's also something that needs to be maintained by a group - not by an individual. We've already lost wiki's due to unpaid subscriptions.

    Hmm, it's a tricky one.
    Dave Hein wrote: »
    Here's a link to the source code for the booter, runner and the interpreter. It appears that the original thread is no longer available, but Jon McPhalen re-posted the source code in the linked thread.

    I've zipped them up into one file and I'll post it here for the hell of it. Moar copies is moar betterer.

    Keywords for this file so that search engines can find it...

    spin, pasm, pnut, booter, interpreter, runner, parallax, propeller, p8x32a, docs, documentation, how to, source, code.




  • Wossname wrote: »
    How about something like github? If we can collect a bunch of all the non-proprietary information that people have gleaned over the years and are willing to submit it, would that be a good way to maintain (I hesitate to use the term "crowd-sourcing") documentation?

    https://github.com/rosco-pc/propeller-wiki/wiki ... this one?
    Or let's clone and update it?

    GitLab's/GitHub's wikis are very basic... basically I don't like them... but they are easy to clone (a.k.a. to backup)... so data there should be easy to save and to plug in somwhere else even if GitHub evolves in a bad direction...

    Maintaining static websites with GitHub/GitLab is not much harder but much more flexible... but may need writing everything in HTML like my notes and ceatsheets and so on...
  • WossnameWossname Posts: 174
    edited 2017-06-15 18:33
    yeti wrote: »
    so data there should be easy to save and to plug in somewhere else even if GitHub evolves in a bad direction...

    In a nutshell that's the problem, yeah. The term "platform agnostic" information retrieval springs to mind but I can't put my finger on a system that exists anywhere that isn't owned by some random company that could get bought out overnight.

    I'm starting another thread for this generalised problem of knowledge retention... http://forums.parallax.com/discussion/166867/solving-the-documentation-problem-opinions-please
  • yetiyeti Posts: 818
    edited 2017-06-15 18:20
    Wossname wrote: »
    In a nutshell that's the problem, yeah. The term "platform agnostic" information retrieval springs to mind but I can't put my finger on a system that exists anywhere that isn't owned by some random company that could get bought out overnight.
    Websites and wikis made with GitHub/GitLab are git repositories and can be replicated without GitHub/GitLab too...


  • WossnameWossname Posts: 174
    edited 2017-06-18 11:07
    Anyway, back to the original topic of this thread. :)

    The first two bytes of my Spin block in the compiled EEPROM file I'm looking are 0x37 and 0x22 in that order.

    In this context, 0x37 means PUSH a 32-bit value on the stack. The value pushed is given by left-shifting 0x00000001 by the 5 LSB's of 0x22 and if bit 5 of 0x22 is set, subtract one from the value.

    0x22 is "0010 0010" in binary. The decrement flag (red) is set and the 5 LSB's contain 2.

    So...
    ((1 << 2) - 1) == 3

    But `bstc -ls` decodes this as...
    coginit(7, @ENTRY_POINT_COG_7, 0)
    Addr : 00A0:          37 22  : Constant Mask Y=34 Decrement 00000007
    

    Where did the 7 come from?

    The definition for the behaviour was in a random file named "Propeller Spin Bytecodes.txt" posted on this forum, original author unknown.

    Is this unofficial "documentation" wrong? It works if you use (2<<2)-1==7 but that goes against the specific examples given in the document.

  • The unofficial documentation is wrong. The value is obtained by left-shifting the value 2, and not 1. The reason 2 is used is somewhat arbitrary, but it results from subtracting 0x35 from 0x37. This is because the codes 0x34 to 0x37 are processed together, and are decoded by first subtracting 0x35. The code 0x34 results in a -1, 0x35 is 0, 0x36 is 1 and 0x37 results in a 2, which is then left shifted by the value in the 5 LSBs. Chip saved an instruction in the interpreter by using this definition for 0x37.
  • WossnameWossname Posts: 174
    edited 2017-06-18 11:22
    That makes sense then. Thanks for clarifying. I did try to work it out by looking at the interpreter's PASM source code but it made my brain squeak.

    That's a pretty slick optimisation though.
  • WossnameWossname Posts: 174
    edited 2017-06-20 18:22
    Post deleted... I've decided to post this as a separate thread.
Sign In or Register to comment.