Shop OBEX P1 Docs P2 Docs Learn Events
The Bytecode — Parallax Forums

The Bytecode

Lee MarshallLee Marshall Posts: 106
edited 2007-12-23 00:53 in Propeller 1
Is parallax willing to give us the spin bytecode specs???

i have noticed this being brought up a number of times, but now i am a little confused. Mainly due to the existance of the GEAR project on sourceforge, which has also been discussed on this forum.
wouldnt the developer(s) of GEAR have needed access to the bytecode details, in order to write the interpreter for the program.
is there now a datasheet that there wasnt before? if so, where is it?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
I hear and I forget. I see and I remember. I do and I understand
-Confucius

Comments

  • Ym2413aYm2413a Posts: 630
    edited 2007-10-12 00:33
    Mr Crowley said...
    Is parallax willing to give us the spin bytecode specs???

    i have noticed this being brought up a number of times, but now i am a little confused. Mainly due to the existance of the GEAR project on sourceforge, which has also been discussed on this forum.
    wouldnt the developer(s) of GEAR have needed access to the bytecode details, in order to write the interpreter for the program.
    is there now a datasheet that there wasnt before? if so, where is it?

    I think in most cases, Such as with GEAR. It's just been some really good reverse engineering. [noparse]:)[/noparse]
    People are smart on this forum and will figure anything out giving time.

    --Andrew Arsenault
  • Lee MarshallLee Marshall Posts: 106
    edited 2007-10-12 00:35
    im surprised parallax are ok with that.
    y'know copyright/blahblahblah

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    I hear and I forget. I see and I remember. I do and I understand
    -Confucius
  • rjo_rjo_ Posts: 1,825
    edited 2007-10-12 00:42
    The problem is... in most cases, what can be reversed engineered won't get you where you think you are going.
    There are some road blocks upstream... [noparse]:)[/noparse]
  • Lee MarshallLee Marshall Posts: 106
    edited 2007-10-12 00:45
    another thing is the serial communication between the prop and PC.
    what is the baud rate/protocol?

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    I hear and I forget. I see and I remember. I do and I understand
    -Confucius
  • Mike GreenMike Green Posts: 23,101
    edited 2007-10-12 00:52
    The download protocol has been discussed before. Here's a Spin program that acts like the Propeller Tool. It's intended as a worked example of how to do the download. There's also a Python version.

    http://forums.parallax.com/showthread.php?p=591445
  • AribaAriba Posts: 2,685
    edited 2007-10-12 01:05
    Hippy has made a Bytecode decompiler:
    http://forums.parallax.com/showthread.php?p=665019

    I think he is the person with the best knowledge of the bytecode at the moment (exept the Parallax people).

    But what is your intention with the Bytecode? To use the Spin Interpreter for another language? Or an alternativ Spin Compiler for Linux/MAC?
  • rjo_rjo_ Posts: 1,825
    edited 2007-10-12 01:14
    As you probably already know, Chip also created an object that allows you to load a binary from one prop to another. If your intention is to create a compiler, information that you would need is not in the public domain, has been stated to be propietary, and cannot be discerned through remote viewing.
    Anyone offering you such information... other than Parallax... is either a thief or a liar.
  • Lee MarshallLee Marshall Posts: 106
    edited 2007-10-12 02:16
    im curious as to why it is "secret".
    how can it benefit parallax? they give away the software freely.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    I hear and I forget. I see and I remember. I do and I understand
    -Confucius
  • hippyhippy Posts: 1,981
    edited 2007-10-12 02:17
    rjo_ said...
    As you probably already know, Chip also created an object that allows you to load a binary from one prop to another. If your intention is to create a compiler, information that you would need is not in the public domain, has been stated to be propietary, and cannot be discerned through remote viewing.

    The bytecode has been determined through the efforts of GEAR and by Cliff L. Biffle ( who both deserve much credit ) and I've built my work on top of theirs. Others may have also had input of whom I'm not aware. I know others have worked on things such as Callbacks, determining stack operation and so on ( sorry, cannot give credit without a forum search ).

    Bytecode can be determined for any particular Spin construct and a compiler just has to do the opposite. What the Propeler Tool generates can be observed by looking at the image map or a saved .binary or .eeprom file. The initialisation bytes have been documented and can also be created by a compiler.

    Apart from one undocumented bytecode I don't see what stands in the way of a third-party Spin Compiler other than getting it done. That one bytecode I don't particularly worry about because if it's not in any code decompiled it doesn't have to be generated by the compiler, and when we do finally find it in some Propeller Tool generated image code it will hopefully be reasonably easy to determine its operand(s) and what it does. That's the same for the 'known unknowns' which have been identified, which are potentially there but have not materialised in actual image code.

    Yes, Spin and the bytecode is proprietary, Parallax have clearly said they are not going to document the bytecode and they have put nothing for that into the Public Domain, but there's plenty of reverse engineered information and documentation which is. No, Parallax will probably not confirm the reverse engineering is correct, but that doesn't mean it will not be. The third-party representations of de-compiled bytecode is probably very different to what Parallax themselves use, but that doesn't matter.

    The structure and layout of the image code, the way methods are called, the way objects are included and so forth can all be determined by examining the decompilation. The download protocol has been well documented and implemented by third-parties in a number of ways.
    rjo_ said...
    Anyone offering you such information... other than Parallax... is either a thief or a liar.

    I don't really know how to take that, or give an answer to it. Perhaps you could clarify how you mean ?
  • hippyhippy Posts: 1,981
    edited 2007-10-12 02:26
    Mr Crowley said...
    im curious as to why it is "secret".
    how can it benefit parallax? they give away the software freely.

    It's previously been said by Parallax that they will not be documenting the bytecode because they do not want the support issues which would come with doing that ( paraphrased ).

    Besides, if they documented everything, where would the fun of reverse engineering come from ? smile.gif
  • ericballericball Posts: 774
    edited 2007-10-12 20:52
    One interesting challenge would be to create an HLL to SPIN bytecode cross compiler.
  • deSilvadeSilva Posts: 2,967
    edited 2007-10-12 21:12
    A good question is, why people have not yet ported the host of portable languages to the Propeller: Java, C, Python, Perl,...

    It can be done by three approaches:
    - Use SPIN bytecode
    - Use own bytecode (and provide interpreter) , example is FORTH
    - Use of a variation of LMM, as "C in work"

    I see five problem areas - there are more for sure!
    - Speed of SPIN interpreter is not very convincing
    - SPIN bytecode might not match well with requirements from different languages
    - COG memory is quite low for any other sufficiently powerfull VM
    - HUB memory is low for run-time and library systems needed for sophisticated applications
    - Common "Operating System Support", as libraries, linkers is totally missing.

    I should expect when a running LMM is available (and the interface to it documented), as well as a standard overlay-mechanism (from EEPROM or SD), things will accelerate considerably.

    It seems to be a matter of helpful de-facto standards, not so much of workload or feasibility....
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2007-10-12 21:31
    Another possible reason: producing a compiler takes a whole wad of time and effort to develop and support; and there's no money in it unless you're either a) selling the target hardware, b) able to convince the hardware manufacturer to pay for it, or c) have a cadre of lawyers on staff to enforce your copyrights. Anyone who attempts it just for the glory is either retired, living on royalties from a hit single, or simply delusional. smile.gif

    -Phil
  • Mike GreenMike Green Posts: 23,101
    edited 2007-10-12 21:37
    Phil,
    The other choice is for a student or group of students to do it as a class or master's level open source project.
  • deSilvadeSilva Posts: 2,967
    edited 2007-10-12 21:40
    That's it: "delusional" smile.gif
    It is not so much work as you expect to port one of the free compilers when you have already done it once. Maybe this "community" is a little bit biased to "electronics" rather than to computerscience.

    There are two annoying tasks: To really understand the VM (or any intermediate code) - when it is your first contact to that (portable) compiler.
    Adding the interfaces to the library system (I/O, memory manegement, arithmetic support, string handling,...)

    Writing the interpreter for the VM or a generic code generator is fun (and debugging smile.gif )
  • Phil Pilgrim (PhiPi)Phil Pilgrim (PhiPi) Posts: 23,514
    edited 2007-10-12 22:02
    Mike, you're right! Aside from money and glory, I forgot grades and a degree!

    Seriously, though, it would make a nice master's level project, and I didn't mean to sound cynical.

    -Phil
  • Lee MarshallLee Marshall Posts: 106
    edited 2007-10-12 22:06
    Why doesnt anyone write an operating system, im probably gonna try, but interrupts would be useful, just the ability for one cog to make another cog jump to another cog ram address without having to restart the cog...

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    I hear and I forget. I see and I remember. I do and I understand
    -Confucius
  • Mike GreenMike Green Posts: 23,101
    edited 2007-10-12 22:46
    There are no interrupts in the Propeller. It was specifically designed not to need/have them. One cog cannot make another cog do anything except stop or reload/restart.

    There already is an operating system (see the sticky thread for Graham Stabler's Good Thread Index). Not much has been done with it for some time other than incorporate a lot of the routines into the various flavors of FemtoBasic.
  • Fred HawkinsFred Hawkins Posts: 997
    edited 2007-10-13 08:42
    deSilva said...
    A good question is, why people have not yet ported the host of portable languages to the Propeller: Java, C, Python, Perl,...

    When people get·good enough at Spin and Prop Assembly to port one of these, maybe they decide they don't need·to -- if they have an interest in using the Prop.

    And if their real·interest is porting,·they may feel that there·are less difficult hurdles to jump.·
  • hippyhippy Posts: 1,981
    edited 2007-10-13 13:11
    deSilva said...
    A good question is, why people have not yet ported the host of portable languages to the Propeller: Java, C, Python, Perl,...

    Primarily I believe it boils down to an individual or group having a need or want for that, or ( in rare cases ), is altruistically aiming to provide what others may want or need, or is aiming to engender support for their favourite programming language or provide greater accessibility to the Propeller.

    To port a language to the Propeller there are a number of requirements; a good understanding of the language and/or tools being ported, a good understanding of the Propeller Chip, plus the will, motivation and skills to do it. It is likely that this combination hasn't occurred yet.

    I've actually been so impressed by Spin ( against my expectations ), that I'm starting to become more interested in porting Spin to other micro-controllers.
  • Ken PetersonKen Peterson Posts: 806
    edited 2007-10-13 15:45
    I think it comes down to necessity. The effort required to port a language to the Propeller may outweigh its benefits. If someone had a project where this solution was the best solution for the project then it would make sense to expend the effort. Some people like to build engines to try to make their car faster, but in the end it still gets you to work. Most of us are more apt to try to get a good price on a car with good fuel economy. This is what I think the Propeller provides for the average engineer or hobbyist.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


    The more I know, the more I know I don't know.· Is this what they call Wisdom?
  • potatoheadpotatohead Posts: 10,260
    edited 2007-10-13 16:05
    I like this analogy.

    Personally, I find the whole SPIN / ASM combination to be just excellent. The up and coming C compiler is gonna address some wants of mine, but SPIN / ASM will handle the needs.

    Half the time, me wanting a feature, or cursing the lack of one, comes down to me wanting to do it the "old" way, or "traditional" way. The Propeller really isn't either of these things. Some time spent thinking, or maybe just away from the problem, often yields some simple solution.

    Then there is scale. The current Prop is right at, or maybe below, the scale where other languages and environments make better sense for more use cases. Prop II will hit square into that territory, and that's when we will see a whole bunch of activity on that front.

    Do we have some terminology for that aspect of things?

    There is the word micro that gets globbed onto computer and controller. Does the same work for mini? As in minicontroller, where a mini is bigger than a micro, but not so big as to warrant some other word, besides controller?

    If so, Prop I is a micro, Prop II will be a mini. Languages and such work for minis better than micros.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Propeller Wiki: Share the coolness!
  • RedNifreRedNifre Posts: 84
    edited 2007-12-22 19:41
    This looks like a giant waste of time to me. I understand that Parallax don't want to be annoyed with questions about the byte code.
    But why don't they just release it with a huge disclaimer like:
    WARNING!
    By downloading the Bytecode/VM/DownloadProtocol-Specifications you agree to never ask any questions.
    If you violate this rule you will be shot.

    Then everyone would be happy...

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Hydra in a LEGO NES:
    http://forums.parallax.com/showthread.php?p=654788
  • Mike GreenMike Green Posts: 23,101
    edited 2007-12-22 20:30
    That's an idealistic view. It doesn't really work that way, particularly for a small company like Parallax.

    For one, I'm sure there is no specification that could be released. The way these things are done is that
    there are a lot of napkin notes, often with pizza sauce splotches. The napkins are not necessarily numbered
    and the notes are for the benefit of the creator and not necessarily written for someone else to read. It's a
    lot of work to rewrite the napkin notes into a form for an expert to understand, let along someone without
    just the right combination of expertise to understand. People will call and e-mail for further information and
    either the creator will have to spend lots and lots of time to document further or have to hire someone
    relatively expensive to write it up and still have to devote too much time to answer questions from the tech.
    writer.

    In any event, everyone will not be happy. It just doesn't happen.

    I remember an operating system that I worked on where a lot of time was spent over Pizza Hut food and drink.
    The owner even gave us the heavily marked up table cloths since we were such good customers (and did offer
    to pay for them). They got hung on the walls back at work as part of the formal documentation.
  • deSilvadeSilva Posts: 2,967
    edited 2007-12-22 20:37
    Send me the napkins smile.gif
  • AleAle Posts: 2,363
    edited 2007-12-22 21:55
    RedNifre said...
    By downloading the Bytecode/VM/DownloadProtocol-Specifications you agree to never ask any questions.
    If you violate this rule you will be shot.

    The part about violence, is not really needed here. just a license "as-is" is just good enough.
    OTOH, what for ?.... alternatives will emerge/are emerging.


    Edit: Mike, not all businesses are that unprofessional. Some are worst smile.gif. But from what I saw, parallax is not one of them.

    Post Edited (Ale) : 12/22/2007 10:18:56 PM GMT
  • RedNifreRedNifre Posts: 84
    edited 2007-12-22 23:33
    Oh, I'm sorry. I thought that there already IS a tiny byte-code-documentation.

    Oh well...

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Unterwelt (My first game for the HYDRA):
    http://forums.parallax.com/showthread.php?p=696204

    Hydra in a LEGO NES:
    http://forums.parallax.com/showthread.php?p=654788
  • CJCJ Posts: 470
    edited 2007-12-22 23:35
    has it been determined whether the interpreter is encrypted in ROM?

    just wondering if we are just looking for an easy way to the byte code, femtobasic and propdos both have the address of the interpreter.

    ▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
    Parallax Forums - If you're ready to learn, we're ready to help.
  • Mike GreenMike Green Posts: 23,101
    edited 2007-12-23 00:53
    The interpreter is indeed encrypted in ROM. The hardware knows that the address is in ROM and decrypts it as it's copied to the cog's memory, probably using a LFSR.
Sign In or Register to comment.