what exactly is a
Archiver
Posts: 46,084
What exactly is a "token"?
Is this in assembly language? Can I view it?
Does anyone have an example so I can better grasp the idea of
a "token"?
Much thanks in advance.
Is this in assembly language? Can I view it?
Does anyone have an example so I can better grasp the idea of
a "token"?
Much thanks in advance.
Comments
a byte, but not always) that calls a piece of assembly language. Tokens can
also be data, or pointers to data (i.e., a specific variable). The BASIC Stamp
interpreter reads the tokens from EEPROM, then calls the appropriate assembly
language routine(s) built into the Stamp.
There's no easy way to see them as they change with data. If you want,
though, you can select File | Generate Object Code, then save the OBJ file.
This
will be a 2K file that contains the compressed tokens and DATA for your Stamp
program. You'll need a binary editor of some sort to look at the file's
contents.
-- Jon Williams
-- Parallax
In a message dated 6/3/2003 11:47:39 AM Central Standard Time,
basicstampede@y... writes:
> What exactly is a "token"?
>
> Is this in assembly language? Can I view it?
> Does anyone have an example so I can better grasp the idea of
> a "token"?
>
> Much thanks in advance.
>
[noparse][[/noparse]Non-text portions of this message have been removed]
Usually, a programming statements gets "compiled" into machine code (or byte
code, as Java, and as, I assume, the basic stamp handles it). For example,
the code below:
A VAR WORD
B VAR WORD
C VAR WORD
A = $0001
B = $1234
C = A + B
The first three lines really don't have anything to do with the execution of
the code. They tell the stamp to set aside a bit of RAM to store some
numbers. The last three lines actually say what to do with this space. Three
words are set aside, with a reference to each being represented by "A", "B"
and "C". Internally, the code for the operations are encoded, then written
to the EEPROM as bytes that represent the operations to perform.
Let's say $00 represents "put a value into a memory location", and $01
represents "add the contents of these two memory locations together, and put
the sum into a third memory location". Now lets say that "A" above
represents memory location $00 (NOT the operation $00, but a memory
location), "B" is $02, and "C" is $04. Then the code above becomes:
$00, $00, $0001 (A = 1, or, the first $00 means put the third portion,
$0001, into the memory location of the second part, $00 - "A")
$00, $02, $1234 (B = $1234, or the first $00 means put the third portion,
$1234, into the memory location of the second part, $02 - "B")
$01, $00, $02, $04 (C = A + B, or the first $01 means add the second
portion, $00 [noparse][[/noparse]meaning "A"], to the second portion, $02 [noparse][[/noparse]meaning "B"], and
put the result into the third portion, $04 [noparse][[/noparse]meaning "C"])
Now, to me token may mean two things. Either each element in the original
source code could represent a token (A, B, C, VAR, WORD, "=", "+"), or each
"portion" of actual code could be a token (a portion might be a byte
representing an action, or a byte, bit, nibble, word, etc, representing a
variable, memory location, or constant).
It all depends on how the team at Parallax meant the work "token" to be
used.
A disclaimer - I have no idea how the internal functioning of the stamp work
exept what I have gleaned from the documentation or the stuff I read from
the web or this list, but is how I interpret it from my experience with
programming both the stamps and software for the PC.
Steve
Original Message
From: "basicstampede" <basicstampede@y...>
To: <basicstamps@yahoogroups.com>
Sent: Tuesday, June 03, 2003 6:46 PM
Subject: [noparse][[/noparse]basicstamps] what exactly is a "token"?
> What exactly is a "token"?
>
> Is this in assembly language? Can I view it?
> Does anyone have an example so I can better grasp the idea of
> a "token"?
>
> Much thanks in advance.
>
>
> To UNSUBSCRIBE, just send mail to:
> basicstamps-unsubscribe@yahoogroups.com
> from the same email address that you subscribed. Text in the Subject and
Body of the message will be ignored.
>
>
> Your use of Yahoo! Groups is subject to http://docs.yahoo.com/info/terms/
>
display on the left shows the contents of the memory of the
interpreter chip. You have to scroll down to the bottom of the list
to see the program tokens. The tokens start at the highest addresses
(2047, $7ff) and work downward. DATA starts by default at the lowest
addresses (0) and works upwards.
A token is a single group of bits that represents a complex program
operation, such as, "call the subroutine that generates a pulse on an
output pin, using the duration stored on the top of the operations
stack", or "put the current value stored in the word variable w7 onto
the top of the operations stack" or "add the two top entries on the
operations stack and put the result back onto the top of the
operations stack". Each of the complex operations is encoded into
just a few bits that cause the interpreter to branch out to the
proper subroutines, which are in fact written in assembly language.
But the tokens themselves are not assembly language. They are more
like shorthand. The BASIC Stamp has a particularly compact form of
tokens. In some computers, like the old Tandy model 100, each BASIC
token took one byte, or a multiple of bytes. In the Stamp, the
length of individual tokens can be anywhere from one bit to many
bits. That is why it is hard to see a pattern as you look at the
CTRL-M display.
As you expand and change your program, you will see the CTRL-M
display fill up, but the pattern of the tokens will not leap out at
you, because the token length often changes across byte boundaries.
However, the bytes you see in that display are the ones that are
downloaded to the Stamp when you choose RUN.
If you are interested in details, take a look at Brian Forbes' book,
<http://members.aol.com/stamp2book/>
or the BS1 article by Chuck McManis (a great lesson in deductive reasoning)
<http://www.mcmanis.com/~cmcmanis/robotics/stamp-decode.html>
-- best regards
Tracy Allen
electronically monitored ecosystems
http://www.emesystems.com
mailto:tracy@e...
>What exactly is a "token"?
>
>Is this in assembly language? Can I view it?
>Does anyone have an example so I can better grasp the idea of
>a "token"?
>
>Much thanks in advance.
statements and data into smaller / shorter codes and stores them in the
Stamp memory. The Stamp converts these codes into machine code and runs
them.
Original Message
> Hmmm ... tough to describe easily. Think of a token as a small chuck
(often
> a byte, but not always) that calls a piece of assembly language. Tokens
can
> also be data, or pointers to data (i.e., a specific variable). The BASIC
Stamp
> interpreter reads the tokens from EEPROM, then calls the appropriate
assembly
> language routine(s) built into the Stamp.
>
> There's no easy way to see them as they change with data. If you want,
> though, you can select File | Generate Object Code, then save the OBJ
file. This
> will be a 2K file that contains the compressed tokens and DATA for your
Stamp
> program. You'll need a binary editor of some sort to look at the file's
> contents.
> > What exactly is a "token"?
> >
> > Is this in assembly language? Can I view it?
> > Does anyone have an example so I can better grasp the idea of
> > a "token"?