What is the 'token'?

k lee

hello.
I am trying to make a new editor instead of Basic stamp editor.
I am using tokenizer.dll with c# to make a tokens but I can not understand what is the token?
From using toknizer.dll, it makes serveral data.. examples.· eeprom data, eeprom flags, packet buffer, packet count...etc.
But which data shoul be download to BS2?
What is exact token·download to BS2?
Please, give a answer.

$WMc%

k lee

I believe a TOKEN is just and Assembly code, Compiled by the compiler ( BASIC Stamp Editor).

Take a look at PIC assembly code.

______________$WMc%________________

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
The Truth is out there············___$WMc%___···························· BoogerWoods, FL. USA

Mike Green

That information is not readily available. There is a document that comes with the tokenizer that discusses how to download a binary program to a Stamp, but it doesn't discuss the actual format of the compiled byte codes (the tokens). Parallax considers that information proprietary and has not published any description.

There are some websites (and books) that describe the byte codes for the BS1, but I'm not aware of any that document the byte codes for the BS2 devices.

The byte codes (tokens) have no relationship with any kind of assembly instructions.

k lee

Thank you for reply.

According to the document 'using tokenizer library', should I download the packetbuffer to BS2 or eeprom data?
or I misunderstanding?
Which data is tokenized data?
I am really complicated.
Please, give me an answer.

dev/null

Indeed you sound a little complicated.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Don't worry. Be happy

k lee

I'm sorry. I'm not English user. ^^;
I should ask step by step.
first, I am trying to make an editor with c# using tokenizer.dll. Is that possible?

Adrian Schneider

You might want to browse through the C++ code inside the tgz-file to get a clue what and how to download and hwo to use the tokenizer.
This is a Linux example, however, it should be sufficiently general to be of use under C# and Windows.
Regards
Adrian

Tracy Allen

Did you know that in the BASIC Stamp editor, when you choose the Memory Map (^M), you can see the hex codes exactly as they have been tokenized and will be downloaded to the BASIC Stamp. The tokenizer dll produces that same HEX file. For downloading to the Stamp, that file is divided into 18-byte packets that consist of 16 bytes of HEX data (exactly as it appears in the Memory Map), preceded by an address of where that data will be placed in the Stamp EEPROM, and followed by a single byte checksum for the packet.

I have more information on the downloading process in PBASIC posted at : www.emesys.com/BS2clone.htm.

The actual tokens are embedded in the HEX file, and you don't have to be concerned with what the actual tokens are. The tokenizer.dll takes care of that for you. The tokens are clever, and complicated. If you are really interested in how the tokens themselves are formed, look for Brian Forbes' self published book, Inside the Basic Stamp II. (The web page is down, but can be found on the web archive here Or Chuck McMannis' original deconstruction of the Basic Stamp I, online here., which is very similar in concept.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Post Edited (Tracy Allen) : 10/13/2009 5:16:49 PM GMT

k lee

Thank you, everyone.

I made packetized data which is packetbuffer, and it consists 18 byte data.
and I just write as following to download these packets to BS2,

byte[noparse]/noparse packet = new byte[noparse][[/noparse]18]{0xFF, 0x33, 0x3b, 0x80, 0xc9, 0xe6, 0x8a, 0x8e, 0x49, 0xc6, 0xf4, 0xd1, 0x9a, 0xd9, 0x03, 0x07, 0xc0, 0x3b}
serialPort1. open();
serialPort1.Write(packet, 0, packet.Length);

but, Nothings happen.
now, I have to study hard;;

annisti

Thank your for sharing this wonderful story. I find your post very informative.
I am very excited to be a part of this Forum and look forward to bringing Value to it.

Acai Force Max
Acai force Max

Tracy Allen

k lee,
Before sending the data packets, the Stamp has to be put into programming mode. This is accomplished by by toggling the DTR line high (ATN reset to the Stamp) while the TX line is held high (BREAK condition). After TX comes back low, there is an exchange of bytes that identifies the Stamp and its firmware version number. Only after that is successful can the PC start to send the 18 byte data packets.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Unsoundcode

Hi k lee, I have a small application that was programmed with Visual Basic Express edition that would transfer a tokenized program to a BS2.

The application is not an IDE programming tool that was not what I needed , the purpose was just for downloading tokens to the Stamp.

During one·stage of development I used a Form with 1 Button control , 1 Textbox control and·1 Serialport control to test the reset and exchange of bytes that Tracy speaks of. The app was set up to display the results of the byte exchange in a Textbox control to verify success.

I am attaching the code ( which you should place in the Button Click event ) , it should not take much to convert to C#.

Each Stamp requires a different byte exchange and this example is only for the plain BS2.

The next step·I took after verifying the this reset and ID·routine was to send just one 18 byte packet , something like DEBUG "Hello"·. Verify the reset and ID outlined in the text file first though.

Jeff T.

k lee

Thank you everyone.
I made it!! ^^