Bytemove question/help
turbosupra
Posts: 1,088
Hi,
I thought I understood bytemove, but now I'm not so sure. Can anyone tell me why the bolded line below reads "Tom" instead of "100" ? My code is shown below and also attached.
Thanks for reading!
I thought I understood bytemove, but now I'm not so sure. Can anyone tell me why the bolded line below reads "Tom" instead of "100" ? My code is shown below and also attached.
Thanks for reading!
Parallax Terminal Reads: wrote:Enter a name, then an equals sign and then their age: tom=100
You typed: tom=100
pst.Str(b_Ptr2b_RxString): tom=100
pst.Dec(variableLength) 0 based: 6
postion of delimiter is pst.Dec(b_PosOfDelimiter): 3
prefix := s2.Parse(b_Ptr2b_RxString, 0, b_PosOfDelimiter): tom
suffix := s2.Parse(b_Ptr2b_RxString, b_PosOfDelimiter, strsize(b_Ptr2b_RxString)): tom
Enter a name, then an equals sign and then their age:
CON _clkmode = xtal1 + pll16x _xinfreq = 5_000_000 byteLimit = 100 VAR byte b_Ptr2b_RxString[byteLimit] byte b_RxString[byteLimit] byte b_Pos[byteLimit] byte b_PosOfDelimiter[byteLimit] byte prefix[byteLimit] byte suffix[byteLimit] OBJ pst : "Parallax Serial Terminal" s2 : "Strings2" PUB Main pst.Start(115_200) pst.Clear repeat pst.Str(String(pst#NL, "Enter a name, then an equals sign and then their age: ")) pst.StrIn(@b_RxString) pst.Str(String(pst#NL, "You typed: ")) pst.Str(@b_RxString) pst.Str(String(pst#NL, pst#NL)) DelimiterFinder(@b_RxString) PUB DelimiterFinder(RxStringAddr) | l_localindex, variableLength bytemove(b_Ptr2b_RxString, RxStringAddr, strsize(RxStringAddr)) ' RxStringAddr has been sent a byte address with the command DelimiterFinder(@b_RxString), so no need for the @ symbols in the method 'b_Ptr2b_RxString := RxStringAddr pst.Str(String("pst.Str(b_Ptr2b_RxString): ")) ' Since RxStringAddr was sent as an pointer, and the pointer was copied to b_Ptr2b_RxString, you do not need the @ symbol pst.Str(b_Ptr2b_RxString) pst.Str(String(pst#NL)) b_Pos := 0 variableLength := strsize(b_Ptr2b_RxString) - 1 pst.Str(String("pst.Dec(variableLength) 0 based: ")) pst.Dec(variableLength) pst.Str(String(pst#NL)) repeat l_localindex from 0 to variableLength if byte[b_Ptr2b_RxString + b_Pos] == "=" b_PosOfDelimiter := b_Pos b_pos := b_pos + 1 pst.Str(String("postion of delimiter is pst.Dec(b_PosOfDelimiter): ")) pst.Dec(b_PosOfDelimiter) pst.Str(String(pst#NL)) ' Parse (strAddr, start, count) 'prefix := s2.Parse(b_Ptr2b_RxString, 0, b_PosOfDelimiter) bytemove(prefix, s2.Parse(b_Ptr2b_RxString, 0, b_PosOfDelimiter), strsize(s2.Parse(b_Ptr2b_RxString, 0, b_PosOfDelimiter))) pst.Str(String("prefix := s2.Parse(b_Ptr2b_RxString, 0, b_PosOfDelimiter): ")) pst.Str(prefix) pst.Str(String(pst#NL, pst#NL)) ' suffix := s2.Parse(b_Ptr2b_RxString, b_PosOfDelimiter, strsize(b_Ptr2b_RxString)) bytemove(suffix, s2.Parse(b_Ptr2b_RxString, b_PosOfDelimiter, strsize(b_Ptr2b_RxString)), strsize(s2.Parse(b_Ptr2b_RxString, b_PosOfDelimiter, strsize(b_Ptr2b_RxString)))) pst.Str(String("suffix := s2.Parse(b_Ptr2b_RxString, b_PosOfDelimiter, strsize(b_Ptr2b_RxString)): ")) pst.Str(suffix) pst.Str(String(pst#NL)) pst.Str(String(pst#NL)) return b_PosOfDelimiter
Comments
The important point for showing Tom again is, that you try to get the suffix by start parsing from position of the delimiter. But you should start from delimiter + 1. The delimiter is set to 0 (=stringend) by the previous parse. This means that strsize returns 0 which tells bytemove to leave the memory untouched. As you forgot to use @ both, suffix and prefix will show the same result in the output, as you actually do output memory location 0.
I did not test my version, but it should at least show some errors.
Why are b_Pos and b_PosOfDelimiter arrays? b_Pos is no longer needed and b_PosOfDelimiter should be a word. That's enough to find the delimiter in a string that has a size of the whole HUB RAM.
Further improvement could be to not copy the prefix and suffix into different arrays at all. But this depends on the usage of b_Ptr2b_RxString. If you need it for further input while suffix and prefix are needed it's fine as it is. But if there is no additional input to parse an alternative would be to store the addresses of suffix and prefix in pointer variables instead of copying the content. Then you would simply make em word variables as well and do:
prefix := s2.parse( @b_Ptr2b_RxString, 0, b_PosOfDelimiter )
suffix := @b_Ptr2b_RxString + b_PosOfDelimiter + 1
Thanks for the reply, I guess I haven't got my head as wrapped around strings and when to use the @ symbol, as I had hoped that I did. b_Ptr2b_RxString is still being modified in the new code, and I had to use the original "repeat l_localindex from 0 to variableLength", because the new one loops through the entire string and not each character/byte.
Does the function itself inherently modify the address it is passed no matter what? Maybe that is the problem? I copied the StrParse below, maybe it needs to be modified?
bytemove(b_Ptr2b_RxString, RxStringAddr, strsize(RxStringAddr))
is adding :
I added some logging and comments so I could understand how it worked, changed the abort to a return, because that was the only way I could get it to prompt me as a loop and not hang, although that may not have been the best way to handle that?
prefix is a label which points to the memory location which is used to store a string. If you use prefix in your program it will read/write the content of that memory location. Like this for example
prefix := "A"
The array notation works a little bit different. prefix[2] will add 2 to the memory address and read from/write into this location.
prefix[1] := "h"
So, both of these access methods really look into the memory location that the label points to. The @ tells the compiler/interpreter that you're
not interested in the content but you want to know the address itself.
Most string-functions for example are not interested in the content but want to know where to find the content. As strings can be any size it does not make sense to pass the whole string to a function.
BYTEMOVE also wants to know addresses for both, the to and the from parameter!
bytemove(@prefix, RxStringAddr, --idx)
So, why does it look like that? Because prefix contains the first character of the prefix string if you use it without @. But BYTEMOVE is not interested in that, it wants to know the location where to store and thus expects an address. That's why you need to use @prefix here.
RxStringAddr contains the address of the b_RxString-buffer. Remember:
DelimiterFinder(@b_RxString)
....
PUB DelimiterFinder(RxStringAddr)
this is simply the same as
RxStringAddr := @b_RxString
So ... it's as easy as that: remember what the variables contain and what the functions/instructions expect. This should help at least until you start working with pointers to pointers .... ;o)
But the from parameter could have, or not have the @ symbol in front of the variable, depending on if I already have a variable address or variable content assigned to that variables name.
If I'm correct, I think I'll have an easier time remembering it from now on (I've struggled with this on and off for probably a year now) but if I'm still not understanding completely, please let me know so I don't try and commit something to memory that isn't right. If most string functions have this logic, that will help a lot as well.
Well ... not exactly ... you're mixing up things again! The bytemove definitely expects an address to a buffer for the to- and for from-parameter. There is no exception - that's how bytemove is programmed. It will treat whatever you pass as first parameter as the address where it has to copy things to and the second parameter as the address from where to copy. By the way .. bytemove does not care about whether it's copying a string or whatever.
Variables are nothing else but names for memory addresses. Your suffix-variable for example simply is the address of the first byte of an area that could be used to store a string. Making it an array simply means that the number of bytes specified in the square brackets are now reserved. If you use the variable name anywhere in the program means that you are not interested in the address but you want to read/write the value of that memory-location. If you use @ you tell the compiler that you are not interested in the content, but you want to know the address of the memory-location.
But variables can contain anything, which means that you can also store the address of another variable in it. And that's what makes the difference.
bytemove( @suffix, @b_rxString, 10)
suffixAdr := @suffix
b_rxString_addr := @b_rxString
bytemove( suffixAdr, b_rxString_addr, 10)
These two are doing exactly the same. suffixAdr is a variable which contains the address of the location to be used as to-buffer. suffix is a variable which contains the first byte of the string, so you have to use the @ to find out at which address suffix starts.
I think I do understand now. What you wrote in your last post is what I understood, but I did a poor job of explaining myself. When I edited my previous post, I put "(unless there is a situation where, with the bytemove command, you've previously assigned the "to" to another variables memory address, but I cannot think of a situation where I would need to do this)" at the end of the second paragraph, instead of at the end of the first paragraph. Me explaining this might sound confusing it, and I apologize for that. I have now edited to read I as originally intended it to read.
Your example and explanation in this last post is perfect and clears up a lot when you say that bytemove ALWAYS expects a memory address, no matter what. It can accept a variable name, only when that variable name is previously assigned to a memory address and that is what was confusing me before, now I understand why!
This would also explain why sometimes when I used the @ symbol, it would return unexpected results, because I believe I was assigning an an address to a variable that was already assigned to another address. Is that correct?
Can I carry this knowledge you have given me, over to any built in spin method that requires an address as one or more of its parameters such as STRCOMP (StringAddress1, StringAddress2 )? And possibly even to methods that others have written, if they imply an address is required by writing their code such as Pub method(varAddr) ?
Another way would be to simply return -1 to indicate that the data was invalid and handle it upstairs.
Assign a byte from the string to c, then increment the index (post-increment). If c is 0 (terminator, (not 0) == TRUE) we haven't found a delimiter, report error.
OK, but as RxStringAddr is an addr (you passed it into this function) I'd say copy from RxStringAddr to @prefix ...
No. This simply adds idx + 1 to the variable RxStringAddr (A += 1 is equivalent to A := A + 1). In the context of this function RxStringAddr is a long variable. RxStringAddr[idx + 1] is therefore some value on the stack, idx + 1 longs after RxStringAddr itself (see below).
OK, again copy from RxStringAddr ...
Imagine this setup: Here the first assignment reads from the first (and only long) in the addr array. The second (index out of range) reads from the next long variable (here second). The same would happen with RxStringAddr[idx + 1]. RxStringAddr[0] accesses the parameter itself ([0] is optional, i.e. A == A[0]). Therefore RxStringAddr[idx + 1] will grab something from somewhere else. Certainly not what we intended here.
HTH
Hi,
Thanks for the reply, that does help explain. I'm still having trouble with strings longer than 7 characters though as my post above this one indicates. Are there any special situations pertaining to that?
Sometimes a variable is just a number, no strings attached involved
Some comments to the code from post #13:
You should alway initialize local variables before using. As far as I remember the function call will only reserve the local variables on the stack but not initialize them!
If you do so, you can also change the behaviour a little bit, which leads to the fact that after leaving the loop idx will already point to the place where the "=" has been found:
What you try to do in variableUpdator is to convert the string representation of a number to the native number representation of the propeller which is a long. What's the idea behind copying the string again?
Anyway ... what's the major problem with
var1value := n.FromStr(@var1value, %000_000_000_0_0_000000_01010)
The point is that var1value is an array of bytes. The assignment var1value only works with the first byte! So, even if the n.FromStr returns a long, it will be cut to a byte with this assignment. And assigning a stringend to a binary variable is some kind of senseless.
Comments to post #15:
"var1 is initialised with an address, var1value looks like it is supposed to hold a number. This means both should be a long."
Actually it is enough to have a word for storing addresses, as the propeller only has 64kB of address-space (32kB HUB-RAM and 32kB HUB-ROM).
@turbosupra:
Maybe there is some lack in understanding of the string()-function and that's why you used VAR byte var1[byteLimit]?!
var1 := string("value1")
string() is a function which is build into the compiler. When the compiler finds such an instruction, it reserves some memory and stores the bytes representing "value1" plus the string-end character ($00) at this memory location. The assignment actually assigns this memory address to var1. What you also should know about string is, that the compiler does not optimize. This means when you use string("value1") at one place and string("value1") on another place again, the compiler won't recognize that both strings are the same. So, in the end you'd have 2 copies of the same string in the memory.
A general comment: SPIN can deal with any datatype that buildin into the propeller which are byte, word and long. So, there can't be an operator which deals with strings directly! All datatypes which might come into your mind have to be handled by functions. And unless these datatypes don't fit into a long the only way to pass these datatypes to and return them from functions is via addresses.
I had thought anytime you were passing a phrase, a string (byte array) is the proper way to "handle" it? You have turned my world upside down by showing me that mentality was wrong. I made the changes you suggested and the program works, no strings attached! I believe my core issue is now not knowing when to use a string (byte array), byte, and long. Do you have any suggestions on how I can learn this? You have been very generous with your time and I do not want to "wear out my welcome" with question after question.
Thinking back on my coding, I've consistently had trouble using functions or writing them, because I was not sure which type variable I should be using or expecting? I've never had this problem when writing in c sharp, but this weakness is exposed in spin.
I appreciate you reiterating, sometimes it is easier to understand when you read 2 explanations about the same thing, thank you.
I will put this into practice now and always initialize local variables, for the most part do you always initialize them to 0? ( localVariable := 0 ) Is this not necessary with global/non-local variables?
This makes such sense when you write it this way. Of course I was only able to get it to work with something that was 1 byte after being parsed, because only the first byte array value would be shown and the remaining bytes may not have even transferred!!
I think I understand this concept, but for the sake of simplicity and not confusing myself I would like to use longs only at this point, if that is ok
I believe I have a dysfunctional understanding of when I should use a string (byte array) or byte or long. It sounds like the built in string function is a wrapper for a byte array and a memory address to that byte array? You have been quite generous with the time you given to me in looking at my code and critiquing it, and I am very appreciative of that. I don't want to wear out my welcome with continued question after question, do you have any suggestions on how I can learn when and where to use a string (byte array), byte or long?
Using a long instead of a word is not problem.
Ok ... it looks like you understood string(). It's creating a byte array somewhere in memory which has exactly the size of the data you put into the brackets plus one for the stringend which is appended automatically. The memory address is so to say the return-value of this function.
Don't worry about asking questions?! This is a forum and that's what a forum is good for.
When to use what ... well ... easy ... you use what fits best!
If you want to enter text via keyboard or terminal, for sure the byte array is the best fit. If you want a PC style input you simply append each key to a input-buffer. When the Enter-key is hit you replace it (0xd) with a stringend (0x00) and your input buffer is ready to be processed by any string-function. This is done by pst.StrIn(@b_RxString) in your code. What I'd do in a professional key-input-function is to limit it to the number of bytes available in the array. Otherwise each keypress which exceeds this limit will overwrite other memory.
From there on most times it makes sense to convert the string representation to something that's more usefull for the propeller. Which means you'd convert to byte, word or long. Which type is the best simply depends on which values make sense in your program. For example if the user is allowed to enter a percentage from 0 to 100 used later to create an equivalent PWM-output, a single byte is enough. If you want to send the input value to a 16-bit D/A converter you need a word ....
If you need bigger datatypes than that you have to work with arrays and propably with your own code that can deal with these datatypes. For example if you need 64bit integers and need to calculate with these, the normal operators alone won't help.
Thank you for the reply. This sounds a bit foolish when I type it, but I'd like to recap to make sure I've gotten everything in my brain correctly
1.) A byte array, limited to approximately around 128 array containers is best for string input from a serial connection (I read this somewhere this week while searching the forum)
2.) At the point of splitting and working with that byte array, it is best to convert to a propeller native data type, that specific type depends on the number of bits you expect the "split" values to be
3.) You should reference that byte array with the @ symbol pointing to the address in any method that asks for the address, otherwise it will only read byteArray[0]/the first byte in that byte array
4.) If you assign a variable to a reference memory address, you do not have to reference the newly assigned variable with the @ symbol because the variable is storing that reference memory address
5.) The native string() spin function assigns each character in a string, to a byte array that is 1 byte larger than the number of characters/spaces in the string. An example would be
strVal := string("ThisIsAString") which is a 0-13 or 14 bucket byte array
strVal[0] is a byte for "T" / hex "54" / binary "01010100"
strVal[1] is a byte for "h" / hex "68" / binary "01101000"
strVal[2] is a byte for "i" / hex "69" / binary "01101001"
strVal[3] is a byte for "s" / hex "73" / binary "01110011"
strVal[4] is a byte for "I" / hex "49" / binary "01001001"
strVal[5] is a byte for "s" / hex "73" / binary "01110011"
strVal[6] is a byte for "A" / hex "41" / binary "001000001"
strVal[7] is a byte for "S" / hex "53" / binary "01010011"
strVal[8] is a byte for "t" / hex "74" / binary "01110100"
strVal[9] is a byte for "r" / hex "72" / binary "01110010"
strVal[10] is a byte for "i" / hex 69/ binary "01101001"
strVal[11] is a byte for "n" / hex "6e" / binary "01101110"
strVal[12] is a byte for "g" / hex "67" / binary "01100111"
strVal[13] is a byte for 0 / hex "00" / binary "" (this is to tell spin the string is terminated)
6.) Local variables are uninitialized longs
7.) Global variables are initialized, no matter the type
8.) Methods return longs in most cases
9.) "result" is a local built in return variable, initialized when a method is called
10.) "idx" is an alias for "result", or at least was in the previous code example PUB DelimiterFinder(RxStringAddr) : idx | c
11.) All method parameters are longs
I would venture you'll have some corrections for me ... are there any other things that I'm missing? I've read the Propeller manual and datasheet multiple times and not gotten any of this out of it.
Ehmmm ... no ... this is not a general rule. Maybe it's true in some context for example if data comes in fast and in bursts and you have to buffer it before you can process it or your processing is sometimes to slow to process the package before the next arrives. Of course in average processing needs to be fast enough otherwise you get a buffer overflow no matter how big the buffer is.
If we talk about keyboard input you should simply have a buffer which is big enough to store the longest expected input and have a function which limits the input to this lenght.
"2.) At the point of splitting and working with that byte array, it is best to convert to a propeller native data type, that specific type depends on the number of bits you expect the "split" values to be"
Well ... if you enter string 123 and 234 ... how do you want to add those numbers? Of course you can implement a function which adds numbers in string representation but it's much easier to add native datatypes. If you only read the string from one interface and you have to send it out on another interface which is string-based, there is of course no need to convert.
"3.) You should reference that byte array with the @ symbol pointing to the address in any method that asks for the address, otherwise it will only read byteArray[0]/the first byte in that byte array"
CORRECT
"4.) If you assign a variable to a reference memory address, you do not have to reference the newly assigned variable with the @ symbol because the variable is storing that reference memory address"
I think either my english reading is wrong or your writing:
If you assign a reference memory address to a variable -> means variableAddress := @variable
If so -> CORRECT
"5.) The native string() spin function assigns each character in a string, to a byte array that is 1 byte larger than the number of characters/spaces in the string."
CORRECT .... BUT
strVal := string("ThisIsAString") which is a 0-13 or 14 bucket byte array
strVal[ 0 ] is a byte for "T" / hex "54" / binary "01010100"
strVal[ 1 ] is a byte for "h" / hex "68" / binary "01101000"
all lines with strVal[ x ] are NOT CORRECT
strVal is a variable which needs to be a word or a long because it only needs to store the memory address of the string - better to name it strAddr. Where the string actually is stored will be assigned to strAddr. If you want to access characters of that string you have to use
byte[ strAddr ][ 0 ]
6.) Local variables are uninitialized longs
CORRECT
7.) Global variables are initialized, no matter the type
CORRECT - initialized of course means they are initialized once
8.) Methods return longs in most cases
in ALL cases
9.) "result" is a local built in return variable, initialized when a method is called
CORRECT
10.) "idx" is an alias for "result", or at least was in the previous code example PUB DelimiterFinder(RxStringAddr) : idx | c
CORRECT
11.) All method parameters are longs
CORRECT
Thanks for validating my understandings! I definitely feel 10x more confident about this than I did a few days ago, I owe you a beer!
That makes sense and leads into another question.
My ultimate goal will have a PC sending updates to values every so often. The format will be something like "prefix=suffix", possibly followed by an "end of command" character. So pst.StrIn would receive a command that was loaded into a byte array. Do you suggest I program it so that pst.StrIn receives one command at a time, passes it to a parsing method and then an update method, and then receives the next command? Or should I have the PC send one long string and parse through the string of bytes after it is done sending? Which way shall I set up the buffer?
I don't know if I'll be doing much math, although I will be passing numbers in the prefix=suffix and the suffix will contain numbers to update longs and those longs will have math performed on them inside of the code. What do you think is the best course of action for this?
So if I named it strAddr and assigned it to a string with strAddr := string("ThisIsAString"), would strAddr hold a memory address and strAddr[1] be the byte memory address of strAddr + 1 byte?
The most important points are:
1. Do it step by step
2. Always try to separate code into good functions which work on their own
So, concrete, start with a readLine-function which matches your needs. Then go on with a function that parses the input. And last do the processing. If you have good functions you can reuse them even if you decide to work with several COGs. You only need some glue-code which syncs the functions (call them at the right time) and maybe some simple instructions to convert the buffer to a ring-buffer.
First I'd start with a simple buffer that's long enough to hold the longest input line.
"End of command" makes sense ... and you can use "Enter" for that, so you can test your code without a dedicated PC program - simply use a terminal and enter the update-strings manually. Working on both sides (PC and Propeller) at the same time doubles the chance of bugs and the needed efford to find them.
So ... you're always welcome to tell more details about your project. The more the better the tips can be.
The advantage of longs is that these also need less memory for storage. But in the end it really makes only sense in case you do at least one calculation with it or you have to store the values. If you only display the strings coming from the PC then you'd simply copy the number-part of the input string to the right location on your screen and you're done. No conversion, no storage.
If your PC values don't fit on one screen and you have to page it starts making sense to store the long values because storing complete screens is to wastefull and it's pretty easy to recreate a string out of a long.
Ok ... again the string(;o) What happens if you access strAddr[1]? (which by the way is the same as long[ @strAddr ][ 1 ])
Well ... you would not access the string but the variable nextLong because @strAddr + 1 long = @strAddr + 4 bytes = @nextLong
As I already posted, for accessing content of the string (for example the 2nd character) you have to use
byte[ strAddr ][ 1 ] which is equivalent to byte[ $0200 ][ 1 ] or byte[ $0200 + 1 ]. Byte is simply the way to tell the compiler that you want to read the content found on the given address.
I do not for see any of my projects approaching anywhere near 80mhz speed, I would guess that the fastest I would have to monitor anything would be about 500hz per input lol! I don't think the prop will even break a sweat, but not having a lot of experience with this, I cannot say that authoritatively, nor can I be sure that one cog could handle that.
I will probably use a "enter" as my delimiter, it's a natural fit. Currently I have been testing with the pst console, to try and weed out the bugs before I get to the point where I try and have the firmware I wrote, communicate with the PC software that I wrote. I can control the longest length of the update command I will send, so that is good and gives me the ability to size the string (byte array) properly.
My project is an automotive logging device that samples some analog and some digital signals. It will have the ability to display and log the values to an excel spreadsheet, as well as control some outputs, most of them being on/off, but a few might be pwm. Even better, almost all my data will be sent from the propeller to the PC, only a small amount of data with the amount of times being few and far in between, will be transmitted to the propeller to update a certain variables value. I believe I can get away with the receiving and transmitting of the data on the same cog. If I were to take a guess, I would estimate that I will be less than (probably far less than) 5000 bytes per second. It will probably be 50 different strings of about 8 bytes per piece. I haven't decided if the refresh rate is going to be 10 or 25 times per second yet.
Now I understand what I did wrong, I did not ask for the first/second/third byte of strAddr, I instead asked for the first/second/third long of strAddr, of which anything after the first did not exist! I get it! . I hope to be able to write a tutorial soon based on what you two in this thread (and Stefan) have taught me about this topic. I would guess I'm not the only one quite confused with strings in spin, that has a high level language background like I do (c sharp).
Thanks again for all of the help and time spent on educating me on this.