What was really interesting when migrating to EEWORDS was coming up with a simple indexing/hashing method that would return with a 6-bit index rather than the 7-bit I used in SDWORDS. Previously I used part of the count and the first character to form an index to tell the search system which "sector" to look in where matching words were stored (reducing the search dramatically) and this worked quite well but I wasn't sure it would distribute as well with only half the number of sectors, which it didn't once the dictionary grew too much. So instead I used part of the count and instead of the first character I simply added up all the characters together and used the lower 4-bits along with 2 bits from the count to form an index.
How well does that work? Well on a full dictionary I can see that there are no sectors that even use up half of their 512 bytes so that means it's quite possible to reduce the eeprom sector size down to 256. That's only 16kB which is only a little more than the original dictionary in RAM, quite surprising I thought.
BTW, this is without reclaiming private headers, I just leave them all in the dictionary.
Here is the sector dump from eeprom: (some IP config data at $FF00)
What was really interesting when migrating to EEWORDS was coming up with a simple indexing/hashing method that would return with a 6-bit index rather than the 7-bit I used in SDWORDS. Previously I used part of the count and the first character to form an index to tell the search system which "sector" to look in where matching words were stored (reducing the search dramatically) and this worked quite well but I wasn't sure it would distribute as well with only half the number of sectors, which it didn't once the dictionary grew too much. So instead I used part of the count and instead of the first character I simply added up all the characters together and used the lower 4-bits along with 2 bits from the count to form an index.
How well does that work? Well on a full dictionary I can see that there are no sectors that even use up half of their 512 bytes so that means it's quite possible to reduce the eeprom sector size down to 256. That's only 16kB which is only a little more than the original dictionary in RAM, quite surprising I thought.
BTW, this is without reclaiming private headers, I just leave them all in the dictionary.
Here is the sector dump from eeprom: (some IP config data at $FF00)
Peter,
Here is my routine to program the EEPROM (lower or upper, or in fact anywhere on a page boundary) from my PropOS.
Should be easy for you to modify to what you require.
The forum is broken so here is the source code pasted. It uses the I2C driver - do you require that too?
'' +--------------------------------------------------------------------------+
'' | Cluso's Propeller Operating System - OS module: _EEPROM --> _EEPROM.CMD |
'' +--------------------------------------------------------------------------+
'' | Authors: (c)2012,2013 "Cluso99" (Ray Rodrick) |
'' | Modifications: |
'' | License: MIT License - See end of file for terms of use |
'' +--------------------------------------------------------------------------+
'' Each module is called from, and returns to, the prop binary "_CMD.CMD"
'' This OS module uses an included module "__MODULE.SPIN" to perform most of
'' the housekeeping - see this for history, details and acknowledgements.
''
'' This module......
'' Copies an SD File "<filename>.EEP" to the EEPROM (lower or upper 32KB of 64KB)
''
'' RR20130724 start
''
''
'#include "__MODULE.spin" ' include the common code for OS modules
#include "__MODULE.spin"
CON
PAGESIZE = 32 'eeprom pagesize
MAXPGMSIZE = 16*1024 'max program size to be placed into eeprom !!!
OBJ
i2c : "Basic_I2C_Driver" ' eeprom routines
VAR
byte filename0[13] ' FAT source filename (string)
byte fname0[13] ' filename w/o dot
byte param[13] ' options: -WL, -WU
long e_offset ' eeprom offset
long hardware ' hw def & pins
long DO, CLK, DI, CS, SI, SO
byte Mounted
long ioControl[2] '\\
long i0Control_sector '// sdspiFemto passes ioControl[3] ==> sd sector no
long fbase ' SD first sector
long fsize ' SD filesize
'--------------- buffer for eeprom 16KB!!! --------------------------
long ndx ' index of chars in pgmbuff 0..16K-1
byte pgmbuff[MAXPGMSIZE] ' buffer for program to be copied to eeprom (16KB max!!!)
DAT
_ModuleStr byte "=== COPY ===",0 'module's name (string)
PRI executeCommand(stringPointer) | sector[512], errorString, bytes, n, r
'get the parameters
str.stringCopy(@filename0, str.tokenizeString(0)) ' source filename
str.stringCopy(@param, str.tokenizeString(0)) ' options
if strsize(@filename0) == 0 ' if no parameters, display usage
printStringCR(string(" Program an SD file to lower/upper EEPROM..."))
printStringCR(string(" PROGRAM <source_filename> [-WL][-WU]"))
return 0
'------------------------------------------------------------------------------
' ensure file exists and is 32KB
errorString := \fat.openfile(@filename0, "R") ' open source file
checkError(@filename0, fat.partitionError, errorString) ' if error, display & abort
fsize := fat.fileSize ' get size
if fsize <> 32768
printString(string("t) File size "))
printString(str.integerToDecimal(fsize,8))
printStringCR(string(" - must be 32KB"))
crlf
abort -2
' validateParams ' validate the params
param[1] := param[1] & $DF ' simple convert to uppercase
param[2] := param[2] & $DF ' simple convert to uppercase
if ((param[0] == "-") and (param[1] == "W") and (param[2] == "L"))
e_offset := $0000
elseif ((param[0] == "-") and (param[1] == "W") and (param[2] == "U"))
e_offset := $8000
else
checkError(string(" parameter error"), $F0, @param) ' if error, display & abort
'------------------------------------------------------------------------------
printString(string("i) Program "))
printString(@filename0)
printString(string(" to EEPROM "))
if e_offset == 0
printStringCR(string("Lower 32KB"))
else
printStringCR(string("Upper 32KB"))
'------------------------------------------------------------------------------
r := \Program32KB(@filename0) ' get file & program to eeprom
if r == 0
printString(string(" Eeprom programmed successfully"))
crlf
else
printString(string("*** Programming Failed *** code "))
printString(str.integerToDecimal(r,3))
crlf
abort -1
'-----------------------------------------------------------------
' Close up
fat.closeFile
printStringCR(string("i) EEPROM written."))
return 0
'------------------------------------------------------------------------------
Pri Program32KB(st) | r
' Program the EEPROM from file
{{
bytes := \fat.readData(@sector, 64) ' if good, returns the byte count
checkError(@filename0, fat.partitionError, bytes) ' if error, display & abort
}}
r := FindFile(st) ' eeprom file present?
' program eeprom lower 16KB
r := LoadFile16KB ' load lower 16KB of file
DisplayBuffer(e_offset + $0000)
printString(string(" Programming first 16KB block..."))
printString(str.integerToHexadecimal(ndx,8))
crlf
r := ProgramEeprom(e_offset + $0000) ' program lower eeprom from buffer
printString(string(" Programmed first 16KB block"))
crlf
bytefill(@pgmbuff, $00, MAXPGMSIZE) ' clear the buffer
r := LoadEeprom(e_offset + $0000) ' readback lower eeprom
DisplayBuffer(e_offset + $0000)
' verify eeprom_lo
r := FindFile(st) ' eeprom file
printString(string(" Verifying first 16KB block..."))
crlf
r := VerifyFile16KB(e_offset + $0000) ' verify lower 16KB of file
printString(string(" Verified first 16KB block"))
crlf
' program eeprom upper 16KB (file already points to start of second 16KB)
' bytefill(@pgmbuff, $00, MAXPGMSIZE) ' clear the buffer
r := LoadFile16KB ' load upper 16KB of file
DisplayBuffer(e_offset + $4000)
printString(string(" Programming second 16KB block..."))
printString(str.integerToHexadecimal(ndx,8))
crlf
r := ProgramEeprom(e_offset + $4000) ' program upper eeprom from buffer
printString(string(" Programmed second 16KB block"))
crlf
'now we need to reposition the file at the upper 16KB block
r := FindFile(st) ' eeprom file
r := LoadFile16KB ' load(skip) lower 16KB of file
bytefill(@pgmbuff, $00, MAXPGMSIZE) ' clear the buffer
printString(string(" Verifying second 16KB block..."))
crlf
r := LoadEeprom(e_offset + $4000) ' readback second 16KB from eeprom
DisplayBuffer(e_offset + $4000)
' verify eeprom_hi
r := VerifyFile16KB(e_offset + $4000) ' verify 16KB of file
printString(string(" Verified second 16KB block"))
crlf
PRI FindFile (st) | errorString, r
printString(string(" Locating "))
printStringCR(st)
' r := fsrw.popen(st,"r") 'Open file for read
' Open the FAT source file
errorString := \fat.openfile(st, "R") ' open source file
checkError(st, fat.partitionError, errorString) ' if error, display & abort
Pri LoadFile16KB | r
' Load a 16KB block from file
ndx~
repeat while ndx < MAXPGMSIZE
' r := fsrw.pgetc ' read a byte at a time from the file
r := fat.readByte
if r < 0
printString(string("t) File error"))
crlf
abort -3
else
pgmbuff[ndx++] := r ' save bytes to buffer
return 0
Pri VerifyFile16KB (offset) | r
' Verify 16KB block of file
ndx~
repeat while ndx < MAXPGMSIZE
' r := fsrw.pgetc ' read a byte at a time from the file
r := fat.readByte
if r < 0
printString(string("t) File error"))
crlf
abort -3
else
if r <> pgmbuff[ndx] ' compare bytes
printString(string("t) Verify Error at $"))
printString(str.integerToHexadecimal(ndx+offset,4))
crlf
abort -4
ndx++
return 0
Pri ProgramEeprom (offset) | i, start_time
'program 16KB to eeprom
repeat i from 0 to constant(MAXPGMSIZE/PAGESIZE)-1
if i2c.WritePage(i2c#BootPin, i2c#EEPROM, offset+(i*PAGESIZE), @pgmbuff[i*PAGESIZE], PAGESIZE)
abort -1 'error during write
start_time := cnt 'prepare to check for a timeout
repeat while i2c.WriteWait(i2c#BootPin, i2c#EEPROM, offset+(i*PAGESIZE))
if cnt - start_time > clkfreq / 10
abort -2 'waited more than 100ms
Pri LoadEeprom (offset) | i
'readback 16KB only
repeat i from 0 to constant(MAXPGMSIZE/PAGESIZE)-1
if i2c.ReadPage(i2c#BootPin, i2c#EEPROM, offset+(i*PAGESIZE), @pgmbuff[i*PAGESIZE], PAGESIZE)
abort -1 'error during read
PRI DisplayBuffer (offset) | r, c, lc[16], w, adx
'display first 16KB
w := 16
adx~
ndx~
' repeat while ndx < MAXPGMSIZE
' repeat while ndx < (MAXPGMSIZE / 16) '<==== only show 1KB
repeat while ndx < (MAXPGMSIZE / 64) '<==== only show 0.25KB
' printChar(spc)
printString(str.integerToHexadecimal((adx + offset),4))
printString(string(":"))
c := 0
repeat while c < w
r := pgmbuff[ndx++] ' get next char
printChar(" ")
printString(str.integerToHexadecimal(r,2))
lc[c] := r
c++
c := 0
printChar(" ")
repeat while c < w
r := lc[c]
if r < $20 or r > $7E
printChar(".")
else
printChar(r)
c++
adx += w
crlf
dat
{{
+------------------------------------------------------------------------------------------------------------------------------+
| TERMS OF USE: MIT License |
+------------------------------------------------------------------------------------------------------------------------------+
|Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation |
|files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, |
|modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software|
|is furnished to do so, subject to the following conditions: |
| |
|The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.|
| |
|THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE |
|WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR |
|COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, |
|ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE. |
+------------------------------------------------------------------------------------------------------------------------------+
}}
Okay the changes are comming fast, do you have 1 2 3 procedure to build the new system on say a quickstart?
Yes, the procedure is fairly straightforward now and MJB also has a TeraTerm script for loading the source files automatically. However this is the procedure I use for a new board.
1. Use Spin tool/BST to load the "Spin" kernel to bring the Prop to life.
2. Connect terminal at 230,400 baud and check coms
3. Load (paste or send) P8.SET to set the build required
4. Load EXTEND.FTH
5. Load EEWORDS.FTH (may be included in EXTEND later)
6. Load P8DEFS.FTH to define the P8 hardware (like a header file)
7. Load SDFS (combo of SDCARD and EASYFILE)
8. Run COMPACT
9. Load NET5500.FTH (combo of W5500 plus EASYNET)
10. Optional COMPACT which leaves around 7K free
Quickstart is even easier as there is no SD or Ethernet etc so really it only needs steps 1,2, and 4 and then QS.FTH optionally.
So the serial load could be reduced to P8EXTEND.FTH (combo of P8.SET + EXTEND + EEWORDS + P8DEFS) then SDFS then NET5500. In fact I load the networking source files straight from the SD card since SDFS is in place.
For production I only have to build once and then SAVEROM and use that 64K binary file from my programming Prop which directly loads the EEPROM of the target Prop, so each Prop only takes a couple of seconds to program.
Peter,
Here is my routine to program the EEPROM (lower or upper, or in fact anywhere on a page boundary) from my PropOS.
Should be easy for you to modify to what you require.
The forum is broken so here is the source code pasted. It uses the I2C driver - do you require that too?
Thanks Ray, I was sort of looking for a PC based loader that did it automatically over serial to make it easy for the user if I update a binary. I use a Prop configured as a programmer stick to clone all my boards quickly so the problem really is how does the end user load a new binary that is more than 32k? My kernel is incorporating a bootloader directly into the serial receive driver so that it transparently loads the EEPROM if it detects a valid hex file being loaded, regardless of what the O/S is doing. Of course certain checks need to be made to prevent corruption of the bootloader itself but that's a minor detail. I figure that if I buffer each line of hex and it's ok then I reprogram EEPROM directly then and there or maybe even accumulate a programming page's worth first. The thing is to prevent the startup code from being reprogrammed until the rest of the file has loaded correctly so that the bootloader never loses control. So this means that the user never has to use the Spin tool and then send a hex file through the terminal as well, just sending the file which can be done even from a smartphone is all that is needed.
If the target system is connected to the Ethernet at present I only have to FTP the binary to it and then execute LOADROM which just programs the whole 64k EEPROM from the FIRMWARE.ROM file. For these networked systems I may even make them automatically search for and apply firmware updates if this option is enabled by the user. It's quite a powerful system that's running, and I've still got 7K of RAM left and over four cogs free!
Yes, the procedure is fairly straightforward now and MJB also has a TeraTerm script for loading the source files automatically.
if anybody is interrested - here is the build script I have been using for TeraTerm.
If you set up propellent you can compile and load the spin kernel automatically as well
To confige TeraTerm you need to create a custom ini file with the correct linedelay and baud rate settings.
Here I have a lot of messageboxes active, which you can comment out, when it works for you.
The created log file shows exactly what has been done.
If you want even more diagnostics comment the [~ word in the fth files.
Current pause timings a very conservative.
btw. I save the Google docs documents locally with the google docs time stamp of last change in the name.
This way I can get reproducable builds and know exactly what is in there.
Very important with Peter's lightning speed of change ;-)
; save file with *.ttl extension (TeraTerm Language)
; Tachyon build file for TeraTerm
;
; Spinneret build file 2014-10-31
;
show 1
clearscreen 1
; line delay setting is important, so we load custom ini file
restoresetup 'SPINNERET.ini'
getdir dir
strconcat dir '\'
messagebox dir 'DIR'
getttdir ttdir
messagebox ttdir 'TTdir'
; sendfile needs full path
; set build directory
; setdir 'D:\Tachyon\Spinneret 2014-10-23\'
lf = dir
gettime timestr "%Y-%m-%d--%H-%M-%S"
strconcat lf 'build '
strconcat lf timestr
strconcat lf '.log'
logopen lf 0 0 0 1 1
; assume a plain Tachyon base image
; sendln 'COLD'
; pause 20
; or load kernel via propellent
dispstr 'loading Kernel'
disconnect 0
pause 1
execstr = 'D:\Propeller\Propellent\Propellent.exe /eeprom '
strconcat execstr #$22
strconcat execstr dir
strconcat execstr 'TACHYON V24 2014-12-17 05-58.spin'#$22
messagebox execstr 'Load Tachyon'
exec execstr
pause 50
; connect to COM5
connect '/C=5'
dispstr 'done loading Kernel'
messagebox 'done loading Kernel' 'done load'
pause 1
sendbreak
pause 10
fn = dir
strconcat fn 'EXT_LOAD 2014-10-27 07-28.fth'
messagebox fn 'loading'
sendfile fn 1
pause 20
fn = dir
strconcat fn 'EXTEND 2014-12-17 15-54.fth'
messagebox fn 'loading'
sendfile fn 1
pause 20
; for debug we want to see all
; sendln 'pub [~ ; '
fn = dir
;
;strconcat fn 'EPRINT 2014-07-03 18-13.fth'
;messagebox fn 'loading'
;sendfile fn 1
;pause 10
fn = dir
strconcat fn 'SPINNERET 2013-12-04 12-00.fth'
messagebox fn 'loading'
sendfile fn 1
pause 10
sendln '?BACKUP'
pause 20
fn = dir
strconcat fn 'SDCARD 2014-12-09 03-44.fth'
messagebox fn 'loading'
sendfile fn 1
pause 20
fn = dir
strconcat fn 'EASYFILE 2014-12-11 01-54.fth'
messagebox fn 'loading'
sendfile fn 1
pause 20
fn = dir
strconcat fn 'SDWORDS 2014-12-10 06-55.fth'
messagebox fn 'loading'
sendfile fn 1
pause 30
sendln '?BACKUP'
pause 30
sendln 'COMPACT'
pause 50
sendln '?BACKUP'
pause 30
sendln 'SDWORDS '
; end
pause 30
fn = dir
strconcat fn 'W5100 2014-11-25 00-52.fth'
messagebox fn 'loading'
sendfile fn 1
pause 20
sendln '?BACKUP'
pause 30
fn = dir
yesnobox 'load EASYNET' 'EASYNET'
if result then
strconcat fn 'EASYNET 2014-12-16 02-13.fth'
else
yesnobox 'load MJBHTTP' 'MJBHTTP'
if result then
strconcat fn 'MJBHTTP 14.fth'
else
end
endif
endif
messagebox fn 'loading'
sendfile fn 1
pause 20
sendln 'COMPACT'
pause 30
sendln '?BACKUP'
pause 30
sendbreak
pause 10
sendln 'SDWORDS '
pause 30
logclose
' That's all, but to sweep in 500Hz spacings from 37kHz to 42kHz every 50 ms perhaps try this:
15 APIN BEGIN 37,000 5,000 ADO I HZ 50 ms 500 +LOOP AGAIN
' To lock that into the system so it does it on startup is straightforward:
pub JAMMER 15 APIN BEGIN 37000 5000 ADO I HZ 50 ms 500 +LOOP AGAIN ;
AUTORUN JAMMER
But shouldn't it be this?
#15 APIN BEGIN #42,000 #37,000 ADO I HZ #50 ms #500 +LOOP AGAIN
' To lock that into the system so it does it on startup is straightforward:
pub JAMMER #15 APIN BEGIN #42,000 #37000 ADO I HZ #50 ms #500 +LOOP AGAIN ;
AUTORUN JAMMER
#s added just for clarification is system is not set to decimal but the ADO bounds
' That's all, but to sweep in 500Hz spacings from 37kHz to 42kHz every 50 ms perhaps try this:
15 APIN BEGIN 37,000 5,000 ADO I HZ 50 ms 500 +LOOP AGAIN
' To lock that into the system so it does it on startup is straightforward:
pub JAMMER 15 APIN BEGIN 37000 5000 ADO I HZ 50 ms 500 +LOOP AGAIN ;
AUTORUN JAMMER
But shouldn't it be this?
#15 APIN BEGIN #42,000 #37,000 ADO I HZ #50 ms #500 +LOOP AGAIN
' To lock that into the system so it does it on startup is straightforward:
pub JAMMER #15 APIN BEGIN #42,000 #37000 ADO I HZ #50 ms #500 +LOOP AGAIN ;
AUTORUN JAMMER
#s added just for clarification is system is not set to decimal but the ADO bounds
don't know what you mean ...
from Tachyon.spin
' ADO ( from cnt -- )
...
' DO ( to from -- )
...
and number base default is decimal now, but good to prefix anyhow.
and number base default is decimal now, but good to prefix anyhow.
you are correct, should have check the source!
but from 32500 for 5000 times incrementing by 500 each time we end up with 2,500,000 + 32500 = 2,825,000?
so wouldn't you loop 140 times to increment 32500 to 42500 by increments of 500?
you are correct, should have check the source!
but from 32500 for 5000 times incrementing by 500 each time we end up with 2,500,000 + 32500 = 2,825,000?
so wouldn't you loop 140 times to increment 32500 to 42500 by increments of 500?
? You are confusing me David!
I left the # prefix off just to simplify it as I didn't want to clutter the simple straightforward thing that we know it is but for some reason people are afraid of
So 37,000 5,000 ADO will work the same as 42,000 37,000 DO and if we just used LOOP it would step 1 Hz each time for 5,000 times. Using 500 +LOOP means it jumps 500 at a time so the index would be 37,000 then 37,500 then 38,000 right up to 42,000 when +LOOP detects that boundary of 42,000 has been reached or crossed so it terminates the loop.
ADO grew out a distatse for those traditional Forth DO LOOP parameters and the use of BOUNDS at times to convert to that format. It was more efficient and looked cleaner when I combined the BOUNDS DO into a simple clean operation. I didn't know what to name it at the time and didn't want to make much "ado" about such a little thing so it was named ADO and it stuck!
you are correct, should have check the source!
but from 32500 for 5000 times incrementing by 500 each time we end up with 2,500,000 + 32500 = 2,825,000?
so wouldn't you loop 140 times to increment 32500 to 42500 by increments of 500?
15 APIN BEGIN 37,000 5,000 ADO I HZ 50 ms 500 +LOOP AGAIN
the loop is taken only 10 times
start is 37000
end is start + 5000
step is 500
I left the # prefix off just to simplify it as I didn't want to clutter the simple straightforward thing that we know it is but for some reason people are afraid of
So 37,000 5,000 ADO will work the same as 42,000 37,000 DO and if we just used LOOP it would step 1 Hz each time for 5,000 times. Using 500 +LOOP means it jumps 500 at a time so the index would be 37,000 then 37,500 then 38,000 right up to 42,000 when +LOOP detects that boundary of 42,000 has been reached or crossed so it terminates the loop.
ADO grew out a distatse for those traditional Forth DO LOOP parameters and the use of BOUNDS at times to convert to that format. It was more efficient and looked cleaner when I combined the BOUNDS DO into a simple clean operation. I didn't know what to name it at the time and didn't want to make much "ado" about such a little thing so it was named ADO and it stuck!
woah how does the syntax below know it crossed 42,000?
pub JAMMER 15 APIN BEGIN 37000 5000 ADO I HZ 50 ms 500 +LOOP AGAIN ;
woah how does the syntax below know it crossed 42,000?
pub JAMMER 15 APIN BEGIN 37000 5000 ADO I HZ 50 ms 500 +LOOP AGAIN ;
completely understand DO
with Tachyon reading the code helps in such cases ... ;-)
' ADO = BOUNDS DO - just a quick and direct way as BOUNDS is most often never used elsewhere
' ADO ( from cnt -- )
ADO mov X,tos+1
add tos+1,tos
mov tos,X
' DO ( to from -- )
DO call #_PUSHLP ' PUSH index onto loop stack
the params are simply converted and then DO is used
woah how does the syntax below know it crossed 42,000?
pub JAMMER 15 APIN BEGIN 37000 5000 ADO I HZ 50 ms 500 +LOOP AGAIN ;
completely understand DO
Forth <> syntax
Have a look at LOOP an +LOOP in the kernel source, it detects it crossing each time it executes a loop. That's also why you can manipulate the loop stack parameters with words such as LEAVE which will simply set the index to the value of the limit, when LOOP comes around it checks and says "I'm done for the day" and exits the loop.
' (+loop) ( n1 -- ) adds n1 to the loop index and branches back if not equal to the loop limit
PLOOP jmpret POPX_ret,pDELTA wc ' DELTA calls POPX (code cramming)
' The comparison above is between the call insn (wr) at DELTA and the jump insn (nr) at POPX_ret,
' this will alwaye be carry set. The call itself is indirect.
' 400ns execution time including bytecode read and execute
LOOP if_nc mov X,#1 ' default loop increment of 1
add loopstk+1,X ' increment index
cmps loopstk,loopstk+1 wz,wc
BRANCH if_a mov IP,branchstk ' Branch to the address that is saved in branch
if_a jmp unext
jmpret LPOPX_ret,forNEXT+1 wc ' discard top of loop index stack
' then next loop and its branch address
Have a look at LOOP an +LOOP in the kernel source, it detects it crossing each time it executes a loop. That's also why you can manipulate the loop stack parameters with words such as LEAVE which will simply set the index to the value of the limit, when LOOP comes around it checks and says "I'm done for the day" and exits the loop.
' (+loop) ( n1 -- ) adds n1 to the loop index and branches back if not equal to the loop limit
PLOOP jmpret POPX_ret,pDELTA wc ' DELTA calls POPX (code cramming)
' The comparison above is between the call insn (wr) at DELTA and the jump insn (nr) at POPX_ret,
' this will alwaye be carry set. The call itself is indirect.
' 400ns execution time including bytecode read and execute
LOOP if_nc mov X,#1 ' default loop increment of 1
add loopstk+1,X ' increment index
cmps loopstk,loopstk+1 wz,wc
BRANCH if_a mov IP,branchstk ' Branch to the address that is saved in branch
if_a jmp unext
jmpret LPOPX_ret,forNEXT+1 wc ' discard top of loop index stack
' then next loop and its branch address
Got it, thanks for cheer leading me over the learning hump.
Since I developed the compact dictionary in upper EEPROM I haven't had a chance to go back and do new binaries for the Spinneret as most of my work has concentrated on the IoT5500 modules. But the good thing is that all the files are the same except for the Spinneret header file SPINNERET.FTH and of course using the W5100.FTH driver.
To make this a little easier to load the full 64k image I have created a spinneret.binary image which has the SD filesystem but does not have the networking so that it fits in 32K which can be loaded onto the Spinneret with your favorite Prop loader tool.
(make sure you copy the FIRMWARE.ROM file to your Spinneret's microSD)
Once you fire up the Spinneret you can connect your serial terminal at 230,400 and talk to the Forth shell and ask it to load the full 64k networking image into EEPROM by typing LOADROM, then reboot (either REBOOT or ^C or <break>). Once rebooted the Spinneret will automatically launch the network servers for WEB, FTP, and Telnet while still providing an interactive shell over the serial console. You can easily change the IP addresses and even the MAC which BTW is a random 32-bit number on a cold start (new and blank) with a fixed OUI of $02.FF. Here is the response to an ifconfig command (Linux equivalent of ipconfig):
[B]ifconfig[/B]
************ NETWORK STATUS ************
HARDWARE: SPINNERET using WIZnet W5100 (indirect)
SRC IP 192.168.016.150.
MASK 255.255.255.000.
GATEWAY 192.168.016.001.
MAC 02.FF.42.C5.F3.F6.
SKT HH:MM:SS MODE PORT DEST TXRD TXWR RXRD RXWR RXSZ IR STATUS IP ADDR
#0 15:30:23 TCP 21 54923 . . . . . 00 14 LISTEN
#1 15:30:23 TCP 80 28777 . . . . . 00 14 LISTEN
#2 15:30:23 TCP 10001 . . . . . 00 14 LISTEN
#3 15:30:23 IPRW 511 43024 . .2800. . . A8 10
ok
You can change the IP address via the terminal by typing the IP address in using a & prefix to force the number to "decimal bytes", that is, each group of decimal digits represent one byte.
&192.168.0.14 SIP
To change the gateway:
&192.168.0.254 GATEWAY
To change the port number of the socket which btw is locked in software to a server, so socket 2 is locked to Telnet which actually connects to the Forth shell:
2 SOCKET 23 PORT!
This changes the Telnet port to the standard port 23
All these changes are automatically backed up in the topmost part of the 64k EEPROM and reloaded at network boot.
The Spinneret is online at the moment at tachyonforth.com but you can play with it a bit more using FTP and Chrome at ftp://tachyonforth.com or Telnet to port 10001
(EDIT: looks like my socket reporting is off a bit for some reason on the latest W5100 driver but it everything seems to work, I will fix it up shortly) - FIXED - updated Dropbox
I haven't been able to use Firefox for some time now to ftp to my Tachyon network servers. What happens when it tries to connect is that FIrefox comes up with an alert "200 Type is now I" which is actually the correct response message for a request. However here is a quick text dump from wireshark to see how this became an alert:
No. Time Source Destination Protocol Length Info
21 34.225855000 192.168.16.2 192.168.16.150 FTP 59 Request: PWD
22 34.225875000 192.168.16.2 192.168.16.150 FTP 62 Request: TYPE I
23 34.238073000 192.168.16.150 192.168.16.2 FTP 88 Response: 257 "/" is your current location
24 34.238266000 192.168.16.2 192.168.16.150 FTP 60 Request: PASV
25 34.249351000 192.168.16.150 192.168.16.2 FTP 73 Response: 200 TYPE is now I
26 34.271760000 192.168.16.150 192.168.16.2 FTP 114 Response: 227 Entering Passive Mode with port (192,168,16,150,176,5)
27 34.271854000 192.168.16.2 192.168.16.150 TCP 54 49121 > ftp [RST, ACK] Seq=74 Ack=302 Win=29200 Len=0
So for some reason FF is sending two requests together (PWD and TYPE I) and when it receives a response for the first request it shoots off a request for PASV while the second response is still on it's way. So when PASV looks for a response it gets the previous response that it hadn't read.
I've tried this with a local NAS and FF doesn't try to send two requests together as perhaps the FEAT response cause it to take a different path. It seems like a FF bug to me but I can't see any mention of it anywhere and although it should be possible for me to process all receive packets and combine the responses before sending it just seems like a kludge to me when all other clients behave.
I haven't been able to use Firefox for some time now to ftp to my Tachyon network servers. What happens when it tries to connect is that FIrefox comes up with an alert "200 Type is now I" which is actually the correct response message for a request.
Any ideas anyone?
I just tried to connect using Firefox and got a popup window with the same error.
I just tried to connect using Firefox and got a popup window with the same error.
I've tested this on Windows, Android, and Linux, same thing. Yet I can connect to a local NAS server and it doesn't try to send TYPE I before it receives the response for PWD:
No. Time Source Destination Protocol Length Info
22 14.091912000 192.168.16.2 192.168.16.19 FTP 71 Request: PWD
23 14.092189000 192.168.16.19 192.168.16.2 FTP 100 Response: 257 "/" is your current location
24 14.092415000 192.168.16.2 192.168.16.19 FTP 74 Request: TYPE I
25 14.092667000 192.168.16.19 192.168.16.2 FTP 96 Response: 200 TYPE is now 8-bit binary
26 14.092810000 192.168.16.2 192.168.16.19 FTP 72 Request: PASV
27 14.093258000 192.168.16.19 192.168.16.2 FTP 116 Response: 227 Entering Passive Mode (192,168,16,19,62,227)
the handling of the FEAT OPTS command seems not implemented yet.
so maybe better go with the minimum
211 no Features supported
reply.
fixed in EASYNET.fth
quick fix: just paste the above code into your terminal window. The next FTP request will automatically use the new code.
No rebuild required. then do a BACKUP if you want it permanently in the image.
better even use the following code for now
pub FEAT IMMEDIATE
ON LANLED
autosend C~
PRINT" 211 no Features supported" CR
autosend C~~ CR
;
the handling of the FEAT OPTS command seems not implemented yet.
so maybe better go with the minimum
211 no Features supported
reply.
fixed in EASYNET.fth
quick fix: just paste the above code into your terminal window. The next FTP request will automatically use the new code.
No rebuild required. then do a BACKUP if you want it permanently in the image.
better even use the following code for now
pub FEAT IMMEDIATE
ON LANLED
autosend C~
PRINT" 211 no Features supported" CR
autosend C~~ CR
;
Thanks for tracking that one down MJB, it's so sweet not having to sweat over it and still having the problem solved
So there is still a bug with FF as Chrome and other FTP clients work anyway but I will look at the FEAT response in more detail later.
BTW, the code for FEAT can be even simpler, and yes, we only need to paste this code through the console for it to work although I did reload EASYNET as that only took several seconds anyway.
pub FEAT
ON LANLED
PRINT" 211 no Features supported" CR
;
the handling of the FEAT OPTS command seems not implemented yet.
so maybe better go with the minimum
211 no Features supported
reply.
fixed in EASYNET.fth
quick fix: just paste the above code into your terminal window. The next FTP request will automatically use the new code.
No rebuild required. then do a BACKUP if you want it permanently in the image.
better even use the following code for now
pub FEAT IMMEDIATE
ON LANLED
autosend C~
PRINT" 211 no Features supported" CR
autosend C~~ CR
;
1. Use Spin tool/BST to load the "Spin" kernel to bring the Prop to life.
2. Connect terminal at 230,400 baud and check coms 3. Load (paste or send) P8.SET to set the build required
4. Load EXTEND.FTH
5. Load EEWORDS.FTH (may be included in EXTEND later)
6. Load P8DEFS.FTH to define the P8 hardware (like a header file)
7. Load SDFS (combo of SDCARD and EASYFILE)
8. Run COMPACT
9. Load NET5500.FTH (combo of W5500 plus EASYNET)
10. Optional COMPACT which leaves around 7K free
Renaming these to P8SET.FTH so that extensions are recognized by various clients.
BTW, decided that the HELP: word really needs to be in the form {HELP ............. } as I can insert this into the kernel document to keep the Spin compiler happy, even though I'm not using Spin of course. So this looks like this:
{HELP UM* ( u1 u2 -- u1*u2L u1*u2H )
DESC: unsigned 32bit * 32bit multiply -- 64bit result
TIME: 1..11.8us
}
UMMUL mov R0,tos+1
min R0,tos ' max(tos, tos+1)
mov R2,tos+1
max R2,tos ' min(tos, tos+1)
mov R1,#0
mov tos,#0 ' zero result
mov tos+1,#0
UMMULLP shr R2,#1 wz,wc ' test next bit of u1
if_nc jmp #UMMUL1 ' skip if no bit here
add tos+1,R0 wc ' add in shifted u2
addx tos,R1 ' carry into upper long
UMMUL1 add R0,R0 wc ' shift u2 left
addx R1,R1 ' carry into 64-bits
if_nz jmp #UMMULLP ' exhausted u1?
jmp unext
Which means we can type HELP UM* for the kernel as well as the .FTH files.
As many of you know the bytecode calls in TF are comprised of a single CALL opcode followed by an 8-bit index into a vector table which holds the 16-bit address to the code to call. There are several very good reasons for this and with 4 types of CALLs the system can have up to 1,024 vectors for high-level bytecode words including constants and variables etc. Normally this is more than enough even for a fully loaded SDFS and network system but just to be on the safe side plus MJB needs more to implement his dynamic webpages, I have now fully implemented the CALL16 method which compiles a CALL16 opcode plus the 16-bit address making this a 3-byte call without the need for a vector.
So when a new word is created and TF has run out of spare vectors in the 2K vector table it will compile a header with an attribute that indicates the 2 bytecodes in its header are actually the 16-bit direct address and that to reference this it needs to compile a CALL16 followed by these two bytes. This doesn't make a lot of difference to the code memory consumed because by this time all the important stuff has been compiled and is very compact and any references to words with a CALL16 attribute will automatically compile the correct bytecodes. Of course, all references to opcodes and vectored bytecodes will still compile one or two bytes.
So here is a system with a full EXTEND.fth with all the trimmings along with SDFS and networking, amounting to 1,223 words including 144 opcodes which means there are 1,024 vectored words plus 55 CALL16s. Notice that it has 0 vectors free which is in fact what it had before the EASYNET.fth network server module was loaded. To further test this out I then loaded an application on top of this and it all fits and works.
Do you have to do anything different? NO! You don't even need to force a SMALL EXTEND.fth or use the include filter sets such as P8SET.FTH etc.
wow Peter, this is great news.
Regarding the webserver, I think, that in most webserver applications there is no need for a full PWM32 and some of the other special modules in EXTEND.fth.
So for me it is absolutely fine to selectively load things from EXTEND and have enough space even without the CALL16.
But applications will grow - so good to be prepared.
My JSON writer for example needs 20 words / 278 bytes
I had some instability in my dynamic webserver Spinneret tests so hope to be able to clean this out soon.
As many of you know the bytecode calls in TF are comprised of a single CALL opcode followed by an 8-bit index into a vector table which holds the 16-bit address to the code to call. There are several very good reasons for this and with 4 types of CALLs the system can have up to 1,024 vectors for high-level bytecode words including constants and variables etc. Normally this is more than enough even for a fully loaded SDFS and network system but just to be on the safe side plus MJB needs more to implement his dynamic webpages, I have now fully implemented the CALL16 method which compiles a CALL16 opcode plus the 16-bit address making this a 3-byte call without the need for a vector.
So when a new word is created and TF has run out of spare vectors in the 2K vector table it will compile a header with an attribute that indicates the 2 bytecodes in its header are actually the 16-bit direct address and that to reference this it needs to compile a CALL16 followed by these two bytes. This doesn't make a lot of difference to the code memory consumed because by this time all the important stuff has been compiled and is very compact and any references to words with a CALL16 attribute will automatically compile the correct bytecodes. Of course, all references to opcodes and vectored bytecodes will still compile one or two bytes.
So here is a system with a full EXTEND.fth with all the trimmings along with SDFS and networking, amounting to 1,223 words including 144 opcodes which means there are 1,024 vectored words plus 55 CALL16s. Notice that it has 0 vectors free which is in fact what it had before the EASYNET.fth network server module was loaded. To further test this out I then loaded an application on top of this and it all fits and works.
Do you have to do anything different? NO! You don't even need to force a SMALL EXTEND.fth or use the include filter sets such as P8SET.FTH etc.
Just so I'm clear on this how does COMPACT fit into this? Are the steps for a build different now that CALL16 is alive and well?
My little brain really likes the 1 2 3s as long as I don't run out of fingers I'm happy.
The build still works the same as before except that your code is no longer limited by the number of available vectors, although in practice this takes quite a bit and you may not have gotten anywhere now the limit. What happens is that if you have a fully loaded system with SDFS and NET that before there were only about 80 vectors left over, more than enough for most apps but tight if you perhaps had an assembler loaded which would use a vector for every instruction and mode etc. Now when the system runs out of vectors it will implement the CALL16 method transparently so there is no need to worry about it really. What you won't be able to do with these words is revector them since they don't have a vector, but that's minor.
You can do a full build like this:
F11 the Tachyon kernel
* Load EXTEND.fth (optional CREATE SMALL and/or load an include SET)
* Load EEWORDS.fth
* Load your hardware header such as P8DEFS.fth
* Load SDCARD.fth
* Load EASYNET.fth
At this point it hasn't yet been compacted and I like to save my system to SD with SAVEROM. This makes it easy later on if I want to reload modules as I can easily restore to this point with a LOADROM and reboot without having to start right from the beginning.
* Run COMPACT which will now convert the dictionary in RAM to an indexed list in upper EEPROM and release the RAM space.
* Load W5500.fth or other WIZnet driver
* Load EASYNET.fth after which the system will BACKUP
At this point then the dictionary for the network modules is still in RAM but the earlier stuff has been compacted into the EEPROM. If you don't need a lot more room you can leave it like this or else do an optional COMPACT again which will add the new words to the existing dictionary in EEPROM and release the memory it used in RAM.
* Optional COMPACT if desired.
So out of a total of 32K RAM the system does really well by making practically all of this available for code space with the remainder taken up with various buffers such as 2K for four file buffers etc. There's also no need to worry about reclaiming headers because they don't take up any RAM any more and neither is the dictionary searching speed impacted any appreciable amount.
I'm fixing up WORDS and some other functions now so that they work the same with the EEPROM dictionary as they do with RAM.
is V2.4 YY=15 MM=01 DD=04. Putting it in this format is logical as a new revision/version number will always be a larger number than an old revision as is the YYMMDD which also avoids that American MMDDYY or English DDMMYY confusion thing, both of which are not in logical order especially when compared to HHMM.
Comments
How well does that work? Well on a full dictionary I can see that there are no sectors that even use up half of their 512 bytes so that means it's quite possible to reduce the eeprom sector size down to 256. That's only 16kB which is only a little more than the original dictionary in RAM, quite surprising I thought.
BTW, this is without reclaiming private headers, I just leave them all in the dictionary.
Here is the sector dump from eeprom: (some IP config data at $FF00)
Okay the changes are comming fast, do you have 1 2 3 procedure to build the new system on say a quickstart?
Here is my routine to program the EEPROM (lower or upper, or in fact anywhere on a page boundary) from my PropOS.
Should be easy for you to modify to what you require.
The forum is broken so here is the source code pasted. It uses the I2C driver - do you require that too?
Yes, the procedure is fairly straightforward now and MJB also has a TeraTerm script for loading the source files automatically. However this is the procedure I use for a new board.
1. Use Spin tool/BST to load the "Spin" kernel to bring the Prop to life.
2. Connect terminal at 230,400 baud and check coms
3. Load (paste or send) P8.SET to set the build required
4. Load EXTEND.FTH
5. Load EEWORDS.FTH (may be included in EXTEND later)
6. Load P8DEFS.FTH to define the P8 hardware (like a header file)
7. Load SDFS (combo of SDCARD and EASYFILE)
8. Run COMPACT
9. Load NET5500.FTH (combo of W5500 plus EASYNET)
10. Optional COMPACT which leaves around 7K free
Quickstart is even easier as there is no SD or Ethernet etc so really it only needs steps 1,2, and 4 and then QS.FTH optionally.
So the serial load could be reduced to P8EXTEND.FTH (combo of P8.SET + EXTEND + EEWORDS + P8DEFS) then SDFS then NET5500. In fact I load the networking source files straight from the SD card since SDFS is in place.
For production I only have to build once and then SAVEROM and use that 64K binary file from my programming Prop which directly loads the EEPROM of the target Prop, so each Prop only takes a couple of seconds to program.
Thanks Ray, I was sort of looking for a PC based loader that did it automatically over serial to make it easy for the user if I update a binary. I use a Prop configured as a programmer stick to clone all my boards quickly so the problem really is how does the end user load a new binary that is more than 32k? My kernel is incorporating a bootloader directly into the serial receive driver so that it transparently loads the EEPROM if it detects a valid hex file being loaded, regardless of what the O/S is doing. Of course certain checks need to be made to prevent corruption of the bootloader itself but that's a minor detail. I figure that if I buffer each line of hex and it's ok then I reprogram EEPROM directly then and there or maybe even accumulate a programming page's worth first. The thing is to prevent the startup code from being reprogrammed until the rest of the file has loaded correctly so that the bootloader never loses control. So this means that the user never has to use the Spin tool and then send a hex file through the terminal as well, just sending the file which can be done even from a smartphone is all that is needed.
If the target system is connected to the Ethernet at present I only have to FTP the binary to it and then execute LOADROM which just programs the whole 64k EEPROM from the FIRMWARE.ROM file. For these networked systems I may even make them automatically search for and apply firmware updates if this option is enabled by the user. It's quite a powerful system that's running, and I've still got 7K of RAM left and over four cogs free!
if anybody is interrested - here is the build script I have been using for TeraTerm.
If you set up propellent you can compile and load the spin kernel automatically as well
To confige TeraTerm you need to create a custom ini file with the correct linedelay and baud rate settings.
Here I have a lot of messageboxes active, which you can comment out, when it works for you.
The created log file shows exactly what has been done.
If you want even more diagnostics comment the [~ word in the fth files.
Current pause timings a very conservative.
btw. I save the Google docs documents locally with the google docs time stamp of last change in the name.
This way I can get reproducable builds and know exactly what is in there.
Very important with Peter's lightning speed of change ;-)
But shouldn't it be this?
#s added just for clarification is system is not set to decimal but the ADO bounds
from Tachyon.spin and number base default is decimal now, but good to prefix anyhow.
you are correct, should have check the source!
but from 32500 for 5000 times incrementing by 500 each time we end up with 2,500,000 + 32500 = 2,825,000?
so wouldn't you loop 140 times to increment 32500 to 42500 by increments of 500?
? You are confusing me David!
I left the # prefix off just to simplify it as I didn't want to clutter the simple straightforward thing that we know it is but for some reason people are afraid of
So 37,000 5,000 ADO will work the same as 42,000 37,000 DO and if we just used LOOP it would step 1 Hz each time for 5,000 times. Using 500 +LOOP means it jumps 500 at a time so the index would be 37,000 then 37,500 then 38,000 right up to 42,000 when +LOOP detects that boundary of 42,000 has been reached or crossed so it terminates the loop.
ADO grew out a distatse for those traditional Forth DO LOOP parameters and the use of BOUNDS at times to convert to that format. It was more efficient and looked cleaner when I combined the BOUNDS DO into a simple clean operation. I didn't know what to name it at the time and didn't want to make much "ado" about such a little thing so it was named ADO and it stuck!
the loop is taken only 10 times
start is 37000
end is start + 5000
step is 500
.... didn't see your post Peter .....
woah how does the syntax below know it crossed 42,000? completely understand DO
with Tachyon reading the code helps in such cases ... ;-) the params are simply converted and then DO is used
Forth <> syntax
Have a look at LOOP an +LOOP in the kernel source, it detects it crossing each time it executes a loop. That's also why you can manipulate the loop stack parameters with words such as LEAVE which will simply set the index to the value of the limit, when LOOP comes around it checks and says "I'm done for the day" and exits the loop.
Got it, thanks for cheer leading me over the learning hump.
To make this a little easier to load the full 64k image I have created a spinneret.binary image which has the SD filesystem but does not have the networking so that it fits in 32K which can be loaded onto the Spinneret with your favorite Prop loader tool.
Look in the Dropbox files under binaries and then Spinneret. The Dropbox link is in my sig.
(make sure you copy the FIRMWARE.ROM file to your Spinneret's microSD)
Once you fire up the Spinneret you can connect your serial terminal at 230,400 and talk to the Forth shell and ask it to load the full 64k networking image into EEPROM by typing LOADROM, then reboot (either REBOOT or ^C or <break>). Once rebooted the Spinneret will automatically launch the network servers for WEB, FTP, and Telnet while still providing an interactive shell over the serial console. You can easily change the IP addresses and even the MAC which BTW is a random 32-bit number on a cold start (new and blank) with a fixed OUI of $02.FF. Here is the response to an ifconfig command (Linux equivalent of ipconfig):
You can change the IP address via the terminal by typing the IP address in using a & prefix to force the number to "decimal bytes", that is, each group of decimal digits represent one byte.
&192.168.0.14 SIP
To change the gateway:
&192.168.0.254 GATEWAY
To change the port number of the socket which btw is locked in software to a server, so socket 2 is locked to Telnet which actually connects to the Forth shell:
2 SOCKET 23 PORT!
This changes the Telnet port to the standard port 23
All these changes are automatically backed up in the topmost part of the 64k EEPROM and reloaded at network boot.
The Spinneret is online at the moment at tachyonforth.com but you can play with it a bit more using FTP and Chrome at ftp://tachyonforth.com or Telnet to port 10001
(EDIT: looks like my socket reporting is off a bit for some reason on the latest W5100 driver but it everything seems to work, I will fix it up shortly) - FIXED - updated Dropbox
So for some reason FF is sending two requests together (PWD and TYPE I) and when it receives a response for the first request it shoots off a request for PASV while the second response is still on it's way. So when PASV looks for a response it gets the previous response that it hadn't read.
I've tried this with a local NAS and FF doesn't try to send two requests together as perhaps the FEAT response cause it to take a different path. It seems like a FF bug to me but I can't see any mention of it anywhere and although it should be possible for me to process all receive packets and combine the responses before sending it just seems like a kludge to me when all other clients behave.
Any ideas anyone?
I just tried to connect using Firefox and got a popup window with the same error.
I've tested this on Windows, Android, and Linux, same thing. Yet I can connect to a local NAS server and it doesn't try to send TYPE I before it receives the response for PWD:
Without features there MUST be a SPACE
with this change Firefox works again.
I think it worked with firefox before you added the UTF8" feature.
see https://tools.ietf.org/html/rfc2389#section-2.1 Chapter 3.2. FEAT Command Responses
the handling of the FEAT OPTS command seems not implemented yet.
so maybe better go with the minimum
211 no Features supported
reply.
fixed in EASYNET.fth
quick fix: just paste the above code into your terminal window. The next FTP request will automatically use the new code.
No rebuild required. then do a BACKUP if you want it permanently in the image.
better even use the following code for now
Thanks for tracking that one down MJB, it's so sweet not having to sweat over it and still having the problem solved
So there is still a bug with FF as Chrome and other FTP clients work anyway but I will look at the FEAT response in more detail later.
BTW, the code for FEAT can be even simpler, and yes, we only need to paste this code through the console for it to work although I did reload EASYNET as that only took several seconds anyway.
Thanks
Sorry, didn't notice the new post, done.
Renaming these to P8SET.FTH so that extensions are recognized by various clients.
BTW, decided that the HELP: word really needs to be in the form {HELP ............. } as I can insert this into the kernel document to keep the Spin compiler happy, even though I'm not using Spin of course. So this looks like this: Which means we can type HELP UM* for the kernel as well as the .FTH files.
So when a new word is created and TF has run out of spare vectors in the 2K vector table it will compile a header with an attribute that indicates the 2 bytecodes in its header are actually the 16-bit direct address and that to reference this it needs to compile a CALL16 followed by these two bytes. This doesn't make a lot of difference to the code memory consumed because by this time all the important stuff has been compiled and is very compact and any references to words with a CALL16 attribute will automatically compile the correct bytecodes. Of course, all references to opcodes and vectored bytecodes will still compile one or two bytes.
So here is a system with a full EXTEND.fth with all the trimmings along with SDFS and networking, amounting to 1,223 words including 144 opcodes which means there are 1,024 vectored words plus 55 CALL16s. Notice that it has 0 vectors free which is in fact what it had before the EASYNET.fth network server module was loaded. To further test this out I then loaded an application on top of this and it all fits and works.
Do you have to do anything different? NO! You don't even need to force a SMALL EXTEND.fth or use the include filter sets such as P8SET.FTH etc.
Regarding the webserver, I think, that in most webserver applications there is no need for a full PWM32 and some of the other special modules in EXTEND.fth.
So for me it is absolutely fine to selectively load things from EXTEND and have enough space even without the CALL16.
But applications will grow - so good to be prepared.
My JSON writer for example needs 20 words / 278 bytes
I had some instability in my dynamic webserver Spinneret tests so hope to be able to clean this out soon.
Just so I'm clear on this how does COMPACT fit into this? Are the steps for a build different now that CALL16 is alive and well?
My little brain really likes the 1 2 3s as long as I don't run out of fingers I'm happy.
The build still works the same as before except that your code is no longer limited by the number of available vectors, although in practice this takes quite a bit and you may not have gotten anywhere now the limit. What happens is that if you have a fully loaded system with SDFS and NET that before there were only about 80 vectors left over, more than enough for most apps but tight if you perhaps had an assembler loaded which would use a vector for every instruction and mode etc. Now when the system runs out of vectors it will implement the CALL16 method transparently so there is no need to worry about it really. What you won't be able to do with these words is revector them since they don't have a vector, but that's minor.
You can do a full build like this:
F11 the Tachyon kernel
* Load EXTEND.fth (optional CREATE SMALL and/or load an include SET)
* Load EEWORDS.fth
* Load your hardware header such as P8DEFS.fth
* Load SDCARD.fth
* Load EASYNET.fth
At this point it hasn't yet been compacted and I like to save my system to SD with SAVEROM. This makes it easy later on if I want to reload modules as I can easily restore to this point with a LOADROM and reboot without having to start right from the beginning.
* Run COMPACT which will now convert the dictionary in RAM to an indexed list in upper EEPROM and release the RAM space.
* Load W5500.fth or other WIZnet driver
* Load EASYNET.fth after which the system will BACKUP
At this point then the dictionary for the network modules is still in RAM but the earlier stuff has been compacted into the EEPROM. If you don't need a lot more room you can leave it like this or else do an optional COMPACT again which will add the new words to the existing dictionary in EEPROM and release the memory it used in RAM.
* Optional COMPACT if desired.
So out of a total of 32K RAM the system does really well by making practically all of this available for code space with the remainder taken up with various buffers such as 2K for four file buffers etc. There's also no need to worry about reclaiming headers because they don't take up any RAM any more and neither is the dictionary searching speed impacted any appreciable amount.
I'm fixing up WORDS and some other functions now so that they work the same with the EEPROM dictionary as they do with RAM.
P.S. How are you going with the IOT modules?
Is the version really a year 2415 Jan 05?
Actually it's a combo of version,revision,year,month,day etc in the form VRYYMMDD,HHMM so that:
VER: Propeller .:.:--TACHYON--:.:. Forth V24150105.2330
is V2.4 YY=15 MM=01 DD=04. Putting it in this format is logical as a new revision/version number will always be a larger number than an old revision as is the YYMMDD which also avoids that American MMDDYY or English DDMMYY confusion thing, both of which are not in logical order especially when compared to HHMM.