Tachyon FTP server
Peter Jakacki
Posts: 10,193
I'm in the process of testing an FTP server written in Tachyon Forth for the Wiznet W5200. I initially was using the Spin driver for this chip but I can see numerous advantages in using Forth, including diagnostics. Also the plain text FTP commands are just passed to the Forth text interpreter for execution such as USER PASS SYST RETR etc
Anyway, I'm testing this as well as web, telnet and serial server ports but I'm just wondering if anyone has written an FTP server for the Prop? I'm wondering if there are any gotchas down the road to look out for.
Anyway, I'm testing this as well as web, telnet and serial server ports but I'm just wondering if anyone has written an FTP server for the Prop? I'm wondering if there are any gotchas down the road to look out for.
[FONT=courier new]WIZ? ************ W5200 STATUS ************ LINK *UP* CHIP VER 0003 SRC IP 192.168.016.150. MASK 255.255.255.000. GATEWAY 192.168.016.001. MAC 00.08.DC.01.02.03. *** SOCKETS *** 0: MODE= TCP PORT#0021 STAT=17 ESTABLISHED 192.168.016.108. 1: MODE= TCP PORT#0023 STAT=14 LISTEN 2: MODE= TCP PORT#0080 STAT=14 LISTEN 3: MODE= TCP PORT#0000 STAT=00 CLOSED 4: MODE= TCP PORT#0000 STAT=00 CLOSED 5: MODE= TCP PORT#0000 STAT=00 CLOSED 6: MODE= TCP PORT#0000 STAT=00 CLOSED 7: MODE= TCP PORT#0000 STAT=00 CLOSED ok [/FONT]
Comments
The WIZNET site doesn't really seem to answer any similar questions that others have asked and in typical Asian manner (admit no wrong) just asks "have you used standard driver?" which of course never answers the question or addresses the problem. Shoot me.
Here is a connection weirdness. BTW, I prefill the transmit buffer with "?" so I can see if it's sending other parts of the buffer. After I type GO it inits the chip etc and runs a server polling routine in the background. Ignore the socket number reporting glitch.
FileZilla capture:
Tachyon terminal capture:
Here in this version I am using a simple character EMIT for the welcome message which writes to the tx buffer and issues a SEND command for each and every character rather than a string SEND. Result is the same for a string then SEND.
Typing LAN MODULES CON into Tachyon after this redirects the console output to the LAN (and back) perfectly without any garbage. Here's the last couple of lines from the Telnet session (on port 21)
Yes, but did you experience any problems with the first send after connection established, it just seems to grab a random chunk of the buffer and also seems to transmit repeatedly. After that it's okay.
This works: (non-optimized method for testing)
This does not work: (disregard wraparound for testing)
So the important thing is to write back the TX_WR register with all it's bits even though we need to mask them off for address calculation.
I thought I'd share some results I've had with writing this FTP driver in Tachyon for the W5200. My driver which includes just as much in the way of diagnostics as driver only takes up 1.7K bytes of code and variable space at present. The FTP+TELNET takes only 1.2K bytes and that's also with a stab at HTTP included.
To keep things simple I run the server from the main console cog (under "KEYPOLL") so that I can still access the I/O normally from the terminal. Even though I am still sorting out a few things with FTP data connections, it mostly seems to be working fine with FileZilla etc. As for TELNET that works a charm and I just have it connecting to the Tachyon console so it's like a remote Tachyon console. The FTP command port is also connected to the console where Forth terminal text is normally entered so that the FTP commands are just another word in the Forth dictionary. PASV for instance selects a data port and opens up a socket which is used by LIST and RETR words etc. I can also type these same words in at the serial terminal to see that they work properly.
So writing this driver was a little bit daunting at first but with Forth it's been a lot of fun as I can interact with the chip and sockets etc live. Can't wait to get into setting up the HTTP server part too.
I think there isn't a lot of interest on Tacheon FTP.
If spin, I think everybody is interest on this thread.
I'm very interest on FTP, but I don't understand little about Tachyon.
There is word reference on Tachyon?
Although I installed Tachyon v2.1, there are not QWORDS,I2CBUS,NEWCNT,.LAP etc.
Anymore file need?
The intro page does talk about loading EXTEND.fth but maybe it hasn't been stated clearly and explicitly in the documents although I have mentioned it many times. The Spin tool compiled kernel could work without EXTEND.fth but there is a lot of support files in there and so this is normally loaded as standard. Set your terminal for 20ms line delay (not char) to be safe and paste or send the file after which it will automatically back it up if there were no errors. You can always issue a break command in your terminal to get TF to reset itself no matter what.
Although I had an opportunity to write this in Spin it would have been a lot harder and more complicated. TF is fast enough that I don't need special PASM for anything but really low level SPI.
I try to load EXTEND.fth.
I copied EXTEND.fth and paste it to TeraTerm.
Cannot load. I attached screenshot.
And Tachyon stopped.
I pushed reset button on protoboard.
Next I send EXTEND.fth by 'Send'menu to TeraTerm.
No loading.
Looks like you haven't set a line delay, no handshaking, The line delay is pretty much for the worst case where a lot might be packed onto a line and there are a lot of them as well. I've used TeraTerm as well so I will double check the settings but they shouldn't be much different. It's a pity that XON/XOFF can't be used but PCs tend to buffer the XON/XOFF rather than process them immediately in which case we could have much faster loading without any delay settings.
At the moment I have little free time so no time to experiment, but I hope to catch up soon..
Massimo
In general, you are right, activity does seem to be down a bit on the forums. Hopefully folks are knee deep in fun Propeller projects and there will be a flurry of new activity (not my excuse).
I am following Tachyon closely. Like others I am "tethered" at the moment with my day job and have little time to spare.
This is certainly another validation of Forth's utility on the Propeller... faster than Spin, with the interactivity that PASM lacks. I've been quite lately as I'd rather spend time thinking that debating pros and cons (we who use Forth know it is worthwhile).
pfth has been my main study resource at this point, but Tachyon offers more exotic sophistications.
I set transmit-delay =20msec/line
Thanks loading is success.
I have questions.
Loading EXTEND.fth, this is saved in eeprom?
If I want to erase this, I use which word?
Typing COLD will restart TF with just the kernel although the EEPROM has not been changed which means if you reboot it will still have everything there. However if you type COLD and then load in EXTEND.fth it will of course end up overwriting the old image in EEPROM.
Sometimes I have various modules loaded and then my app in EEPROM and it's a pain to have to load in all the modules again if I want to reload the app so if I have a 64kB EEPROM I use the word BURNROM (for want of a better name) which I use at the point before loading my app or whatever I'm testing. What this does is simply copy the lower 32kB of EEPROM to the top 64kB where it is protected and I then use the RESTORE word to pop that back into the normal space thereby restoring the image with all the stable modules etc. I may actually get around to writing a FORGET word though
I'm reading MCP3208.fth.
There are part that I cannot understand.
Although maybe foolish question, Please give me answer.
1
What does mean 'TACHYON' on top line and 'end on last line?
This need because Tachypn use byte-code?
2
What is "pri MCP3208.fth ." MCP3208 8 channel 12-bit ADC - 120828.1000 " ;"?
3
Comment word are below?
{ .... }
\
''
4 What is last '}'?
Why 'IFNDEF #miso', Why not 'IFNDEF #mcp'?
5
What 'pub FADC@ ( ch -- data )'?
Why not ': FADC@'?
6
What is '[~' and ']~'?
1
What does mean 'TACHYON' on top line and 'end on last line?
This need because Tachypn use byte-code?
A: Simply to ensure that we know that this Forth source file is for Tachyon Forth and not some other Forth and this also informs TF to format the response for streaming mode in that we don't really want every character echoed etc.
2
What is "pri MCP3208.fth ." MCP3208 8 channel 12-bit ADC - 120828.1000 " ;"?
A: This is simply to identify the following code which also reports at startup or when you execute the MODULES word (any words that end in .fth are identified as module names). BTW, the "pri" instead of "pub" or ":" is the same thing at present but may be used in some version to remove the name from the dictionary after it has compiled when it encounters the END word.
3
Comment word are below?
{ .... }
\
''
A: Comments are either a \ or braces { } or brackets ( ) and since there is no traditional TIB or text input buffer then these comments may span many lines. Therefore the braces are really redundant as you could just as easily do multi-line comments with brackets ( ).
Note that " is never a comment and here are some uses for this:
"#" ............This will place the literal value of the ASCII character # onto the stack. Note that there is no space between the " and the # as this works like the other prefix and suffix operators do for numbers.
"<space>a string" ................Will compile a null terminated string (temporary or permanent) and leave it's address on the stack. See NUMBER FORMATS section in the Intro page for further information.
4 What is last '}'?
Why 'IFNDEF #miso', Why not 'IFNDEF #mcp'?
A: KISS leads me to to think if I ignore the rest of this section then it is not much different to a comment except it's conditional so therefore I will treat it like a multi-line comment and terminate it with a right brace. The right brace by itself outside of a comment or if the definition is accepted will be ignored. As for why not then why not #mosi or #sck and as we know if *any* one is missing then that's all we need to know.
5
What 'pub FADC@ ( ch -- data )'?
Why not ': FADC@'?
A: Forth does not force us to conform to a strict syntax so why should I force this on Propeller users who are familiar with Spin? If this helps to put a friendlier face on Forth than it is no skin off my nose. There is a side benefit though as I use Google docs for a lot of my source code as I can include images etc but I find that the full stop and colon characters are harder to see, they don't really stand out but "pub" certainly does. Also pub and pri allow me to get into the habit of deciding which words need to be exposed and which other ones could be culled from the dictionary if need be.
6
What is '[~' and ']~'?
A: These character sequences are normally intercepted by the PASM receiver cog to tell it to filter out comments, whitespace, etc to enable faster loading. They are also interpreted at the Forth level to do something similar. The preprocessor operation has been deprecated since I made the receive routine run at full throughput with a single stop bit at megabit speeds (in another thread). So they are there simply to improve streaming mode where we paste or send a file through the terminal and they could be incorporated back into TACHYON and END, perhaps. BTW, the tilde sequence was chosen because I found that it out of almost everything I looked at would be unique and unmistakable enough as I didn't want the preprocessor triggering falsely.
Hope that answers your questions so just let me know if you have any more as sometimes when I answer them I can think of things I need or want to do.
BTW, if you read the source code from the Google docs links it will make more sense since it includes the diagrams and datasheet info etc.
Please expalin 4 and 5 again.
Sorry, I'm NOT good at english.
Especially, I'm poor at reading long sentense.
So I cannot understand them at all.
I can understand other.
Actually when I watched Tachyon site at first time, I gave up to read it.
It is hard for me to continue to read because your sentense is long.
I'm poor at reading long sentense.
If a sentence is too long you can always put your own full stop in there!
I try to keep the sentence short but as I would speak. I also insert questions in there that could likely pop up in the readers mind, and answer them. So just imagine I am speaking and you ask the question, after which I answer you.
Let's see if I can keep it succinct.
Q4. What is last '}'?
Why 'IFNDEF #miso', Why not 'IFNDEF #mcp'?
A4. I treat the whole statement as a multi-line comment that is ignored if it is already defined.
Just like a { comment } the IFNDEF is treated as a { when it has already been defined.
Compare these two statements that work the same:
{ please ignore this comment }
IFNDEF TACHYON please ignore this comment }
Either #miso or #mosi or #mcp or #sck would satisfy the test. Pick one, any one.
Q5
What 'pub FADC@ ( ch -- data )'?
Why not ': FADC@'?
A5. Because I can do both but Spin people recognize it straight-away as a definition and the name that follows as the name of that definition.
Hi Peter
Sal added Spinerette support a while back (maybe pf4.0?) and included IP, telnet and HTTP. There are two main gotcha's:
1) the support takes up so much space there is hardy anything left for an application. And Sal didn't even include FTP.
2) ethernet on a micro is a bit too specialized. It seems most folks can't find a use for it. Its well suited for a limited set of tasks, and most folks used to thinking PC amounts of resources. This is why we switched over to using Raspberry Pi for ethernet, as it provides pretty much a full linux workstation for free.
Hi Doug,
I don't think I noticed this as I might have taken a look but sometimes it's better not to. With the code size I have achieved so far I can't see it being any problem in a an embedded application, and indeed it already is. There is still plenty of room left over too.
I do think that ethernet can get a bit complicated but it doesn't have to be, after all we are just trying to pass information between nodes. My application interfaces are very simple Here are a couple of snippets:
[FONT=courier new]pub ?TELNET TELNET SOCKET \ Use the TELNET socket CONNECTED? \ New connection? IF LAN ." WELCOME TO THE TACHYON WIZNET TELNET SESSION!" CR CON \ Welcome banner RXWRITE LW@ RXREAD LW! RECV \ Reset rx buffer and receive LANSKT \ update socket and switch Forth terminal I/O to this LAN socket THEN DISCONNECTED? DROP \ Process disconnection and discard result ; [/FONT][FONT=courier new]pub SERVER socket C@ socket 1+ C! \ remember foreground socket ?TELNET \ Poll the TELNET server ?FTP \ Poll the FTP server ?HTTP \ Poll the WEB server socket 1+ C@ SOCKET \ restore foreground socket ; pub GO !LEDS !WIZIO #50 ms \ Setup SPI & I2CBUS LEDS (includes WIZNET RESET & PWRDN) &192.168.16.1 GATEWAY \ Assign a default gateway &255.255.255.0 SUBNET &192.168.16.150 SRCIP $00.08 $DC.01.02.03 SHAR \ dummy MAC, Need to have a unique ID $8000 $4000 ADO 0 I L! 4 +LOOP \ clear WIZnet transmit buffers (for diagnostics) txwr 16 ADO -1 I W! 2 +LOOP \ invalidate all txwr pointers (W5200 workaround) TELNET SOCKET CLOSE TCP #23 PORT! OPEN LISTEN \ Setup TELNET FTP SOCKET CLOSE TCP #21 PORT! OPEN LISTEN \ Setup FTP HTTP SOCKET CLOSE TCP #80 PORT! OPEN LISTEN \ Setup HTTP ' SERVER keypoll W! \ Poll the server in the background WIZ? \ report WIZnet status ;[/FONT]EDIT: Here's a screen grab of a telnet session talking to Tachyon and decompiling the ?TELNET word.
Thank you very much.
I understood them.