Spinneret-msrobots
msrobots
Posts: 3,709
EDIT
This project has evolved a bit so I will change the thread title accordingly.
So PUT, MKCOL, DELETE, HEAD and OPTIONS Handler for W5100/W5200 is now called Spinneret-msrobots
Mike G. supplied me with a space in the official W5200/W5100 code repository at https://code.google.com/p/propeller-w5200-driver/.
You will find a directory Spinneret-msrobots there containing all files needed.
Look at https://code.google.com/p/propeller-w5200-driver/source/browse/trunk/#trunk%2F propeller-w5200-driver%2FSpinneret-msrobots
SVN: https://propeller-w5200-driver.googlecode.com/svn/trunk/ propeller-w5200-driver/Spinneret-msrobots
*CAUTION: the Directory Name of propeller-w5200-driver starts with a space in the repository. Ask Mike G. why, I don't know...
I also keep a copy of the complete W5200/W5100 code repository on my own server since the way google code handles downloads has changed and you are not able to show .htm files on google code either ...
Look at http://parallax.msrobots.net/propeller-w5200-driver/Spinneret-msrobots/_README_Doc.htm for the current documentation.
Click View for the Source directory ...
/EDIT
Mostly I like the Spinnerette. But changing the sd-card is a PITA.
So I looked for the easiest to implement solution to upload files and found PUT.
Like GET, PUT is a HTTP-Verb. Usually not used in Webbrowsers.
There are some other verbs I will implement, but for now it is just PUT.
The PUT Verb allows you to upload a resource to the webserver instead of getting it.
The changes are simple:
Open WebServer_W5100_RTC.spin
locate in first DAT section:
locate in PRI MultiSocketService
add somewhere
Linux and MAC user should have some tool named cURL providing the PUT cmd but Windows user will need a cmdline-tool (see attachment).
what you can do now is something like this from the commandline
PUT http://192.168.1.117/Test.htm c:\test.htm
to copy file c:\Test.htm to the sd-card in the spinnerette running at 192.168.1.117 ...
This works very nice and fast since it is transferring binary data as application/octet-stream according to the RFC for put.
Enjoy!
Mike
edit: here the source for the PUT cmd
.
This project has evolved a bit so I will change the thread title accordingly.
So PUT, MKCOL, DELETE, HEAD and OPTIONS Handler for W5100/W5200 is now called Spinneret-msrobots
Mike G. supplied me with a space in the official W5200/W5100 code repository at https://code.google.com/p/propeller-w5200-driver/.
You will find a directory Spinneret-msrobots there containing all files needed.
Look at https://code.google.com/p/propeller-w5200-driver/source/browse/trunk/#trunk%2F propeller-w5200-driver%2FSpinneret-msrobots
SVN: https://propeller-w5200-driver.googlecode.com/svn/trunk/ propeller-w5200-driver/Spinneret-msrobots
*CAUTION: the Directory Name of propeller-w5200-driver starts with a space in the repository. Ask Mike G. why, I don't know...
I also keep a copy of the complete W5200/W5100 code repository on my own server since the way google code handles downloads has changed and you are not able to show .htm files on google code either ...
Look at http://parallax.msrobots.net/propeller-w5200-driver/Spinneret-msrobots/_README_Doc.htm for the current documentation.
Click View for the Source directory ...
/EDIT
Mostly I like the Spinnerette. But changing the sd-card is a PITA.
So I looked for the easiest to implement solution to upload files and found PUT.
Like GET, PUT is a HTTP-Verb. Usually not used in Webbrowsers.
There are some other verbs I will implement, but for now it is just PUT.
The PUT Verb allows you to upload a resource to the webserver instead of getting it.
The changes are simple:
Open WebServer_W5100_RTC.spin
locate in first DAT section:
_h200 byte "HTTP/1.1 200 OK", CR, LF, $0 _h404 byte "HTTP/1.1 404 Not Found", CR, LF, $0replace by
_h100 byte "HTTP/1.1 100 Continue", CR, LF, $0 _h200 byte "HTTP/1.1 200 OK", CR, LF, $0 _h201 byte "HTTP/1.1 201 Created", CR, LF, $0 _h403 byte "HTTP/1.1 403 Forbidden", CR, LF, $0 _h404 byte "HTTP/1.1 404 Not Found", CR, LF, $0 _h409 byte "HTTP/1.1 409 Conflict", CR, LF, $0
locate in PRI MultiSocketService
if (FileExists(fn)) RenderFile(sockId, fn) else ifnot (RenderDynamic(sockId)) sock[sockId].Send(@_404, strsize(@_404))replace by
ifnot (PutHandler(sockId, fn)) if (FileExists(fn)) RenderFile(sockId, fn) else ifnot (RenderDynamic(sockId)) sock[sockId].Send(@_404, strsize(@_404))
add somewhere
PRI PutHandler(sockId, fn) | bytesToRead, size , status if strcomp(@buff, string("PUT")) status := @_h201 ' 201 created size := StrToBase(req.Header(string("Content-Length")) , 10) if FileExists(fn) ' if file already there status := @_h200 ' 200 OK ( or 202 no Content?) \sd.deleteEntry(fn) ' delete \sd.newFile(fn) ' new file sd.closeFile ' now close file ? ifnot (sd.openFile(fn, IO_WRITE) == true) ' now open file write status := @_h409 ' 409 Conflict else sock[sockId].send(@_h100, strsize(@_h100)) ' send 100 continue sock[sockId].send(@_newline, strsize(@_newline)) ' send _newline repeat until size<1 ' expecting size bytes repeat until bytesToRead := sock[sockId].Available 'Repeat until we have data in the buffer if(bytesToRead < 1) ' Check for a timeout error size := -1 'timeout else sock[sockId].Receive(@buff, bytesToRead) ' Move the Rx buffer into HUB memory size -= bytesToRead sd.writeData(@buff, bytesToRead) ' now write file \ sd.closeFile ' now close file sock[sockId].send(status, strsize(status)) ' send 409 Conflict 201 Created or 200 OK sock[sockId].send(@_newline, strsize(@_newline)) ' send _newline return true ' done return false
Linux and MAC user should have some tool named cURL providing the PUT cmd but Windows user will need a cmdline-tool (see attachment).
what you can do now is something like this from the commandline
PUT http://192.168.1.117/Test.htm c:\test.htm
to copy file c:\Test.htm to the sd-card in the spinnerette running at 192.168.1.117 ...
This works very nice and fast since it is transferring binary data as application/octet-stream according to the RFC for put.
Enjoy!
Mike
edit: here the source for the PUT cmd
[SIZE=2][COLOR=#020fc0][SIZE=2][COLOR=#020fc0]using System; using System.Text; namespace put { class Program { static void Main(string[] args) { int cnt = args.Length; if (cnt < 2) { Console.WriteLine("USAGE: put remotefilename localfilename"); Console.WriteLine("EXAMPLE: put [URL]http://192.168.1.117/upload/test.txt[/URL] c:\\test.txt"); } else { Console.WriteLine(PutFile(args[0], args[1])); } //Console.ReadKey(); } private static string PutFile(string remotefile,string localfile) { string sReturn = ""; try { System.IO.FileStream InputBin = new System.IO.FileStream(localfile, System.IO.FileMode.Open, System.IO.FileAccess.Read, System.IO.FileShare.None); try { System.Net.HttpWebRequest request = (System.Net.HttpWebRequest)System.Net.WebRequest.Create(remotefile); request.Method = "PUT"; request.ContentType = "application/octet-stream"; request.ContentLength = InputBin.Length; System.IO.Stream dataStream = request.GetRequestStream(); byte[] buffer = new byte[32768]; int read; while ((read = InputBin.Read(buffer, 0, buffer.Length)) > 0) { dataStream.Write(buffer, 0, read); } dataStream.Close(); System.Net.HttpWebResponse response = (System.Net.HttpWebResponse)request.GetResponse(); sReturn = response.StatusCode.ToString(); if (sReturn != response.StatusDescription) sReturn += " " + response.StatusDescription; response.Close(); } catch (Exception ex1) { sReturn = ex1.Message; } InputBin.Close(); InputBin.Dispose(); } catch (Exception ex2) { sReturn = ex2.Message; } return sReturn; } } } [/COLOR][/SIZE][/COLOR][/SIZE]
.
zip
3K
Comments
The value / is allowed in url-parameter.
some things need to change (its easy)
So open HttpHeader.spin
locate replace with
locate replace with
locate replace with
This will allow '/' inside of url-params and also allows files without extension and directories with extension (both problematic before)
the returned filename now includes the path as expected by the sd-driver
Enjoy!
Mike
I prefer to use the status line for passing parameters to SPIN methods.
The GET will light the LED connected to PIN 23.
I see leaving the status line intact has its advantages when it comes to the SD driver. What do you think about this the following logic flow? Replace the "/" with a zero only If the request is not a PUT and the file does not exist. The "Does file exist" logic is a bit clunky and I believe this will simplify the intent.
you already have written a nice parser for url-parameter aka http://1.2.3.4/led.xml?L=32&S=1 why using the path for this?
One of my goals here is to stay as close to the RFCs as possible giving the restricted resources of the spinnerette.
Since I try to implement WEBDAV to create a way to 'mount' the sd of the spinnerette to the filesystem of a host pc (win, mac and Linux support WEBDAV) this PUT Verb is just the beginning.
I already have MKCOL, DELETE, HEAD and OPTIONS ready to publish and will do so in a moment.
The next two are more complicated PROPFIND and PROPPATCH and will not fit into the spin-code anymore. But I have already another goodie ready:
Dynamic Web-Pages in PASM. Yes. Server side scripting in PASM.
It is easier as it sounds like. A request to a file with extension .PSM will load the file to the end of the Buffer_2K (I needed to upgrade this to Buffer_3K to have enough space) and then start a COG to execute the PASM-BLOB.
Spin is then sitting in a tight commandloop waiting for cmds of the PASM-COG to deliver the Web-Page/File. It is like the usual mailbox but the other way around. PASM-Cog giving cmds and SPIN executing them.
I hope to clean up the already working code a little bit this weekend and will publish this soon.
Enjoy!
Mike
first we need to change S35390A_SD-MMC_FATEngineWrapper.spin and add some errorhandling
locate replace with
locate replace with
locate replace with
add a new method
now back to WebServer_W5100_RTC.spin
locate in PRI MultiSocketService replace with
and add three handler somewhere
so now the HEAD Verb
locate replace by
Enjoy!
Mike
I guess it is more the other way around ... Replace the "/" with a zero only If the request is a GET and the file does not exist... But to find out IF the file exists you need the whole Path-And-Filename together... some hen and egg problem.
I need to think about this more, but it seems quite complicated to me. Maybe some post-processing of the filename at start of RenderDynamic? replacing '/' then and create the string array just when needed?
Enjoy!
Mike
PM your email address tied to a Google account. I'll add you as a "propeller-w5200-driver" project committer. The project is using Subversion.
PM send.
You wrote a very nice and extendable piece of code there and I enjoy reading it. I do NOT have the W5200 for the QS just a couple of Spinneret's.
I still look at your NETBIOS stuff, screaming to be integrated like DNS and DHCP, and think about the following:
By now you reserve one socket for DHCP we can use/rename this for NetBIOS/tmpsocket.
This socket could be a multi-function one.
Let me explain.
NetBIOS needs constant attention. - almost!
Its UDP so we can - just sometimes - NOT listen.
But instead - close NetBIOSsocket - (do what we need this socket for) - open NetBIOS to listen again.
and reuse the same socket for NetBIOS, DHCPrenewal, SNTP, and SMTPsend just sharing it.
this would open up a lot of short time usages of a socket NOT attached to HTTP/multisocketservice
Enjoy!
Mike
Thanks for the kind words... The W5100/5200 code base is designed for users at different experience levels. A beginner can drop HTML on an SD Card while a more experienced developers might extend objects. The work you're doing is exactly what I was hoping would come along. Have fun and I look forward to whatever you create.
I updated the google repo at http://code.google.com/p/propeller-w5200-driver/source/browse/trunk/#trunk%2F%20propeller-w5200-driver%2FSpinneret-msrobots
I made another version of HttpHeader where I tried to accommodate mine and Mike G's needs - RESTful access to be tested still.
This version already supports dynamic pages/responses out of PASM. So you can compile .spin files containing a PASM prog to .binary rename to .psx and save to sd. These files can be called in a browser, get loaded into a cog and executed, producing the response to the request.
This allows some smart web-pages, extending the available space in the spin program.
All of this is work in progress, I can just work on weekends on this. (Why is it that as soon I have some interesting stuff to program on a propeller my paid work load increases?)
Anyway
Enjoy!
Mike
propfind is elusive. So after fiddeling around with it for a while I decided that I need NetBIOS support to do this.
Mike G. already had some code for the w5200 so I started working on that too.
I now published a NetBIOS.spin file running inside of the webserver.
You now can ping the server by its name and issue a nbstat -a command against it. - but still no joy with connecting a network drive.
The available documentation about NetBIOS is unclear in some points, since there is no real standard, MS did stuff and SAMBA copied the behavior.
So its wireshark to help me out, looking at requests/responses of other machines in my network and trying to simulate the same with the spinneret.
And I stumbled about the following Problem:
if I call nbtstat -a propnet (or any other name in my net)
I find a broadcast message over UDP and get it bravely delivered/read to/by the spinneret.
if I call nbtstat -A 192.168.1.117 (or any other IP of one of my computers) the request is not send to the broadcast ip but as unicast message to the ip (one computer in my net).
understandable for me.
The crux with name/status query's in NetBIOS (as far as I can see in wireshark) is that the behavior of windows machines is different if the request comes in as broadcast or to the direct IP of the target computer (unicast) .
You just see the difference if you use the wildcard ("*") name.
In this case the windows computer seem to just answer if you send a unicast message, not on a broadcast query.
So if anybody is still with me here the question:
How I can find out if a UDP packed I received on the spinneret was send from the sender to the broadcast ip OR to my own ip?
Enjoy!
Mike
I spend a lot of time optimizing and documenting the code. I also commented out lot of unused functions to save LONGS.
The App Server is working and I can run PASM scripts from SD card.
NetBIOS Name Service works and the Spinneret is available by IP and NAME in the network.
This uses (and shares) one of the sockets with DHCP and SNTP.
NetBIOS Datagram Service will need another socket to monitor port 138. I am not sure if it is worth to do that, yet.
The remaining sockets run HTTP
Next step will be replacing PST by 4portserial to open communication channels to other microcontroller.
this would allow 2-3 serial clients to share a spinneret (even more with Xbee Broadcast ?)...
The Spinneret is a really nice network card.
And since I now can PUT, DELETE and MKCOL I do not need to touch that small SD card EVER AGAIN.
Complete control of SD card over the Ethernet ... from command line only since PROPFIND is still elusive so no mapped Network Drive, yet.
Enjoy!
Mike
So PUT, MKCOL, DELETE, HEAD and OPTIONS Handler for W5100/W5200 is now called Spinneret-msrobots
Mike G. supplied me with a space in the official W5200/W5100 code repository at https://code.google.com/p/propeller-w5200-driver/.
You will find a directory Spinneret-msrobots there containing all files needed.
Look at https://code.google.com/p/propeller-w5200-driver/source/browse/trunk/#trunk%2F propeller-w5200-driver%2FSpinneret-msrobots
SVN: https://propeller-w5200-driver.googlecode.com/svn/trunk/ propeller-w5200-driver/Spinneret-msrobots
*CAUTION: the Directory Name of propeller-w5200-driver starts with a space in the repository. Ask Mike G. why, I don't know...
I also keep a copy of the complete W5200/W5100 code repository on my own server since the way google code handles downloads has changed and you are not able to show .htm files on google code either ...
Look at http://parallax.msrobots.net/propeller-w5200-driver/Spinneret-msrobots/_README_Doc.htm for the current documentation.
Click View for the Source directory ...
I also updated the first post to contain this information.
The documentation is still work in progress.
I am using the brilliant spin documenter from PhiPi found at http://www.phipi.com/spin2html/
Documenting this way includes all documentation in your spin file and after changes you just upload the file and save the generated htm.
Simple like that. PhiPi is - well - brilliant. He wrote the code (and provided the service since then) about 2006. We just need to use it.
Enjoy!
Mike
I was able to integrate NetBIOS and DNS into the Spin code. Now running very low on ram (200 longs left for stack). Will need more optimization
But I made a huge step with dynamic web-pages in PASM.
The concept is quite simple.
On Page Request the Spin code starts a Cog loaded from SD. Then Spin loops around a command loop, waiting for commands from the PASM cog. Mailbox the other way around. The PASM cog / Web-Page can call subroutines in Spin to resolve GET and POST parameter, DNS query's, NetBIOS query's and other things. Directory and File access is also provided to the PASM cog / Web Page. While running the PASM cog / Web-Page creates the response for the request and the quits itself after telling spin it is done. wash rinse repeat.
The PASM programs / Web-Pages are not bound to deliver html. You can also produce xml or bmp or whatever you want ... it is just a response to a request.
Another thing I added is ETag handling for all static files served from SD. Since Kye's Fat_Engine supports the RTC of the spinneret the last modified date in the directory entry was a good candidate for the ETag. If the file is changed the ETag gets changed also ... working flawlessly since 2 weeks ...
Finally I put Propnet1 out there in the Internet. Fixed external IP, NO firewall. And got shut down with irritating requests on port 80. Watched, thought about and fixed the issues ... now stable since a week.
This is just my Test Server. so nothing fancy yet. http://38.110.184.19/Index.htm is running from the spinneret. (Propnet1)
I will update the sources at the spinneret Google code repository, Parallax.MSrobots,net and the Spinneret Propnet1 itself tonight. By now you are a week behind or so...
Enjoy!
Mike
If I try to compile this project I'm exceeded the memory limit by 45 longs? Looks like this is just what I'm after (as you pointed out previously Mike :thumb: ) and I wanna get stuck into it. I have VS2012 and I was also having some issues with opening the VS files as they are 2008 version. Will I need the 2008 version?
Mike, can I take a socket out of rotation like in the HttpServer example in the WebServer_5100 as I'm trying to move across to the new Version?
Has anyone had a serious issue with the SNTP today? Seems like mine is going crazy , keep returning weird times even with the demo programs?
One other strange issue I'm having with the Spinneret.. my Spinneret generally draws around 220mA , when powered down overnight sometimes when I start it up it draws an extra 100mA and the Spinneret/program locks up. It is just running as a client and not running the Server software. I'm pretty sure it has something to do with the Ethernet PHY, as if i power down and restart it without the ethernet cable inserted the current draw is back to 220mA
I have tried it with all of msrobots code (the netbios version) and also just with Mike server with the code additions in this thread before the edit (Just trying the PUT) but can tseem to make any headway.
I can access files on the sd from the browser so it is running but cant seem to get these file transfers going
Any thoughts appreciated. Doing what ms robots profile pic is at
PUT.exe is a cmd-line tool.
so you need to call it from a commandline aka dos window.
like copy
first parameter is the URL on the spinneret the file should go, second parameter the filename on the local disk
Enjoy!
Mike
I give that a craic in the morning. So I'm not using the VS C# file just the .exe ? And you are transferring files in the opposite direction(Spinner - PC) using the MKCOL ? (This is actually the more important aspect I'm trying to achieve )
Thanks
Des
You need to create exe files out of the c# solutions. And then call them from a command prompt.
PUT puts a file from pc to spinneret.
GET gets a file from spinneret to pc (simple download in webbrowser)
MKCOL creates a directory on the spinneret
DELETE deletes file or empty directory on the spinneret
Enjoy!
Mike
Thanks for the update
OK working on it again, talked about in ZOOM,
here a current snapshot