Using PUT and/or FTP to send a file to a server with spinneret as a client- help?
Rforbes
Posts: 281
Heya folks,
Alright- I'm once again stumbling and not getting anywhere. I'm trying to figure out how to use PUT with the spinneret and send a file to a server.
So far, my PUT header looks like this:
Once I put this into the tempbuff, I then put the content length value (which is the file size of my log.xml file.)
Then, I do the following:
My RxTx_BUFFER and tempbuffer size is $200 (512) because I have to keep space available for other methods.
I can't seem to get this to work at all. However, I can do the following use POST and it "sort of" works....
My header is:
Then I add the content length, and "name=" etc...
I still use the same code afterwards:
Using this, I seem to be able to xmit the entire file into the formtest.php file and in turn stuff the data into a file on the server named log.xml.
But if I try to send more than roughly 6kb it doesn't get placed into the log.xml file correctly. Some of the xml tags get corrupted similar to "<data01>xxx<da<data>" where it should be "<data01>xxx</data01>
What is it I'm missing here? I need to send the entire file content, but I'm struggling with understanding how to use PUT. Is there a better way to do this?
If it helps, I'm attempting to send this file to the server 1 time per minute. Other methods will be modifying this file throughout the minute.
As always, thanks sooooooooo much!
Robert
Alright- I'm once again stumbling and not getting anywhere. I'm trying to figure out how to use PUT with the spinneret and send a file to a server.
So far, my PUT header looks like this:
Put_Hdr byte "PUT /log.xml HTTP/1.1",13,10, {
} "Host: blahblah",13,10, {
} "User-Agent: Wiz5100",13,10, {
} "Connection: close",13,10, {
} "Authorization: Basic xyz==",13,10, {
} "Content-length:",0
Once I put this into the tempbuff, I then put the content length value (which is the file size of my log.xml file.)
Then, I do the following:
bytemove(@txdata, @tempbuff, hdrsize)
Socket.txTCP(id, @txdata, hdrsize)
if fileSize =< RxTx_BUFFER ' send the file in one packet
SDCard.readFromFile(@txdata, fileSize)
Socket.txTCP(id, @txdata, fileSize)
else ' send the file in a bunch of packets
repeat 'while Socket.Connected(id)
SDCard.readFromFile(@txdata, RxTx_BUFFER)
Socket.txTCP(id, @txdata, RxTx_BUFFER)
fileSize -= RxTx_BUFFER
if fileSize < RxTx_BUFFER and fileSize > 0 ' once the remaining fileSize is less then the max packet size, just send that remaining bit and quit the loop
SDCard.readFromFile(@txdata, fileSize)
Socket.txTCP(id, @txdata, fileSize)
quit
My RxTx_BUFFER and tempbuffer size is $200 (512) because I have to keep space available for other methods.
I can't seem to get this to work at all. However, I can do the following use POST and it "sort of" works....
My header is:
Post_Hdr byte "POST /formtest.php HTTP/1.1 200 OK",13,10, {
} "Keep-Alive: 115",13,10, {
} "Host: blahblah",13,10, {
} "User-Agent: Wiz5100",13,10, {
} "Cache-Control: public",13,10, {
} "Connection: keep-alive",13,10, {
} "Content-Type: application/x-www-form-urlencoded",13,10,{
} "Authorization: Basic xyz==",13,10, {
} "Content-length:",0 {
Then I add the content length, and "name=" etc...
I still use the same code afterwards:
bytemove(@txdata, @tempbuff, hdrsize)
Socket.txTCP(id, @txdata, hdrsize)
if fileSize =< RxTx_BUFFER ' send the file in one packet
SDCard.readFromFile(@txdata, fileSize)
Socket.txTCP(id, @txdata, fileSize)
else ' send the file in a bunch of packets
repeat 'while Socket.Connected(id)
SDCard.readFromFile(@txdata, RxTx_BUFFER)
Socket.txTCP(id, @txdata, RxTx_BUFFER)
fileSize -= RxTx_BUFFER
if fileSize < RxTx_BUFFER and fileSize > 0 ' once the remaining fileSize is less then the max packet size, just send that remaining bit and quit the loop
SDCard.readFromFile(@txdata, fileSize)
Socket.txTCP(id, @txdata, fileSize)
quit
Using this, I seem to be able to xmit the entire file into the formtest.php file and in turn stuff the data into a file on the server named log.xml.
But if I try to send more than roughly 6kb it doesn't get placed into the log.xml file correctly. Some of the xml tags get corrupted similar to "<data01>xxx<da<data>" where it should be "<data01>xxx</data01>
What is it I'm missing here? I need to send the entire file content, but I'm struggling with understanding how to use PUT. Is there a better way to do this?
If it helps, I'm attempting to send this file to the server 1 time per minute. Other methods will be modifying this file throughout the minute.
As always, thanks sooooooooo much!
Robert
Comments
Buffer issues in the repeat loop or the PHP script might be the source of corruption. What column and line does the corruption occur? Is it always the same spot, byte-wise, in the file?
If the goal is to store data on a web server why not POST the record on creation? Real time as opposed to batch. If viewing the log file is important, create an html link or src that points to the log file on the PHP web server.
Let's see.... yes, it does seem to get corrupted at the approximately the same places in the file once the file size get's over approximately 5-6kb in size.
The xml file I create on the sdcard is pretty simple- looks like this:
Every minute, I append to the file on the SDcard, in effect removing the </root> and replacing it with <data><date>13/3/9</date><time>7:37:1</time><B0>0</B0> on and on and finally adding </data></root> at the end again.
So the first couple minutes, the file is fairly small in size and I send it to the server without a hitch using the following POST header:
Post_Hdr byte "Post /formtest.php HTTP/1.1 200 OK",13,10, { } "Keep-Alive: 115",13,10, { } "Host: blahblahblah",13,10, { } "User-Agent: Wiz5100",13,10, { } "Cache-Control: public",13,10, { } "Connection: keep-alive",13,10, { } "Content-Type: application/x-www-form-urlencoded",13,10,{ } "Authorization: Basic blahblah==",13,10, { } "Content-length:",0 {I then add more to the header, consisting of the value for the content length and the phrase "name=" and then I start sending the file contents to the txdata buffer as explained in the first post.
The server get's it and the php file on the server writes the content to an xml file on the server. When the server get's it again, it repeats, overwriting the original file.
I send this file to the server from the spinneret every minute- and as you can imagine, after 20 minutes or so the file size will be getting much larger.
As long as the file is 5-6kb or less, it seems to work fine. But bigger than that, it's getting the hiccups. The xml file on the server can be viewed from a regular ole browser, and it looks something like this:
It will have a few of these hiccups, and the one consistent thing about the hiccups is that where it get's messed up, it just keeps on rolling along as if all is well.... so the <B99>0</B99> will continue with <B100>0</B100> etc etc.
I can't set up the spinneret to verify if the errors are always in the same location right now but I seem to remember it being at line 565 in the server side xml file quite often and a few other lines as well.
Concerning the repeat loop in the buffer- I'm going to really study that and try to see if I can figure out if that might be the cause. I basically used your StaticFileHandler(id) but I had to modify it a tad because of my rxdata and txdata buffer lengths.
I'm assuming the rxdata and txdata buffer size do not have to be equal to max_packet_size , yes?
What is the min and max value of <bx>?</bx>
Min/Max of B0 to B1 is 0-255.
B2 to B46 is 0-1
B47 to B106 is 0-59
B107-136 is 0-23
Then there is <W0> through <W29> where each is 0-744 and then <L0> through <L14> where each is 0-8760.
By logging to the SDcard as mentioned, I have a good "backup" of the data as long as the spinneret itself doesn't go down. By sending the contents of the file on the sdcard instead of sending the array "live" and letting the server sort out the time/date stuff it then becomes "ok" if the remote server goes down and misses a few beats... because when it comes back up, the spinneret will again send the contents of the file on the sdcard which has the data sets with all the previous time/date information, etc.
Throughout the course of a day, the xml data will start at an initial size of about 2kb worth and grow to around 3 mb worth by the end of the day because it contains the value of all the stuff with the date/time it was logged, over and over. Of course, I can always change the scenario to logging on the sdcard and POSTING the data only once an hour, or something like that to minimize the length of the data posted.I hope that makes sense?
However, there's still my original problem of the data getting screwed up once the amount of data being posted is over 5 or 6 kb.
I guess what I'm trying to figure out is- does the http PUT offer advantages over POST with regards to the size or integrity of the data I'm sending?
If not, would transferring the file on the sdcard to the remote server via ftp be doable? It seems like it would be, but once again this is a learning experience for me so I'm not quite sure how/where to begin tackling it.
Sorry if this is sounding convoluted... feel free to say "I give up" if I'm not making much sense here.
But this might work a little better for ya. Don't POST the data to the server. Notify the web server the log is ready for upload by sending an HTTP GET. Maybe send the file name as a parameter. The server sends back OK to complete the GET. Next the server fires up its own HTTP GET to the Spinneret using the log file name it received on the initial request.
I've transferred 2M pdf with the Spinneret so the regular old web server should be up to the task. BTW, MTU (Max Transmission Unit) defaults to 1460 bytes. That's the sweet spot of sending and receiving data.
Ok- back to the drawing board... I'm going to try to tinker with it some more and see if I can make your recommendations work out.
As always, thanks tons for your input and advice! Will be posting more questions soon, I'm sure!
Robert