If a browser tries to make lets say 8 simultaneous socket connections to HTTPServer, will it result in error messages or will the 5-8 connections politely wait their turn for a free wiznet socket each?
It really depends on the browser and the number of bytes requested. It's common for 2 sockets to handle a potential 4 socket request. Likewise, a potential 4 socket request could consume all 4 sockets.
An error message requires that a socket actually handled the request. Errors come in the form of a poorly formatted web page because the CSS file was not downloaded or a red X in place of a missing image. Again, it depends on the browser. Firefox tends to give up and cancel a request where MACs running Safari seem to churn waiting for a response that may never complete.
Socket management is a huge concern when working with only 4 sockets.
There must be a way.for HTTPServer to advise the browser to limit itself to only 1 socket connection at a time.
Maybe the Wiznet creators can give us some pointers.
Why does HTTPServer need to have a RemoteTCP server?
What is the use of having a remote server?
There must be a way.for HTTPServer to advise the browser to limit itself to only 1 socket connection at a time.
Web servers fulfill client requests. A web server has no control over the number of client sockets requested. Plus HTTP is an anonymous protocol.
Caching can be used to limit sockets requests by retrieving resources downloaded to the client's local cache. Set globalCache to a 1 to cache all requests in HTTPServer.
globalCache byte $1
Some browsers have configuration that limit the number of socket connections. However, this is a client setting. Folks would be up in arms if a server started changing client browser settings.
Why does HTTPServer need to have a RemoteTCP server?
HTTPServer does not need a remote TCP call. I want to make remote TCP calls. Plus a remote IP is a default W5100 configuration. Please see the W5100 manual.
Thanks Paul, which images are you referring to, the links? What browser are you using and platform?
Hi Mike.
Ms-XP, Firefox 3.6.12. The image on the main page for one. But the good news is it took me 4 reloads to get this today. Friday it was almost every time. The image usually disappears if it gets "torn up" like the attach file shows. Maybe it is load sensitive?
Paul
Thanks Paul, It's a relatively large file. I can't seem to reproduce the issue. I hit the image from several locations, home office, work, friend's house, etc. I'm not sure what's going on.
Darn... Something happened. Well the Spinneret ran for about 2 months before locking up. Unfortunately, I disconnected the logger and serial terminal so I have no idea what the last request was. So if anyone locked the Spinneret I'd like to hear for you.
Very nice project and congrats on your win. I just want to point out, in your source files for the HTTPServer, the request object is missing the "Post(id, pname)" method. Works great except for the "POSTing to the server" page (wont compile with this method missing).
Again Congrats.
P.S. I just figured out how to turn on your Spinneret user led. look its on
marzec309, thank you for the heads-up. I updated all archives files.
And yes I saw the user LED was on
A little stat update. The Spinneret served up ~4000 requests in the last couple of days. Some are search engine crawlers but a fair amount are folks reading the tutorial. Please let me know if you find errors. I'll do my best to fix the errors quickly. If there is functionality you would like added, please just let me know. I can't make make any promises but I can certainly look into the request..
Right now I'm looking at SNMP because of this thread.
I've been working on an SPI version of Mike's great software for use with the ETHShield and the Propeller ASC. I have HTTPServerBase.spin successfully communicating with the W5100 setting and reading configurations. But I've hit a wall and can't go any further without some help. I've narrowed the problem down (maybe)to memory locking.
CmdWait
'----------------------------------------
' Mike Gebhard 4/17/2011
' Writes all 4 status register values
' to HUB ram during idle time waiting
' for a command
'-----------------------------------------
:init_state mov statereg, stsreg0 ' status register 1 $0403
mov statests, stateidx ' point to status[0]
mov counter, #4 ' countdown 4, 3 ,2 ,1, 0
:mem_lock lockset stssemid wc
if_c jmp #:mem_lock
:write_ram mov reg, statereg ' Register address ($0403)
mov ram, statests ' destination of socket's status byte
mov ctr, #1 ' We're just moving one byte
call #ReadMulti ' Execute the command
add statereg, #$0100 ' Next socket register $0403 + $100
add statests, #1
djnz counter, #:write_ram ' Do this 4 times
lockclr stssemid
'-----------------------------------------
' End of 4/17/2011 update
'-----------------------------------------
:skip_cmd rdlong cmdAdd, par wz 'Check for a command being present
if_z jmp #CmdWait 'If there is no command, jump to check again
From what I can tell this part of the main PASM loop is only letting the semaphore unlock for 700nS and staying locked for 19.5uS. That's ~94.6% in a locked state and the unlocked window is so short that the SPIN lockset() below is only getting a lock every few hundred iterations of the PASM lockset.
PUB GetStatus32(writeTo) | i
repeat until not lockset(statusSemId) '<--- We seem to wait here forever...
repeat i from 0 to 3
byte[writeTo][i] := status[i]
lockclr(statusSemId)
This appears to be the blockage but I could be totally wrong. I've been flailing at it for a couple of days and had to admit defeat. Current version (with gobs of debug code added) attached. Hoping that someone spots an obvious error in my mod.
Martin Hodge, what happens when you invoke a potential dirty read? That negates the memory lock.
PUB GetStatus(id) : value
value := status[id]
OR
' Read all 4 bytes
PUB GetStatus32(writeTo) | i
'repeat until not lockset(statusSemId) '======================PROBLEM HERE======================
repeat i from 0 to 3
byte[writeTo][i] := status[i]
lockclr(statusSemId)
Frankly, I've never been able to get to the demo SPI driver to function. Do you have the demo SPI driver working without the mod?
Commenting out the lock in GetStatus32 allows it to get all the sockets to the "LSTN" state, but it's dead after that. Is the serial output supposed to continuously update?
The demo SPI driver works fine, out-of-the-box, on the ETHShield. Strange that it won't work on the Spinneret, all the connections are there.
Is the serial output supposed to continuously update?
I'm not totally sure what you are referring to but the status registers are continuously polled and their state is written to RAM. In your case, this is over an SPI link.
The demo SPI driver works fine, out-of-the-box, on the ETHShield. Strange that it won't work on the Spinneret, all the connections are there.
You have TCP to working using W5100_SPI_Driver.spin? Please post your working code. I've had the UPD demo working but I was never was able to get TCP working on W5100_SPI_Driver.spin.
What's a ETHShield? Is there a datasheet?
Edit: I just tried the W5100_SPI_Driver.spin and W5100 UDP Echo Demo SPI Ver. 00.4 from the Google repository. I can't get the W5100 to initialize. I tried your code and just got weird characters in PST. It seems to occur when starting the SPI driver. I kinda' suspect that the I/O is not configured correctly for the Spinneret, not sure though. I'll look into it later tonight.
I'm not totally sure what you are referring to but the status registers are continuously polled and their state is written to RAM. In your case, this is over an SPI link.
I wasking about the serial terminal readout on PST. It was a lazy question though, I'll figure it out.
You have TCP to working using W5100_SPI_Driver.spin? Please post your working code. I've had the UPD demo working but I was never was able to get TCP working on W5100_SPI_Driver.spin.
It's just the web server demo from the Google repository. Swapped out the OBJ and made sure MOSI, MISO, CLK and /SCS are set to the correct pins in the configuration. Also SEN (tied to P15) will need to be asserted high on the Spineret to put the W5100 in SPI mode. SEN is permanently tied high on the ETHShield, so you won't find code in there to do that (outa[15] := 1). From what I can tell from the Spineret schematic the correct pins should be:
The server is up has has been up since Roy's post. The server logged a little over 4000 requests today. It might have something to do with my home network.
Could it be that most of the 4000 requests did not complete properly?
For example, maybe the connection was accepted but the html data was not fully sent out.
The first time I tried to access your site (about 5 mins ago), the request went out from Firefox and then it just started clocking time. I closed that tab and tried again and got connected. Now, I've been hitting it, trying menus, loading and reloading various pages without any troubles. Now it consistently works like a champ!
Could it be that most of the 4000 requests did not complete properly?
Possibly, but I've had positive feedback on the tutorial. I assume that folks are getting through.
The Spinneret is getting hit pretty hard lately. From 7:30 to 8:30am PST the Spinneret logged around 200 requests. Many of the requests come in on the same second. I imagine there is some concurrency issues with the 4 socket device.
I just followed the first part of your tutorial (through to Hello World) and all worked as simply and easily as advertised. I downloaded your httpserv.zip changed the parameters to match my config, created an index.htm on my sd card, loaded it all up and it worked like a champ. I've externalized it at http://50.43.142.149:8080 and was able to reach out from outside my network and reach myself!
Excellent work! Now it's time to build a meaningful web site,,,and clean up the pile of wires and boards that make up my web server!!
The link above works well the first time though I have to refresh the screen to seen the LED changes.
If the logs show that the response time is slow, can HTTPServer code be changed to allow longer time before timeout?
Have you thought much about requiring a log in user name and password to access the spinneret?
Well yes, and I implemented a very basic authentication scheme when uploading a binary to program a second propeller. If the password matches then the user is authorized to upload the file.
William, I'm not sure how to fix the reported problem. I can't reproduce the problem on my end. However, I can reproduce a large latency and timeouts using the tracert command to your WAN IP. That's why I asked you to hit http://www.agaverobotics.com/spinneret/controlpanel.htm which is a full blown web server. You reported
The link above works well the first time though I have to refresh the screen to seen the LED changes.
Refreshing has nothing to do with the issued command. It does indicate a poor video connection. That's two different servers on very separate domains that appear to have the same issue, latency. Add to that the folks that have successfully accessed the tutorial. I'd gladly fix the problem but I have no idea where to start.
Have you done any troubleshooting on your end? How about connecting a Spinneret on your end?
Comments
It really depends on the browser and the number of bytes requested. It's common for 2 sockets to handle a potential 4 socket request. Likewise, a potential 4 socket request could consume all 4 sockets.
An error message requires that a socket actually handled the request. Errors come in the form of a poorly formatted web page because the CSS file was not downloaded or a red X in place of a missing image. Again, it depends on the browser. Firefox tends to give up and cancel a request where MACs running Safari seem to churn waiting for a response that may never complete.
Socket management is a huge concern when working with only 4 sockets.
Maybe the Wiznet creators can give us some pointers.
Why does HTTPServer need to have a RemoteTCP server?
What is the use of having a remote server?
Caching can be used to limit sockets requests by retrieving resources downloaded to the client's local cache. Set globalCache to a 1 to cache all requests in HTTPServer.
Some browsers have configuration that limit the number of socket connections. However, this is a client setting. Folks would be up in arms if a server started changing client browser settings.
HTTPServer does not need a remote TCP call. I want to make remote TCP calls. Plus a remote IP is a default W5100 configuration. Please see the W5100 manual.
To execute logic on a remote server like sending email, writing to a database, solving complex math problems...
Here are a few resources to help your understanding of HTTP and Web Servers.
http://spinneret.servebeer.com:5000/
http://www.w3.org/
http://www.parallax.com/StoreSearchResults/tabid/768/txtSearch/spinneret/List/0/SortField/4/ProductID/710/Default.aspx
http://www.parallax.com/Portals/0/Downloads/docs/prod/prop/W5100Datasheetv1.2.2.pdf
Ms-XP, Firefox 3.6.12. The image on the main page for one. But the good news is it took me 4 reloads to get this today. Friday it was almost every time. The image usually disappears if it gets "torn up" like the attach file shows. Maybe it is load sensitive?
Paul
Paul
EDIT: turned on the logger.
Again Congrats.
P.S. I just figured out how to turn on your Spinneret user led. look its on
And yes I saw the user LED was on
A little stat update. The Spinneret served up ~4000 requests in the last couple of days. Some are search engine crawlers but a fair amount are folks reading the tutorial. Please let me know if you find errors. I'll do my best to fix the errors quickly. If there is functionality you would like added, please just let me know. I can't make make any promises but I can certainly look into the request..
Right now I'm looking at SNMP because of this thread.
CmdWait '---------------------------------------- ' Mike Gebhard 4/17/2011 ' Writes all 4 status register values ' to HUB ram during idle time waiting ' for a command '----------------------------------------- :init_state mov statereg, stsreg0 ' status register 1 $0403 mov statests, stateidx ' point to status[0] mov counter, #4 ' countdown 4, 3 ,2 ,1, 0 :mem_lock lockset stssemid wc if_c jmp #:mem_lock :write_ram mov reg, statereg ' Register address ($0403) mov ram, statests ' destination of socket's status byte mov ctr, #1 ' We're just moving one byte call #ReadMulti ' Execute the command add statereg, #$0100 ' Next socket register $0403 + $100 add statests, #1 djnz counter, #:write_ram ' Do this 4 times lockclr stssemid '----------------------------------------- ' End of 4/17/2011 update '----------------------------------------- :skip_cmd rdlong cmdAdd, par wz 'Check for a command being present if_z jmp #CmdWait 'If there is no command, jump to check againFrom what I can tell this part of the main PASM loop is only letting the semaphore unlock for 700nS and staying locked for 19.5uS. That's ~94.6% in a locked state and the unlocked window is so short that the SPIN lockset() below is only getting a lock every few hundred iterations of the PASM lockset.
PUB GetStatus32(writeTo) | i repeat until not lockset(statusSemId) '<--- We seem to wait here forever... repeat i from 0 to 3 byte[writeTo][i] := status[i] lockclr(statusSemId)This appears to be the blockage but I could be totally wrong. I've been flailing at it for a couple of days and had to admit defeat. Current version (with gobs of debug code added) attached. Hoping that someone spots an obvious error in my mod.
HTTPServerBase - Archive [Date 2011.09.05 Time 19.06].zip
OR
' Read all 4 bytes PUB GetStatus32(writeTo) | i 'repeat until not lockset(statusSemId) '======================PROBLEM HERE====================== repeat i from 0 to 3 byte[writeTo][i] := status[i] lockclr(statusSemId)Frankly, I've never been able to get to the demo SPI driver to function. Do you have the demo SPI driver working without the mod?
The demo SPI driver works fine, out-of-the-box, on the ETHShield. Strange that it won't work on the Spinneret, all the connections are there.
You have TCP to working using W5100_SPI_Driver.spin? Please post your working code. I've had the UPD demo working but I was never was able to get TCP working on W5100_SPI_Driver.spin.
What's a ETHShield? Is there a datasheet?
Edit: I just tried the W5100_SPI_Driver.spin and W5100 UDP Echo Demo SPI Ver. 00.4 from the Google repository. I can't get the W5100 to initialize. I tried your code and just got weird characters in PST. It seems to occur when starting the SPI driver. I kinda' suspect that the I/O is not configured correctly for the Spinneret, not sure though. I'll look into it later tonight.
I wasking about the serial terminal readout on PST. It was a lazy question though, I'll figure it out.
Here's an archive
It's just the web server demo from the Google repository. Swapped out the OBJ and made sure MOSI, MISO, CLK and /SCS are set to the correct pins in the configuration. Also SEN (tied to P15) will need to be asserted high on the Spineret to put the W5100 in SPI mode. SEN is permanently tied high on the ETHShield, so you won't find code in there to do that (outa[15] := 1). From what I can tell from the Spineret schematic the correct pins should be:
http://www.arduino.cc/en/Main/ArduinoEthernetShield
I can't access it too. After the browser (Firefox) established connection, the connection becomes lost.
For example, maybe the connection was accepted but the html data was not fully sent out.
I still can't access it.
The first time I tried to access your site (about 5 mins ago), the request went out from Firefox and then it just started clocking time. I closed that tab and tried again and got connected. Now, I've been hitting it, trying menus, loading and reloading various pages without any troubles. Now it consistently works like a champ!
Possibly, but I've had positive feedback on the tutorial. I assume that folks are getting through.
The Spinneret is getting hit pretty hard lately. From 7:30 to 8:30am PST the Spinneret logged around 200 requests. Many of the requests come in on the same second. I imagine there is some concurrency issues with the 4 socket device.
William please try issuing a service command. The LED state is echoed back to the caller in XML.
http://spinneret.servebeer.com:5000/led/25/on
You could also try access through a web server. Just click the LED On/Off links.
http://www.agaverobotics.com/spinneret/controlpanel.htm
I just followed the first part of your tutorial (through to Hello World) and all worked as simply and easily as advertised. I downloaded your httpserv.zip changed the parameters to match my config, created an index.htm on my sd card, loaded it all up and it worked like a champ. I've externalized it at http://50.43.142.149:8080 and was able to reach out from outside my network and reach myself!
Excellent work! Now it's time to build a meaningful web site,,,and clean up the pile of wires and boards that make up my web server!!
First off, I need a cool favicon.ico file!!
Thanks for the great work, Mike!!
@william chan, I set the logger up to monitor IP. Let me know if you try requesting a page so I can look at the log and hopefully see what's going on.
For the LED link, I get the XML also after about 10 to 20 tries.
Access thru the web server works well.
The link above works well the first time though I have to refresh the screen to seen the LED changes.
If the logs show that the response time is slow, can HTTPServer code be changed to allow longer time before timeout?
Well yes, and I implemented a very basic authentication scheme when uploading a binary to program a second propeller. If the password matches then the user is authorized to upload the file.
Access is still difficult from my end but it seems to have improved.
William, I'm not sure how to fix the reported problem. I can't reproduce the problem on my end. However, I can reproduce a large latency and timeouts using the tracert command to your WAN IP. That's why I asked you to hit http://www.agaverobotics.com/spinneret/controlpanel.htm which is a full blown web server. You reported
Refreshing has nothing to do with the issued command. It does indicate a poor video connection. That's two different servers on very separate domains that appear to have the same issue, latency. Add to that the folks that have successfully accessed the tutorial. I'd gladly fix the problem but I have no idea where to start.
Have you done any troubleshooting on your end? How about connecting a Spinneret on your end?