[but worked around] Not all images on page getting loaded.
![Phil Pilgrim (PhiPi)](https://forums.parallax.com/uploads/userpics/612/nMG83RHE7MV61.gif)
'Just wondering if my Spinneret server software is the victim of a timeout mechanism I don't understand and how I might go about fixing it. My webcam page includes links to six dynamically-generated images. They don't alway all get loaded by my browser, though. Here's an example:
![attachment.php?attachmentid=78918&d=1299195947](http://forums.parallax.com/attachment.php?attachmentid=78918&d=1299195947)
Here's the HTML source with the links to the missing images highlighted:
The curious thing, as the attached "net sniff" dump shows, is that the browser never even requested the missing images. So I'm wondering:
1. Was there some kind of timeout in effect? IOW did the server fail to keep up with the browser's timing expectations and it just gave up requesting more stuff?
2. Did the browser want to request the images but got cut off when the Spinneret reset the socket between requests?
3. Or ... ?
One more note: This failure occurs on my Windows computer running Opera on the same local net. It has also been reported by a client from outside my local network. Curiously, I have yet to see this happen with Opera on my iMac (on the same local network), even though the images seem to take forever to load.
-Phil
Here's the HTML source with the links to the missing images highlighted:
<html xmlns="http://www.w3.org/1999/xhtml" dir="ltr" lang="en"> <head> <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1" /> <title>SpinneretCAM</title> </head> <body style="background-color: #ccccaa"> <div align="center" style="color:navy;font-size:20pt;font-family:sans-serif"> <hr/> <b>Spinneret WebCam</b><hr/> <div style="color:black;font-size:10pt"> <form method="get" action="/change"> <img src=/webcam.bmp?pan=5&target=6.00&view=21&map=Gray width=384 height=288/><p/> Sensor exposure is set dynamically, depending on the light level. The<br/> goal is to keep the average pixel as close to the user-settable target<br/> value as possible by adjusting the exposure time and gain setting.<p/> <table style="font-size:10pt"> <tr align="center"> <td> Current exposure time: <b style="color:red">7.77</b> ms.<br/> 0 <img src=/hbar.bmp?v=59&fg=ff0000/> 21<br/> 0 [color=red]<img src=/hbar.bmp?v=77&fg=00aa00/>[/color] 31<br/> Current gain setting: <b style="color:#00aa00">15</b><br/> </td> <td> Target average brightness: <b style="color:#6666ff">6.00</b><br/> 0 [color=red]<img src=/hbar.bmp?v=64&fg=6666ff/>[/color] 15<br/> 0 <img src=/hbar.bmp?v=58&fg=6666ff/> 15<br/> Actual average brightness: <b style="color:#6666ff">5.48</b><br/> </td> </tr> </table> <p/> <input type="submit" name="change" value="Cool" style="width:60px"> <input type="submit" name="change" value="Gray" style="width:60px"> <input type="submit" name="change" value="Warm" style="width:60px"> <input type="submit" name="change" value="Darker" style="width:60px"> <input type="submit" name="change" value="Refresh" style="width:60px"> <input type="submit" name="change" value="Lighter" style="width:60px"><p/> <table style="font-size:10pt"> <tr align="center"> <td> <input type="submit" name="change" value="|<<" style="width:30px"> </td> <td> <input type="submit" name="change" value="<<" style="width:30px"> </td> <td> <input type="submit" name="change" value="<" style="width:30px"> </td> <td> Current pan position: <b style="color:#808080">5</b><br/> -5 <img src=/pan.bmp?5/> 5<br/> </td> <td> <input type="submit" name="change" value=">" style="width:30px"> </td> <td> <input type="submit" name="change" value=">>" style="width:30px"> </td> <td> <input type="submit" name="change" value=">>|" style="width:30px"> </td> </tr> </table> <p/> <input type="hidden" name="target" value="6.00"> <input type="hidden" name="pan" value="5"> <input type="hidden" name="map" value="Gray"><p/> There have been <b>21</b> views since last reset. </form> </div> </div> </body> </html>
The curious thing, as the attached "net sniff" dump shows, is that the browser never even requested the missing images. So I'm wondering:
1. Was there some kind of timeout in effect? IOW did the server fail to keep up with the browser's timing expectations and it just gave up requesting more stuff?
2. Did the browser want to request the images but got cut off when the Spinneret reset the socket between requests?
3. Or ... ?
One more note: This failure occurs on my Windows computer running Opera on the same local net. It has also been reported by a client from outside my local network. Curiously, I have yet to see this happen with Opera on my iMac (on the same local network), even though the images seem to take forever to load.
-Phil
Comments
I added caching. Caching helps a lot but it does not fix the problem. And it's not just images but any src file; css, javascript. It seems like there is an exponential relationship to the number of src files and time to download. I have no hard evidence and I'm reading files from the SD card.
BTW, very nice WebCam page.
There was no change in behavior. Next I commented out the return that follows it. No change. So I don't think it has anything to do with that section of code. Also, my timeout is five seconds, and nothing is taking that long.
That left me wondering if Opera was getting screwed up with caching, even though each image has the Cache-control: no-store pragma in its HTTP header. To test this, I added the views variable to the images' GET strings, so each address instance would be unique. 'Still no joy.
-Phil
But I can right-click on any of the broken images, select Reload Image, and it displays right away.
-Phil
1. Every browser appears to be trying to obtain the image data as soon as it encounters the <img> tag in the input stream, i.e. before the webpage containing it is completely loaded. At least that's how I interpret the SYNs from the client machine in the TCP dump.
2. These SYNs are always answered by the Spinneret with a RST until it has sent the entire page.
3. Some SYNs received by the Spinneret after the page has been sent are answered with SYN ACK; others, with RST. Those that get answered with SYN ACK are followed by requests for the image data, which the Spinneret then provides and which gets displayed.
4. Opera on my iMac sends inter-page SYNs, as described above, but doesn't give up due to the RSTs and simply resends the image requests again later. But this process is ponderously slooooow. The PC is much snappier but seems to give up more quickly.
It would appear that being able to signal the client not to try opening a new connection until it receives all data from a pending request would clear up this logjam. But I don't know how to do that or if it's even possible.
-Phil
I agree this needs to be fixed. I just have not had the time to mess with it. I have a couple of PASM object I really want to finish up. Then I thought I would hammer on the WizNet driver, I guess really the WIZ5100.
Phil, do you think this is a question for WIZ5100 support?
-Phil
-Phil
Once the page has loaded, you can actually see the images being loaded in the order of their appearance on the page. Although it works reliably, it seems like an ugly solution to a problem that should have a more elegant one.
That, as it turns out, is with Opera only. Firefox is another matter. With Firefox, each image takes forever to finish loading; and, for each one, a box pops up after the image has loaded to say the script is stalled, do I want to continue? If I continue, it goes on to the next image, and so on, popping up the box for each until they're all loaded. I thought maybe it was simply timing out after each one because the images didn't have a Content-length: header. But adding that header didn't help.
-Phil
But that's for ma
-Phil
I'll have to keep my mouth shut until I test with due diligence.
I would call this a browser bug, but they (Mozilla) already violates the RFC, so I guess they don't care ("single-user client SHOULD NOT maintain more than 2 connections with any server or proxy"). I.e. if in HTTP/1.1 and keep-alive, not succeeding opening more sessions, not using the already open session(s) for newly enqueued requests, is at least a misfeature.
Firefox-3 may default use up to 6 concurrent sessions, and this is not possible to handle with the W5100 (except raw mode, and it is not possible to implement in the Propeller and expect to have space left for the user application).
I do not see any workaround at present.
http://en.wikipedia.org/wiki/Data_URI_scheme
-Phil
Rather than requiring the onload=loadimages() in each <img> tag, I tried assigning it in the JS code: p.onload='loadimages()' before setting its src attribute. This worked for Opera, but not for the other browsers.
Anyway, if anyone feels like trying it and providing feedback, the page is accessible here:
-Phil
John Abshier
I cannot even load http://phipi.homelinux.org:3456/webcam.bmp indicating you may chase the wrong bug. It starts loading, but never finish before it gets a reset, making my browser tell me the site is down (connection broken).
This with Firefox 3.6.14 on GNU/Linux.
I'm quite sure that the problem I've addressed was the issue, since it was also solved in Opera (pre-JavaScript) by setting the maximum connections per server to one.
There is one puzzlement in the TCP dump that I'm at a loss to explain, though. And that's the huge number of packets containing one byte of data. Since complaining about this before, I've changed my program to buffer everything in a 512-character buffer and to send full buffers whenever possible. There's no way that the following code could revert to sending 500+ single characters in a row. (BTW, I don't see this happening on responses to my local network, only to those requests originating from outside; and then, only to some.)
So there's something going on with txTCP or the W5100 chip itself that's causing things to dribble out this way.
-Phil
Addendum: I upgraded my Linux Firefox to 3.6.14. It still loads the entire page and all the images.
Here's a recent error from FF 3.6.15 Windows XP
"The image http://phipi.homelinux.org:3456/webcam.bmp cannot be displayed, because it contains errors."
I get a partially displayed image that's all.
Thanks,
-Phil
Sorry 'bout that. I believe it's a driver misfeature caused by my deadlock fix (still, one byte is better than deadlock ,-) and the fact that txTCP() is asynchronous.
If I'm right, you'll fix this best by making your flush synchronous, not returning until the W5100 tx buffers are empty. This is the semantically sound behavior for your flush command. Or, if lazy, delay a short while after performing the txTCP().
If/when we (I?) implement Nagles algorithm in the driver, this issue will be resolved.
FYI, try this...
...to setup the "next" callback for loadimages. In JavaScript, as most event driven object-oriented languages, callbacks are objects, and you can't always pass params when "assigning" the function object to an event callback. The fact Opera interpreted the "string" 'loadImages()' as a callback function object is a lucky break, but I wouldn't expect that to work in most browsers.
That works like a champ! Thanks!
-Phil
If you don't mind indulging me, please give the site another try. I've made a temporary change to txTCP to test your idea:
Thanks,
-Phil
-Phil
Done.
1 - Picture only - no RST this time, everything ok.
2 - Main pake, loads fine. Js default disabled, no pictures.
3 - Main page, JS enabled. Work just fine, all pictures eventually loads (quite slow).
Your trace looks better?