404 handling

Javalin

Hello!

Running MikeG's excellent multi-port web server code, with some modifications. I want it to return "err404.htm" when the user requests a missing file/page.

Modified the code as follows:

PRI StaticFileHandler(id) | fileSize, i, headerLen, temp, j, statuscode
    ''Serve up static files from the SDCard
    SDCard.changeDirectory(@approot)
    statuscode := 200
    
    'Make sure the directory exists
    ifnot(ChangeDirectory(id))
        'send 404 error
        statuscode := 404        
      
    ' Make sure the file exists
    ifnot(FileExists(Request.GetFileName(id)))
        'send 404 error
        statuscode := 404

    if statuscode == 404
        request.setfilepath(id,string("err404.htm"),10) 

and

' request.spin - test (very) code
PUB setfilepath(id, srcAddress, count)
    Set(@@name[id], string("err404"), 6)
    Set(@@extension[id], string("htm"), 3)    
    Set(@@filepath[id], string("err404.htm"), 10)

this works - if you run "getfilename(id)" you get the err404.htm page.

But.... The size got by

    SDCard.openFile(Request.GetFileName(id), "r")
    fileSize := sdcard.listSize

in the static file handler is 0, so the request stops there. If I request the file directly ie.. http://website/err404.htm - its fine.

Any ideas? Its like the code is checking for file existance somewhere else, but can't see it. Maybe the index's are wrong in the SD card?

James

Mike G

Why not extend the existing 404 handler?

PRI WriteError(id) | headerOffset
  '' Simple 404 error
  pst.str(string(13, "Write 404 Error",13 ))
  headerOffset := Response.BuildHeader(Request.GetExtension(id), 404, false)
  Socket.txTCP(id, @txdata, headerOffset)
  return

Javalin

Because I want to display a friendly page with images - not enough programming space and silly use of it really. Also IE doesn't display a 404 page < 500 or so bytes, just replaces it with its own rubbish one.

And why not really.....

Mike G

To extend mean to enhance an object. In this case the WriteError(id) method will encapsulate the custom error message as shown below.

I'm not sure why SDCard.readFromFile is returning zero bytes. As a work around, we can invoke the SDCard.readFromFile method to read the custom error file as long as the error file does not exceed the Tx Buffer. SDCard.readFromFile will only read to the end of the file.

PRI WriteError(id) | headerLen, fileSize
  '' Simple 404 error

  pst.str(string(13, "Write 404 Error",13 ))

  ' Build the 404 header
  headerLen := Response.BuildHeader(string("htm"), 404, globalCache)

  ' Open the custom error file and write to the txdata buffer after the header
  SDCard.openFile(string("error.htm"), "r")
  fileSize := SDCard.readFromFile(@txdata+headerLen, 1000)

  ' Sent the error file
  Socket.txTCP(id, @txdata, strsize(@txdata))
  pst.str(@txdata)

  'CLean up
  SDCard.closeFile
  SDCard.changeDirectory(@approot)
  return

We'll also need to update the BuildHeader() method of the Response.spin object. Comment the three lines at the bottom of BuildHeader() else the default error message will be part of the response stream.

  'if(statusCode <> 200)
    'bytemove(ptr, @genError, strsize(@genError))
    'ptr += strsize(@genError)

Javalin

>I'm not sure why SDCard.readFromFile is returning zero bytes
that does seem the crux of the issue. Im going to put Kye's version 2.0 SD card driver in place tonight and see if its a file system driver bug. Can't see why though. More testing required I think,

thanks for the code - i'll try the

fileSize := SDCard.readFromFile(@txdata+headerLen, 1000)

and see. I've changed your WriteError function to be a more generic WriteHttpStatus function - allows the calling function to set the http_status to allow for 404, 500, 501 etc.

Out of interest, and slightly off topic, when you implemented your spinneret.servebeer.com website, you've hosted your images elsewhere. Is that for a particular reason - i've done a similar site and the images work ok at home, and from another broadband connection, but not from corporate networks. Tried changing the site to port 80, made no difference.

Cheers,
James

Mike G

Javalin, the intention is to let the Response.spin object handle the status codes. The framework is already in place with the BuildHeader method. We just have to add the header text and some conditional logic.

Out of interest, and slightly off topic, when you implemented your spinneret.servebeer.com website, you've hosted your images elsewhere. Is that for a particular reason

Because I needed more than 4 sockets to handle the requests and I wanted the tutorial to look nice.

i've done a similar site and the images work ok at home, and from another broadband connection, but not from corporate networks. Tried changing the site to port 80, made no difference.

Proxy...Firewall... HTTPS...

What do you mean, Corporate Networks? Can you view the image in your browser by entering the image URL (absolute address) in your browser's address bar?

Javalin

Corporate as in company, i.e. behind firewalls, proxies, IPS/IDS etc.

Not firewall or HTTPS as such though as images from other servers via HTTP work, proxy maybe. Some testing and planning to setup some logging tonight. Direct requests for the images don't work either. Hmmmm. Setting IE to use 1 connection per server rather than 25 (!) didn't fix it.

Fiddler see's a 200 response but then nothing. Like its send the header back, but ignored the data (image). File system bug again I ponder?

(ETA - Yes the BuildHeader is what i've updated, not clear earlier as I didn't have the source code to hand)

Javalin

fixed.

    ' query the card and get the file size
    sdcard.listEntry(Request.GetFileName(id))
    SDCard.openFile(Request.GetFileName(id), "r")
    fileSize := sdcard.listSize 

need to update the file system index before reading the file.

ta.

Mike G

Not firewall or HTTPS as such though as images from other servers via HTTP work, proxy maybe. Some testing and planning to setup some logging tonight. Direct requests for the images don't work either. Hmmmm. Setting IE to use 1 connection per server rather than 25 (!) didn't fix it.

Fiddler see's a 200 response but then nothing. Like its send the header back, but ignored the data (image). File system bug again I ponder?

If the images are on a web server other than the Spinneret, the HTTP request/response is to/from the web server with the images not the Spinneret.

Glad you got the file size working.

Javalin

>Glad you got the file size working.
thanks.

Still problems with images served from the spinneret. Not sure yet 100% where this lies. It works 100% ok on LAN at home, and from a friends broadband. But the two systems at work don't load or don't completely load the images. But the Ajax code, and the htm display fine. I wonder about encoding or file size maybe, with different versions of IE and via a proxy server?

The index.htm page is itself and one image, so should be a max of two connections - shouldn't be a many (>4) connection issue.

As a test I put a link to a wordpress hosted (nginx server) image, and that works. I've taken some captures with fiddler and the header/responses are different between the wordpress image and the spinneret image. I've also put logging onto the log.txt file...

It doesn't appear to be browser, as IE6 (eek!), IE8, IE9, tablet and phone browser work @ home or via home LAN....

Hmmm.

Mike G

Post your source code and an image.

Does this statement not apply anymore?

Out of interest, and slightly off topic, when you implemented your spinneret.servebeer.com website, you've hosted your images elsewhere. Is that for a particular reason - i've done a similar site and the images work ok at home, and from another broadband connection, but not from corporate networks. Tried changing the site to port 80, made no difference.

Are the images stored on the Spinneret SD card?

Javalin

yes Images on the SD card. Yes that statement is still accurate, though I've changed it back to port 5000 now.

Javalin

so it seems to be size related. small images / text are ok, but larger content seems to get stuck.

there shouldn't be a difference on sending small (50kb) vs large (150kb) should there? haven't seen anything in the http protocol on it....

spin code obviously working well as its just via corporate networks, local lan and remote broadband are ok. packet size reduction helped a bit but not a lot.

site is : http://jabhost1.dynalias.org:5000/

issue can be seen on the left image on main page.

james

Mike G

I receive the following error message.

The image http://jabhost1.dynalias.org:5000/spin2a.jpg cannot be displayed, because it contains errors

I've seen this error before, IIRC, the context-type does not match the file type. I've also seen where folks change the file extension without converting the image.

Zip and post your source code include the HTML and images.

Javalin

will do. works here and it is a jpg. what browser?

does the one on the bottom of the pachube page work for u?

Mike G

does the one on the bottom of the pachube page work for u?

I don't know what you mean

Javalin

the jpg image?

lol - google have just found the site and read the robots.txt file. its been up for weeks but not advertised anywhere till just now.

i see ur using firefox browser though.

Javalin

Mike,

Attached. I added a pause(5) to the send loop - seems to help with the "bigimg.jpg" on the LAN. I wonder if the indirect.txTCP is a blocking call? (i.e. it waits till the ASM is finished)

James