Using Multiple Sockets

Mike G · 2011-04-23 07:56

After considerable time and frustration I finally have multiple socket coordination. http://www.agaverobotics.com/spinneret/source/

Nothing fancy just the ability to handle up to 4 socket requests. I welcome any respectful comments and I will do my best to fix any bugs. I'm sure there are bugs. In a nutshell, I created a status register state machine and a FIFO buffer to hold socket IDs.

Secondly, I took some liberties in changing the Indirect Driver code.

There was a point yesterday where I considered giving up. I keep at it and learned a lot in the process.

Mike G · 2011-04-23 17:43

I created a demo site to test the multi-socket update. We'll see how well it works. Of course everything works great in the development environment. I'll leave it up until some time ma

Phil Pilgrim (PhiPi) · 2011-04-23 19:15

The one time I was able to connect to the AJAX page, it came up blank. Now the entire server seems to be hanging.

-Phil

Zoot · 2011-04-23 20:55

Works great until it doesn't (first views were great, then on a few subsequent visits the server hangs and delivers no content).

Mike G · 2011-04-23 21:03

Thanks for giving it a shot Phil and Zoot. I've reset the Spinneret more than a few times tonight. I'm not sure where the program goes south or why. I'll try to make sense of the logs and go from there.

If anyone has guidance to share, I'm all ears man... I've spent a lot of hours on this project.

Phil Pilgrim (PhiPi) · 2011-04-23 21:12

Mike, you're a trooper, man. Keep up the good work. I've gotten sidetracked from my own Spinneret programming. I'm really surprised and somewhat disappointed, given the Spinneret's initial promise and popularity, that there hasn't been more activity in this forum. Have people simply lost interest and given up? Or ... ?

-Phil

Mike G · 2011-04-24 19:06

Thanks for the encouragement. Spent the better part of today working on the Spinneret.

I cleaned up a few logic blocks and added more debugging output. This is what happens...

I can load the index.htm page 12 times in a row. The 13th time always locks the server. When I look at the request object (memory) , the expected alignment is off by several bytes.
I can load the ajax.htm page 10 times. The 11th times locks the server like index.htm. At least I can reproduce the issue.
A very noticeable time delay along with statuses 0x18 and 0x00 start popping up a few clicks before a crash.

At this point, I'm not sure if the issue is with my server code, the indirect driver itself, or my modifications to the indirect driver.

Latest code

Mike G · 2011-04-28 06:44

I narrowed down the problem. I believe it has to do with my initial understanding of...

OBJ
  Request[4]       : "StateData"

I thought that Request[4] would carve out memory for each of the 4 object DAT blocks. I just ran a little test this morning and it looks like DAT sections of the indexed objects all have the same memory address. I did not expect that. I'll do a little more testing and reading to make sure that my new observation is correct.

I'm actually surprised that the code worked for 61 requests before blowing up.

Mike G · 2011-04-30 20:53

This was one of the hardest pieces of code I've had the pleasure of troubleshooting. Of course, I created the bug. The bug was simply a PASM variable not reinitialized after first use.

The error did not show itself until a fix number of iterations down a specific path. I realized early that the error was not random but I could not figure out the source. After writing several debug templates I realized the error was dependent on one variable. It took me 6 days of troubleshooting for a two line fix. I fixed a lot of other stuff a long the way... I think.

I'll leave the Spinneret up for a while.
http://spinneret.servebeer.com:5000/index.htm

Mike G · 2011-05-01 21:03

Wow, the Spinneret served up 7943 files from 9:30am 5/1/2011 until now. The amazing thing is it's still up and running.

I ordered a second Spinneret today so I can leave one server online and still develop.

bsnut · 2011-05-02 00:00

I am glad you got your code working Mike. Are you going to post your code for us to see what you came up with?

Mike G · 2011-05-02 05:31

bsnut, follow the link on the first line of the first post.

Mike G · 2011-05-15 13:19

I updated the code a bit by commenting this logic block in the StatusMonitor method.

    if(closeWaitState > 0)
      'd := DecodeId(closeWaitState)
      if(id > -1)
        Socket.Close(id & tcpMask)

So, here's the deal. This code block is looking for a socket in the 0xC1 status. If it finds one, then the socket is forcibly closed. That’s bad because the socket, in most cases, will close on it’s own. However, there is a chance that a 0xC1 will persist. That’s also bad because now you have a stuck socket and it can not be used until a restart. I have to fix that…

Anyway, I hit the spinneret index.htm with 40,000 requests and 5 concurrent users using Apache Bench.

ab -n 40000 -c 5 http://spinneret.servebeer.com:5000/index.htm

No locks or squawks. However, I have been able to cause a stuck 0x1C status holding down ctrl-F5 (refresh) in Firefox. Like I said, I have to fix that.

Mike G · 2011-05-17 06:16

One more bug fix. Somewhere along the line, I messed up the file not found logic. The Spinneret locked whenever a crawler hit the Spinneret looking for robots.txt.

Mike G · 2011-05-25 19:09

TCP is serving web pages on port 5000 through sockets 0, 1, and 2. UPD is on port 5050, socket 3.

Web site using TCP
http://spinneret.servebeer.com:5000/

UDP testing site. You can use this to test your UDP site too, just change the default IP and port.
http://www.agaverobotics.com/spinneret/udptest.aspx

Source is in the same spot
http://www.agaverobotics.com/spinneret/source/

I'll leave it up for a while.

Using Multiple Sockets

Comments