After discovering this I decided I wanted to have the spinneret as the core for my version of this project instead of the Arduino. I've begun porting the Arduino code to Spin and am able to request twitter searches via GET commands. Now I'm working on parsing the JSON responses, but I'm stuck on something. The W5100 driver is set up to copy everything that is currently in the W5100 to a local buffer, and the twitter JSON response is sent in multiple 1024 byte chunks (could be 12K of data).
Does it make sense to copy all of the JSON chunks into a local buffer and then parse it (I would need a huge buffer) or should I add to the W5100 driver so I can read data from the W5100 buffer as I'm parsing it (smaller buffer, but data might came in faster than I can parse)?