 |
|
 |
| Parallax Forums > Public Forums > Propeller Chip > PropTCP Beta - Now fully MIT Licensed (w/ AJAX Enabled HTTP Server Example!) | Forum Quick Jump
|
  |  Rayman Registered Member
        Date Joined Jul 2007 Total Posts : 3127 | Posted 12/8/2009 6:37 AM (GMT -7) |   | | | |
 |  BR Omelette du fromage!

       Date Joined Dec 2008 Total Posts : 42 | Posted 12/8/2009 6:42 AM (GMT -7) |   | Oh, jeez. Now I've GOTTA get off my rear and put that YBOX2 kit together...
Thanks for this, Harrison. | | Back to Top | | |
  |  jazzed Registered Member
        Date Joined Jan 2008 Total Posts : 3125 | Posted 12/8/2009 8:47 AM (GMT -7) |   | | Thanks Harrison. Do you have a DHCP example? | | Back to Top | | |
 |  Bill Henning Registered Member

       Date Joined Sep 2006 Total Posts : 1997 | Posted 12/8/2009 4:59 PM (GMT -7) |   | Thank you Harrison for MIT'ing your PropTCP!
I just took a quick glance at the driver, and I have a couple of questions:
1) You require a pin for INT, but it does not seem to be used anywhere... must it be hooked up?
2) You provide a 25MHz clock to ENCJ, what changes, if any, are needed if a 25MHz crystal is provided for the ENCJ? (other than removing the prop generating the 25MHz clock)
I am trying to see if I can use your stack with just four pins connected: /CS, CLK, DO, DI
Best Regards,
Bill
Harrison. said...I would like to announce that PropTCP is now completely MIT licensed. I have decided that it's in the best interest of the Propeller community that all code be released under the MIT license. GPL compliance was impossible to enforce anyways, especially for small projects like PropTCP (violators probably wouldn't have cared about licensing anyways). PropTCP now ships with a simple HTTP server example which only requires an ENC28J60 interface (configured for the standard YBOX2 hardware). The server supports 3 'simultaneous' connections, although it can only actually service one request at a time (the other requests are queued). The HTTP server serves up an AJAX enabled page which constantly displays a random number from Propeller. It also supports downloading of the Propeller HUB ram for speed test purposes. I am currently working on porting an older HTTP server example which serves files off of a SD card (it also has a 'full' featured file manager for file uploads and downloads). I have attached my latest version. It is still in beta, but I am actively testing it on multiple Propeller boards to ensure that it is stable before I post it to the OBEX. I would appreciate any feedback anyone has. There have been many updates to PropTCP over the past year. Below is a summary of changes: + Flexible Buffers - Buffer arrays are now supplied by the user application, which allows for different buffer sizes per socket. Buffer sizes must still be powers of 2. + TCP Retransmit - The sockets layer now performs retransmission whenever a packet is lost. This results in very reliable communications. Unfortunately this also decreases thruput on high latency links. + RST Timeout - All socket states now have a timeout to prevent lockup issues. + Thread Safety - Public methods are now all thread safe. This means you can call them from different cogs without the previous lockup and corruption issues. + Handle Validation - Socket handles are now validated using socket state information. + Improved Error Codes - Better error codes are now returned. + Speed Improvements - The stack can now transmit data at up to 70KBytes/sec. This is dependent on link latency and buffer sizes. + Smaller Code - I refactored a bunch of code to improve efficiency. This in turn reduced compiled code size, and increased speed by roughly 15%. + MIT License - The code is now MIT licensed for OBEX compatibility!
Here's a short listing of examples to-be-released: + HTTP Bootloader - Program your Propeller over the internet (similar to Darco's YBOX2 bootloader, but hardware independent) + HTTP Webserver - A webserver with a SD card filesystem, web based file manager, and user loadable SPIN binary applications (should be better than the Parallax PINK...hopefully) + DNS Resolution - Resolve domain names (using TCP-based DNS queries w/ OpenDNS)
www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95 Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller | | Back to Top | | |
   |  Harrison. Registered Member

       Date Joined Jul 2004 Total Posts : 474 | Posted 12/8/2009 6:48 PM (GMT -7) |   | Thanks for the kind words everyone. Below are my answers to the questions posted above.
Bill Henning said... 1) You require a pin for INT, but it does not seem to be used anywhere... must it be hooked up? The INT (interrupt) pin isn't being used right now. Just pass in -1 (or any other number) for now. I will likely remove that argument before I post the stack to the OBEX.
Bill Henning said... 2) You provide a 25MHz clock to ENCJ, what changes, if any, are needed if a 25MHz crystal is provided for the ENCJ? (other than removing the prop generating the 25MHz clock) Pass in -1 for the xtalout pin number if you don't need the 25MHz clock. No other hardware or software changes need to be made.
Bill Henning said... I am trying to see if I can use your stack with just four pins connected: /CS, CLK, DO, DI Yeap, those are the only pins that need to be connected if you don't need the 25MHz Prop generated clock. Most of my boards only use those four standard SPI pins.
Dr_Acula said... Can I ping your board? Can I log into your board, ie I send a few nominal bytes and it sends me back something like 'hello, you are now logged in' The stack doesn't currently support icmp ping, but that isn't difficult to add. I could probably whip up a quick demo running http and telnet. In fact, one of the earliest tests I did back in 2007 was to run Donglebasic (serial version of Femtobasic) over TCP w/ telnet.
jazzed said... Thanks Harrison. Do you have a DHCP example? Unfortunately I don't have any DHCP code yet. I can't figure out a good way of integrating it without using massive amounts of code space. If you really need DHCP then you might want to take a look at Darco's DHCP code for the YBOX2. | | Back to Top | | |
  |  T Chap Registered Member

       Date Joined May 2006 Total Posts : 1766 | Posted 12/9/2009 8:22 PM (GMT -7) |   | | Thanks Harrison for posting this. I have been wanting to experiment with this for a while. Seems like a lot of fun. | | Back to Top | | |
 |  Patrick1ab Registered Member
        Date Joined Oct 2009 Total Posts : 112 | Posted 12/10/2009 3:12 AM (GMT -7) |   | Hi Harrison!
I was also working on a simple http server: http://forums.parallax.com/forums/default.aspx?f=25&m=406015
The structure of my code isn't very nice and uses a lot of memory due to the huge bufferspace, but it's pretty fast: 90 kBytes/s @ 80 MHz Propeller clock frequency 106 kBytes/s @ 100 MHz | | Back to Top | | |
  |  Patrick1ab Registered Member
        Date Joined Oct 2009 Total Posts : 112 | Posted 12/10/2009 5:14 PM (GMT -7) |   | I think I found a bug:
The dec function in the api_telnet_serial isn't working correctly, although the code of this function hasn't changed compared to previous versions of PropTCP.
I found out by chance, while I tried to modify my webserver to work with the newest version. | | Back to Top | | |
 |  localroger WTF? OMG! LOL

       Date Joined Mar 2009 Total Posts : 764 | Posted 12/10/2009 6:57 PM (GMT -7) |   | Thanks, Harrison. This is a very positive thing for the Propeller.
I started working on a stack of my own mainly because of the license problem. I will probably finish it because with your changes our functionality has diverged pretty far too. I am using the Spin object model to do code memory optimization that would probably save about 4K of Hub RAM if I just copied your logic. (If you define an array of Socket objects, you can get rid of *all* the dereferencing e.g. socketdata[pkt][position_in_pkt] stuff. That saves an AMAZING amount of code Hub RAM.)
I also want to optimize mine to work right with lots of fast, short connections, because that's how I have to deal with some embedded controllers I use that I can't reprogram if I want decent data bandwidth. That's how I discovered the timeout problem, something that you might notice once every day or two in a normal application was showing up every five minutes for me because my app requires opening a connection, doing your stuff, closing it, and then repeating several times a second. With the default timeouts in the usual TCP stacks and no option for UDP in the pre-programmed other device, it's the only way to get throughput. But I'd be the first to admit it's a specialty app and what I'm going to do to make it work might not be anybody else's cup of tea. | | Back to Top | | |
 |  Harrison. Registered Member

       Date Joined Jul 2004 Total Posts : 474 | Posted 12/10/2009 7:54 PM (GMT -7) |   | Patrick1ab said... The structure of my code isn't very nice and uses a lot of memory due to the huge bufferspace, but it's pretty fast: 90 kBytes/s @ 80 MHz Propeller clock frequency 106 kBytes/s @ 100 MHz You were probably using a very old version of the stack that didn't support TCP retransmit. Basically it just sent out packets blindly, as fast as it could go, which is why you saw pretty nice speeds. The latest version waits for ACKs after every packet (in fact, it sends a data packet and then an empty packet to force the remote side to ACK). Waiting for ACKs adds in a few ms of delay, which slows down the effective thruput. I think the benefit of having reliable transfers is worth the speed loss. I really wish the Propeller Tool supported conditional compilation, which would allow me to make TCP retransmit optional.
Patrick1ab said... The dec function in the api_telnet_serial isn't working correctly, although the code of this function hasn't changed compared to previous versions of PropTCP. Could you post or send me your code? My demo webserver included in the first post uses .dec() to send the random number. You could be forgetting to flush the transmit buffer before closing the connection.
Toby Seckshund said... You can clock the Prop from the ENC, rather than the other way round. At wake up the ENC puts out 25MHz/4 = 6.25MHz, just right for 100MHz with a pllx16. Or the output divider could be set to /2 and give 12.5MHz with pllx8. That's a great idea! When I first wrote the stack (Dec 2006) the Propeller hadn't been tested at speeds above 80MHz. Seems like everyone is running the Propeller at 100MHz these days (in fact, I ordered ten 6.25MHz xtals last night).
localroger said... I started working on a stack of my own mainly because of the license problem. I will probably finish it because with your changes our functionality has diverged pretty far too. I am using the Spin object model to do code memory optimization that would probably save about 4K of Hub RAM if I just copied your logic. (If you define an array of Socket objects, you can get rid of *all* the dereferencing e.g. socketdata[pkt][position_in_pkt] stuff. That saves an AMAZING amount of code Hub RAM.) Very Interesting. The beta release posted in the first post switched from using massive pointer dereferencing to using direct arrays. I moved the packet array into the socket layer, and switched to using separate arrays for all the socket state variables. This saved a couple hundred longs (while increasing speed a bunch), but I don't think it saved as much as 1K longs. I also no longer have to do complicated pointer math to get to the socket state variables.
localroger said... I also want to optimize mine to work right with lots of fast, short connections, because that's how I have to deal with some embedded controllers I use that I can't reprogram if I want decent data bandwidth. That's how I discovered the timeout problem, something that you might notice once every day or two in a normal application was showing up every five minutes for me because my app requires opening a connection, doing your stuff, closing it, and then repeating several times a second. With the default timeouts in the usual TCP stacks and no option for UDP in the pre-programmed other device, it's the only way to get throughput. But I'd be the first to admit it's a specialty app and what I'm going to do to make it work might not be anybody else's cup of tea. I ran into those connection issues about 8 months ago when I was working on a project that used a bunch of small short term connections. It became more apparent over high latency / lossy networks (like wireless). I had to add a bunch of timeouts and retransmit code to make communication reliable. The current beta should be very resilient to connection problems. I use it in a couple of mutli-cog projects, with long term and short term connections and haven't seen any issues yet.Post Edited (Harrison.) : 12/11/2009 3:00:54 AM GMT | | Back to Top | | |
 |  localroger WTF? OMG! LOL

       Date Joined Mar 2009 Total Posts : 764 | Posted 12/11/2009 7:34 AM (GMT -7) |   | Harrison: The beta release posted in the first post switched from using massive pointer dereferencing to using direct arrays. I moved the packet array into the socket layer, and switched to using separate arrays for all the socket state variables. This saved a couple hundred longs (while increasing speed a bunch), but I don't think it saved as much as 1K longs. I also no longer have to do complicated pointer math to get to the socket state variables.
You might want to look into this trick since it saves even more speed and RAM; the principle is that you create a socket object with its own data and code, and in the obj block declare an array of them:
con
max_sockets = 5
obj
nic := "driver_enc28j60"
sck[max_sockets] : "socket" As elsewhere, the different instances of socket.spin share DAT but have their own VAR areas. This means that socket.spin code doesn't need to use arrays at all -- the socket index is implicit in the function call...
This way, within the tcpmsg routine you don't need to use array references at all, just direct memory variables. I also added functions in the nic object to read and write common data structures to the packet buffer and SRAM without dereferencing to [pkt] every time.
var
byte state
long srcip
long seqnum
long acknum
'... etc.
obj
nic := "driver_enc28j60"
pub tcpmsg
case state
sck_closed: return
sck_listen:
if (hdrflags & tcp_syn)
srcip := nic.readpktlong(ip_srcaddr)
srcport := nic.readpktword(ip_srcport)
'...and so on
Of course you need to provide functions for sck to report its state and such to the parent object, and the nic object must be available to both the parent and socket (which is no problem since it uses DAT storage exclusively). In code blocks where I was just duplicating your functionality this was consistently shrinking it to about 60% of its original size. | | Back to Top | | |
  |  localroger WTF? OMG! LOL

       Date Joined Mar 2009 Total Posts : 764 | Posted 12/12/2009 6:55 PM (GMT -7) |   | | Harrison, if you're willing to refactor using my techniques it's a total waste of time for me to recode the object. My fundamental objective is reducing Hub RAM use. This is imperative if you're also including FSRW or some substitute for it in a project, and you want to have business logic on top of all that. | | Back to Top | | |
 |  Bill Henning Registered Member

       Date Joined Sep 2006 Total Posts : 1997 | Posted 1/4/2010 2:24 PM (GMT -7) |   | Ok, I am finally trying to get ethernet running on my props :)
I built a Ybox2, and a board of my design - the idea was that I can use the Ybox2 (known good design) to help debug my board.
Harrison or localroger - would you guys have any tcp stack debugging code kicking around that might help me?
Here's where I am at:
I downloaded the latest PropTCP beta from the first post in this thread, and made the following changes:
mac_addr byte $02, $01, $02, $03, $00, $05 ' device mac address, must be unique
ip_addr byte 192, 168, 1, 8 ' device's ip address ip_subnet byte 255, 255, 255, 0 ' network subnet ip_gateway byte 192, 168, 1, 254 ' network gateway (router) ip_dns byte 192, 168, 1, 254 ' network dns
I am trying to get the web server demo running.
On a Ybox 2, I start with:
{ybox2} sock.start(1, 2, 3, 4, -1, -1, @mac_addr, @ip_addr)
Maybe I did a bad job of soldering, because I don't even get a link light on my switch!
On my board, I start with:
{other} sock.start(23, 20, 21, 22, -1, -1, @mac_addr, @ip_addr)
Here at least I get a link light!
I'm now going back to debugging... www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95 Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller | | Back to Top | | |
  |  Bill Henning Registered Member

       Date Joined Sep 2006 Total Posts : 1997 | Posted 1/4/2010 2:30 PM (GMT -7) |   | Thanks - will try that right now!
I forgot to restore the 7 when it did not work on my board and I added back in the alternate start line for Ybox2.
(WOW that was a fast response!)
Harrison. said...The example_httpserver.spin object included in the zip file is built to run unmodified on a ybox2. The ybox2 hardware requires that the prop generate the 25MHz clock for the enc28j60, so you would want to run with the following (this is what example_httpserver.spin is pre-configured with): sock.start(1, 2, 3, 4, -1, 7, @mac_addr, @ip_addr) www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95 Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller | | Back to Top | | |
 |  Bill Henning Registered Member

       Date Joined Sep 2006 Total Posts : 1997 | Posted 1/4/2010 2:34 PM (GMT -7) |   | That did it.. IT WORKS ON YBOX2!!!!!!!!!!!!
Now I can compare it to my setup and find out where the problem is...
Thanks!
Harrison. said...The example_httpserver.spin object included in the zip file is built to run unmodified on a ybox2. The ybox2 hardware requires that the prop generate the 25MHz clock for the enc28j60, so you would want to run with the following (this is what example_httpserver.spin is pre-configured with): sock.start(1, 2, 3, 4, -1, 7, @mac_addr, @ip_addr) www.mikronauts.com Please use mikronauts _at_ gmail _dot_ com to contact me off-forum, my PM is almost totally full Morpheusdual Prop SBC w/ 512KB kit $119.95, Mem+2MB memory IO board kit $89.95, both kits $189.95 Propteus and Proteus for Propeller prototyping 6.250MHz custom Crystals run Propellers at 100MHz Las - Large model assembler for the Propeller Largos - a feature full nano operating system for the Propeller | | Back to Top | | |
 |  Patrick1ab Registered Member
        Date Joined Oct 2009 Total Posts : 112 | Posted 1/7/2010 6:15 PM (GMT -7) |   | Hi Harrison!
I tried to work on my web server again, but I still can't find the bug.
It works fine, if the data that is sent back to my computer consists only of a few bytes:
HTTP/1.0 200 OK
<html>Liste der Dateien auf dem Webserver: ------------------------------------
DRWHO11.AVI INDEX.HTM INTHEEND.MP3 LICENCE.TXT PRAKTI.JPG PROBE.TXT SCHUH1.M4A SCHUH4.M4A SCHUH13.M4A SCHUH16.M4A T-HOME.MP3 36GRAD.MP3 BOOTEX.LOG CONFIG.HTM DAVINCI.M4A SECURITY.HTM PIRATE.MP3
Dateien insgesamt: 17</html>
As soon as bigger files are being transmitted, this is what happens (first line is the reply of Fiddler2):
HTTP/1.0 200 This buggy server did not return headers
HTTP/1.0 200 OK Server: PropServer dy bgcolor="000000"> <head> <font color="65FF85" size="10"> Testseite </font> </he<html> <title>Testseite</title> <body bgcolor="000000"> <head> <font color="65FF85" size="10"> Testseite </font> </head> <font color="65FF65">
Das ist eine Testdatei mit der ich überprüfen möchte, ob die Übertragung von Daten der SD-Karte [...]
As you will see below, the header is corrupted and parts of the file are being send twice. It looks like a memory management problem, but how can that be possible?
This is an excerpt of the code:
if sdfat.popen(@fname,"r") <> 0 web.str(@http404) else web.str(@http200) web.str(string("Server: PropServer")) web.str(@crlf) web.str(string("Content-Length: ")) web.dec(size) web.str(@crlf) web.str(string("Content-Language: de")) web.str(@crlf) web.str(string("Content-Type: ")) web.str(type) web.str(@crlf) web.str(string("Connection: close")) web.str(@crlf) web.str(@crlf) if m==1 return 0 else repeat h:=sdfat.pread(@sendwebBuffer,BufferSize) if h=<0 quit else if h<BufferSize repeat n from h to (BufferSize-1) sendwebBuffer[n]:=32 sendwebBuffer[BufferSize]:=0 web.txdata(@sendwebBuffer,h) return 0
Basically this code is doing the following things: 1. Try to open the file; if it's not found, send an error message 2. else send the header (if m==1, the request was "HEAD") 3. send the file in pieces of BufferSize bytes (I set BufferSize to 4096) to the buffer
The subroutines "webserver" and "_webReadLine" are nearly the same ones you are using, except that I only allow one socket.
I can post the full code too, but at first I have to translate my comments into English  | | Back to Top | | |
 | 40 posts in this thread. Viewing Page : 1 2 | | Forum Information | Currently it is Thursday, July 29, 2010 5:23 PM (GMT -7) There are a total of 462,441 posts in 62,066 threads. In the last 3 days there were 90 new threads and 802 reply posts. View Active Threads
| | Who's Online | This forum has 20143 registered members. Please welcome our newest member, ME01. 57 Guest(s), 15 Registered Member(s) are currently online. Details John Abshier, Erik Friesen, RossH, Kevin Wood, simpsonmichael1, BradC, David Betz, Julian800, Martin Hodge, RDL2004, Harley, Sapieha, wiresalot, Ravenkallen, Tubular |
Forum powered by dotNetBB v2.42EC SP2.02 dotNetBB © 2000-2010 |
|
|