Wiznet official driver working (FlexC) (now with W6100 support!)
Rayman
Posts: 14,745
Spent a little time trying to port the Wiznet HTTP server for RP2040 to P2:
https://github.com/Wiznet/RP2040-HAT-C
Got it to the point now where it seems to be working.
Had to overcome some issues in FlexC like structures not initializing correctly and lack of strtok() function.
But, seems to be more or less working and does serve up the example web page.
I'm not 100% sure DHCP is working, probably isn't. Also, think I need to implement a 1 second timer still.
But, this is a pretty good start.
Update: Now have DNS, DHCP, FTP, and HTTP all working together, see post #38
1248 x 594 - 23K
640 x 480 - 87K
Comments
I messed up with the adapter board and it's too wide to work on P2 Eval, so had to use with my own board.
Also, the holes are too small in the adapter board to accept the Wiznet directly so had to use headers.
Need to fix that too...
Also, I need to remember to immediately save code as soon as it works.. too many times I’ve broken code before saving…
Good reminder for us all. Today I got the initial version of a program working and immediately appended _001 to the name and resaved it.
Awesome! Great work!
@ke4pjw Thanks. BTW: I looked at your Spin2 code for newer 6100 chips and then looked at the datasheet for 5500 and didn't see how to adapt it. So, gave this a shot...
Just got DHCP working. But now, I'm not sure I want to use it... Because now, I have to check to see what IP was assigned. For what I'm doing probably easier to just use a fixed IP.
Looks like Edge and Chrome support BMP image format. Not sure that was always the case, but thinking could be useful...
Could have P2 generate graphics as a BMP image and serve that up over HTTP...
If you use your router as a DNS proxy and it's smart enough, and if you make the P2 include the correct DHCP options, you should be able to access your P2 by hostname at something like
<hostname>.local.That'd be neat, but don't see a way to do it... I went to the router and was able to rename it Wiznet, but still can't get to it.
I see there is a local DNS list in the router, but this device isn't listed... I could add this device by IP address, but not sure that helps anything...
Tried to get it to serve up a .png file stored in HUB RAM and it wouldn't to it... Was getting garbage...
Took a while, but I see the code assumes that the data is stored on a data flash and needs to be copied into RAM to be used.
For some reason, they are using strncpy() to do this. This only works for text though, as soon as it came to a zero in the source, it stopped.
Seems like they are assuming that anything stored in memory is going to be text and anything binary would be on uSD or external flash.
Works now that I copy the data over a different way.
There is a problem with the strtok() function I found on internet, or maybe with the way I'm using it...
If I keep reloading the website it eventually fails because the heap runs out and the malloc function in it fails.
Can keep it going longer by increasing heap size by adding this:
enum { heapsize = 32000 }; //override the default heapsize to give more, if needed
But, that's not a good solution. Fortunately, @ersmith just updated github so that FlexC now has strtok().
I'll hope that works better. But, could be I need to be using free() in a different way after calling it...
The heap size is fixed at compile time, so malloc()/free() don't really do anything I don't think. For dynamic allocations, you have to use FlexC's stack based auto-freeing
__builtin_alloca(). It is some sort of concession to simplify garbage collection in FlexBasic. I didn't try very hard to understand.Of course malloc/free do something. That the heap is a fixed size doesn't mean that allocation can be known ahead of time. (It is fixed size on most every platform without OS/virtual memory, as it were.) If you do _gc_alloc_managed(), you get a garbage-collected allocation that is auto-freed when it is no longer used ("used" meaning that there is no pointer to it anywhere in hub ram or in the cog ram of the cog that is running the GC (which happens when a new allocation is to be made, but the heap is full))
__builtin_alloca() is, of course, a stack allocation that is inherently freed when the function returns.
Is there documentation?
PS: A fixed heap just seems will pointlessly consume space no-one will use. A fixed stack still makes sense though.
EDIT: Hang-on, are you saying that _gc_alloc_managed() uses heap space?
Of course it does? Where else would it go?
https://github.com/totalspectrum/spin2cpp/blob/master/doc/general.md#memory-allocation-and-management
I didn't know that function existed at all. I thought the GC code was a Basic thing.
It still seems backward to have a fixed heap size for all that. I've been solely using __builtin_alloca() to avoid it.
Now I wonder what the default size is ... huh, intriguing, changing the HEAPSIZE enum doesn't affect size of compiled binary. Yet, adding a malloc() does. ... Oh, right, it's like null-code removal. No heap allocation used so the heap isn't present at all.
I was originally shocked to see the binary change size. That's what got me looking for a workaround and I found __builtin_alloca().
The HTTP server can now serve .bmp files!
Thanks to @ersmith for fixing something in flexspin that was messing it up.
This only works with new source at github for now (see flexspin thread for how to get latest version, or wait for next official release).
Had to hack the parser a bit to add .bmp support by adding:
else if (strstr(buf, ".bmp") || strstr(buf, ".BMP")) *type = PTYPE_PNG; //RJA pretending bmp is pngWithout this, it sends a 10 byte header that gets put at start of file and messes image up.
The speed needs a great deal of improving, but at least it can now do what I want.
Got DNS working. Looks up ip address for www.wiznet.io using DNS
Guess could be useful...
Starting to clean up code and improve speed now...
Awesome @Rayman ! Something I learned quickly is that using javascript includes is the easiest way to create dynamic content from a little webserver on the P2. Let the client do the heavy lifting. Forget about creating dynamic html. Just use the DOM and javascript to your advantage. ie. Just generate javascript from .js requests to your server, where it is an include on the page you want to update content on.
That's probably a good idea... Instead of sending a bitmap, send the data to draw the image and have the host create the graph or what have you.
Tinkered with spi upload speed a bit, but it still takes ~2 seconds to transfer a 640x480 8bpp bitmap (302 kB). Guess that's ~150 kB/s or so. Seems like it should be faster...
Ok, seems the slow down is all the debug messages being sent over serial port...
Can turn most of them off (except ones I added) by commenting out:
#define _HTTPSERVER_DEBUG_But now, it's too fast it seems and doesn't work unless I lower the system clock to 160 MHz...
Guess there's more work to do...
Made it a bit faster and also so it works at up to 300 MHz.
Removed most of my debugging printf().
Used inline assembly for SPI (although didn't help all that much).
It can now load the 302 kB bitmap in about half a second.
Still seems like it should be faster than that though.
Maybe it's just some overhead in HTTP...
Nothing inherent in HTTP - your browser uses HTTP for every other server on the planet as well, and they manage much higher speeds. Could be a TCP problem, though.
Can you capture the connection using Wireshark and upload the capture here?
Some things of note that I had to do with my W6100 webserver to make it perform better........
On static images and CSS requests, I added additional headers to cache that information in the browsers.
resp_css_ptr := String("HTTP/1.1 200 OK", $0D, $0A, "Content-Type: text/css",$0D, $0A, "Cache-Control: max-age=604800", $0D, $0A, "Connection: close", $0D, $0A, $0D, $0A) resp_jpg_ptr := String("HTTP/1.1 200 OK", $0D, $0A, "Cache-Control: max-age=604800", $0D, $0A, "Content-Type: image/jpeg", $0D, $0A, "Connection: close", $0D, $0A, $0D, $0A) resp_png_ptr := String("HTTP/1.1 200 OK", $0D, $0A, "Cache-Control: max-age=604800", $0D, $0A, "Content-Type: image/png", $0D, $0A, "Connection: close", $0D, $0A, $0D, $0A) resp_ico_ptr := String("HTTP/1.1 200 OK", $0D, $0A, "Cache-Control: max-age=604800", $0D, $0A, "Content-Type: image/x-icon", $0D, $0A, "Connection: close", $0D, $0A, $0D, $0A)No need to request the same content over and over again.
I only use one socket on the W6100 for the http server. This can be problematic with multiple browsers hitting it at the same time. Browsers just love to pipeline these days. They will connect to a server and just wait for you to request something. This eats up the one socket I made available. I added code that forcibly closes the socket after the request is serviced. Browsers will immediately open a new connection. If the request doesn't send anything in a few milliseconds, I close that socket, too. Usually, the browsers give up at that point and don't try again, until the next request. I saw many people complaining about this problem in the Wiznet forums with no solutions. Luckily I am very familiar with http and browser behavior and was able to quash these problems through observing the traffic with Wireshark and coding around the problem.
Just me $.02
There's your problem. Constantly closing the connection kills performance - TCP starts out very slow by design, since it needs to assume the network is already very congested until it can prove otherwise in order to avoid congestive collapse. You need to be able to service more than one open socket at once if you want performance - sorry.
In any case, caching is a very good idea.
I haven't noticed any performance problems with my http server. The pipelining is great for "setup time" performance, as the TCP 3way handshake is latent, but does nothing for actual throughput once the connection is established. For serving up a status/configuration page on a small, embedded device with limited resources, it makes little sense.
The problem I was addressing is what happens when you have multiple browsers (ie 2 or more users) accessing the Wiznet example code. The first browser wins by gobbling up all of the sockets. (I believe the Wiznet http server uses multiple sockets) They will stay connected too. The other browsers get connection refused. This is because it pipelines every socket available. You can do that with Apache and IIS, because there are plenty of resources available to manage those connections that are doing nothing. When you only have 6 sockets available, you can't allow the browsers to run rough-shod over your 6 sockets. You have to smack them down for being resource hogs.
There are many, many people posting stuff just like this:
https://forum.wiznet.io/t/topic/6563
https://forum.wiznet.io/t/topic/5623
@ke4pjw Interesting links... The code I have now uses 4 sockets for HTTP, 1 for DNS, and 1 for DHCP.
The actual application I'm interested in doesn't need DNS or DHCP. I just want the P2 to deliver data as a web page.
So, I suppose I could use all 8 sockets, no problem. But, I haven't seen an issue yet.
I think I did see Edge downloading a webpage before I finished typing it out, which I'm not sure I like...
That seems to be an old thing: https://www.wired.com/2012/02/chrome-17-released-will-preload-autocompleted-urls-as-you-type/
I'll download Wireshark and see what it says...
Ok, here's the capture of downloading the 302 kB bitmap file with Wireshark.
Looks like 0.4 seconds to download.
Edge will absolutely download the page as you are typing in the URL, if you have visited it before. I laughed the first time I saw that. It makes for a good user experience.
That's ~ 6Mbps. I don't see a problem with that. If you send larger files you most likely will see that rise, as the algorithm that ramps up speed has a linear increase and exponential fallback.