How to log data from web device into a server
Timothy D. Swieter
Posts: 1,613
The subject is kind of a generic question. Let me add some more details here.
I am creating a concept for a home based sensor system. The first sensor I am considering is a current sensor to measure my home electricity usage. The electronics would be made of a current sensor, Propeller and a WIZnet ethernet module.
I hope to have a home "server" or at least my laptop available to log data. The "server" may in fact be a Network Attached Storage (NAS) device that can run PHP/MySQL. I was thinking the AC current logger would send out a message via the ethernet every so often to a web page and then the data is logged into a MySQL table. Is this is a good idea? Is there a better method?
For the method described, is the best way to pass data to a web page to use the URL info such that you include the variables? Something along the lines of "test.htm?Variable1=1234@variable2=5678" is what I am thinking.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
I am creating a concept for a home based sensor system. The first sensor I am considering is a current sensor to measure my home electricity usage. The electronics would be made of a current sensor, Propeller and a WIZnet ethernet module.
I hope to have a home "server" or at least my laptop available to log data. The "server" may in fact be a Network Attached Storage (NAS) device that can run PHP/MySQL. I was thinking the AC current logger would send out a message via the ethernet every so often to a web page and then the data is logged into a MySQL table. Is this is a good idea? Is there a better method?
For the method described, is the best way to pass data to a web page to use the URL info such that you include the variables? Something along the lines of "test.htm?Variable1=1234@variable2=5678" is what I am thinking.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
Comments
You need to setup an HTTP server program on the server computer. Look up Apache. Then you can plug-in PHP into Apache (there is a specific procedure for that). Then, MySQL would be running alongside Apache.
You write the PHP script(s) to handle the data. Setup the Apache server in such a way that when you retrieve a PHP page from the computer, Apache runs the PHP interpreter, and runs the script. The script will automagically generate an HTML page and send that to the client.
So you may have a PHP file named "retrieve.php", and you would access it by typing "<server computer>/retrieve.php" in your browser and it would return an HTML page with a nicely formatted table or even a graph, of your house's current consumption over time, based on the data in the MySQL tables. You can probably even have "<server computer/retrieve.php?date=20090418" so you can retrieve data from a specific date.
You may have another PHP file named "logdata.php" but is requested only by the Propeller. The Propeller would construct a URL like "<server computer>/logdata.php?amperes=230×tamp=041820091210" and send that to the server. The PHP script will be able to extract the info and insert into the MySQL table via an SQL query.
It's the most practical Rube Goldberg machine yet!
Are there other ways you would propose to log the data for later retrieval and analysis? Perhaps the current monitoring device would have a SD card that logs the data to that device and only dumps it out over Ethernet(or USB) when a PC application requests it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
Your SD card idea is sound. It's a very good idea for the sensor node to be able to store the data locally, especially in the case the server computer goes out for some reason. Also, it relieves the burden of the server computer.
I've seen people host a (tiny) HTTP server on the microcontroller and you access the data by directly requesting from that microcontroller. I'm pretty sure the Propeller is more than capable of doing that. Although you can get away with just a TCP/IP stack.
I would either use an SD card or a high-capacity EEPROM attached to the Propeller, and a real time clock. The Propeller would take samples at specific time intervals, or if a significant event occurs (for example, the central air conditioner turns on and current spikes up), and record the ampere reading from the sensor and a timestamp to the mass storage device. The Propeller would also support network sockets (http://en.wikipedia.org/wiki/Internet_socket). The Propeller·creates a socket and listens for incoming connections.
Your PHP server would have a script like "retrieve.php" which contains code that creates a socket and connects to the Propeller. Then data transfer can occur, with the propeller do a dump on the mass storage device.·Then, that PHP script can do whatever it wants with the data, maybe store it in the MySQL database for archival purposes, etc.
So, in this setup, the Propeller becomes a server and your Synology station is the middleman talking to it via sockets. Note that MySQL works sort of the same way - the PHP script accesses the MySQL database via a socket (although PHP includes libraries that make MySQL access easy).
My home-brew home-security system has a web interface and tells me the status of the system as well as various other information like backup battery level, temperature, etc. I may even expand it later to monitor other gas levels. Currently this information is hosted on a PINK module, which I find perfect for this task. Not only do I have over 100 variables I can watch and set, but I can get e-mail updates (which I get on my cell phone as text messages) and I can also send/receive command information via UDP. I haven’t posted the project yet as I am in the process of redesigning it for the Propeller, but I thought I would mention what I was using for my purposes. Take care.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Chris Savage
Parallax Engineering
I've been learning to use Delphi For PHP to hopefully interface a Propeller chip to a web server eventually.
Delphi For PHP web site:
www.codegear.com/products/delphi/php
Currently I have it working with Apache(automatically setup when installing Delphi For PHP) and mysql .
Learning how to do the PHP scripts will be next.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Aka: CosmicBob
The last couple days have been busy. Next week I have a flight to the USA from HK so I plan to do more thinking and reading then and we will see what I come up with. Ultimately though I think it will be the actual experimenting and testing that will prove what works best. So it won't be until June that I can do that once I am moved and settled. In the mean time lots of research. Keep the inspiration coming.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
New ICON coming, gotta wait for the INK to heal, now we have colour!
Better yet why port http://www.uclinux.org/ to the prop?
-dan
The project will start with an AC current sensor. i don't think each device will be a web server per se. But there may be a master device that receives and logs data to memory or SD. Then that device can serve data or upload the data to a database/web site app.
If the web server is on my home network, I can make it a fixed IP address. However if I want to send data from the ethernet enable sensor to an external web site, what hiccups are there? I know there are some like the fact that my home may be on a dynamic IP and maybe even the web server itself. Would having a fix IP on the web server help when the web server is on the 'net?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
My setup looks like this:
The actual HTTP request looks like:
I highly recommend rrdtool (oss.oetiker.ch/rrdtool/) for logging and graphing the data. rrdtool uses a fixed size binary file for logging your readings and can perform various analysis functions. My PHP scripts directly call `rrdtool update` to log readings.
Right now I have an example up at hdpham.com/sensors/apart-indoor.htm. It's a simple setup that uses a Propeller to submit temperature readings every minute to a remote webserver.
Post Edited (Harrison.) : 4/24/2009 6:53:55 AM GMT
OK - I like the example page you have posted. If I leave the web page up for a while does it automatically update or reload? AJAX enabled graphs? The graph appears to be an image. Just wondering, it is not a requirement yet in my idea, but I like the cool factor of seeing the data stream in right in front of your eyes.
I looked briefly at rrdtools. At first glance it looks a like I need to some reading and get in on a learning curve in order to use it.
This project will be a fun learning project. I was thinking more of using a Google time line chart item like this: code.google.com/apis/visualization/documentation/gallery/annotatedtimeline.html since I think this graph is very functional. But then again, for getting started maybe that is going too far into the deep end of the pool.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com
Post Edited (Computer Geek 101) : 4/29/2009 5:30:10 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com