Wiznet pusing data to an SQL table
Igor_Rast
Posts: 357
I there ,
I was wondering if anyone had any experiance with pushing data to an sql database table tru the wizet (data logger )
sure it must be possible and someone already done it , but cant find the info on that
Would love some help
I was wondering if anyone had any experiance with pushing data to an sql database table tru the wizet (data logger )
sure it must be possible and someone already done it , but cant find the info on that
Would love some help
Comments
The service can be any HTTP gadget like a web page or a node.js web service.
you mean like using an raspberry pi and run somekinda php script that takes xml and sends that to an sql database ?
sounds plausible, or you mean something else
any code clues ? demo on a tread here somewere
thanks
if we request a xmltemp from the wiznet at its ip
It will trow out this xml to the given ip with the requested values , in this example those values are Settemp, Setlv etc
now it seems to me the easy way is to change the xml that gets spit out on request , so that this does the SQL query ( read, write etc )
and let it submit it at a given rate (automated)and not on a request (get)
Just dont have the idea how it should be ,or am i seeing the hole thing the wrong way
if the xml can change to an sql query i woulded need the php at all right ?
Thanks in advance
full code
versie14.spin
EDIT: Updated below
Guess it needs to do something like below
0. define values , or just add them to the command
$dbhost="ip of database";
$dbuser="db username";
$dbpass="dbpass";
$dbname="xxxx";
1. connect to DB
[PHP] function connectToDB() {
global $link, $dbhost, $dbuser, $dbpass, $dbname;
($link = mysql_pconnect("$dbhost", "$dbuser", "$dbpass")) || die("Couldn't connect to MySQL");
// select db:
mysql_select_db("$dbname", $link) || die("Couldn't open db: $dbname. Error if any was: ".mysql_error() );
}
[/PHP]
2. Update values to table
[PHP]function newdata($sensor1, $sensor2,$sensor3,$sensor4) {
global $link;
$query="INSERT INTO Users (sensor1, sensor2,sensor3,sensor4) VALUES('$sensor1', '$sensor2','$sensor3','$sensor4')";
$result=mysql_query($query, $link) or die("Died inserting username info into db. Error returned if any: ".mysql_error());
return true;
} // end func newUser($username, $pass)[/PHP]
UPDATE :
Ok , ive been php coding a bit, and i think this should work .
i Think then the php code below should have to be spit out of the wiznet the way the xmltemp did , so on request we get an php script
and the script updates the variables to the sql database. ( php script works in a brouwser)
In this configuration i will need a middle man ( raspberry pi) to run the script request to get the database updated.
or is there some way else to not need the middle man ,
Lets say the database to be used is located on a much larger server (my high schools) so i will not want this database to be running a small php script ,so thats why i need the middle man (pi) or the prop sending direct to the database
bye the way , i cant seem tho change the localhost in my php script to an ip adres, it will not work , only does on localhost . have another test db here but cant connect , any ideas ?
[PHP]<?php
($link = mysql_pconnect("localhost", "db_user", "db_pass")) || die("Couldn't connect to MySQL");
// select db:
mysql_select_db("groendak", $link) || die("Couldn't open db: groendak DB. Error if any was: ".mysql_error() );
$query="INSERT INTO meeting (sensor1, sensor2, sensor3, sensor4) VALUES('sensor1', 'sensor2','sensor3','sensor4')";
$result=mysql_query($query, $link) or die("Died inserting sensor readings into db. Error returned if any: ".mysql_error());
return true;
?>
[/PHP]
Thanks,
Create a php page check ,(last update ) those few php lines should do it (tested)
Im now making all changes nececary to load it up in a prop , ( was a bit dusty , long time )
but i still had a question
after the parse the xml (working on that) , that the prop does i understand , but what would be calling that.
should i use a raspberry pi or another computer to start the parse of the php page.
Example
pi -->requst xml --> prop
prop --> send back php to pi
pi --> sends the received data to the database ( other pc in future , now localhost)
Its seems a fairly simple pice of php code , maybe we theres a way to get that
prop --> on time interval --> get sensor data and send --> direct to database (by ip /not localhost )
so there is no need for a loose php script running elsewhere
but cant seem to be able to sniff the transmitted data to the database to see how its build up with wireshark
Ill work on that for now, any advice would be apriciated
Igor
You've been given great advise by two different people.
Your ultimate goal is to take variable data from a spinneret and get that information into a sql database over the internet, or at least over a network of some kind- right? Mike G and Electrodude have definitely given sound advice. I think you're making it too complicated to be streamlined (I went through the same learning curve and frustration, confusion, etc!! )
1- Create a php script on your server. This script can employ any number of techniques to stuff data into a db, but I would recommend using the php PDO object. (I'm not completely sure it works with sql, but it definitely works with mysql)
* I would NOT recommend using "magic quotes" and all that Smile- it won't be supported or included in future releases of php.
2- Use the spinneret to send an HTTP POST to the php script. The name/value pairs will need to be converted to ascii in order to do this since HTTP is ascii driven.
3- Use include files with the php script to handle errors and send appropriate responses back to the spinneret to let it know whether the POST was successful or not. Retry or dump the data as required.
4- Forget about xml for this project. Using the PDO object and a sql db means there is no real reason to even consider sending data to the server in an xml format unless you are just trying to get some good exercise in data manipulation.
I agree with you , i did get some good advice from the best coders around, so i guess its best for me to listen to it +
so i will go with tht php page on the server approch to get it working , still need to make some changes and not everything going like i want it yet
the PDO part, i havent heard of that ( did a search just now) so that can take me some time
the php page that i have posted above does doe the job
If i open the php page in a brrouwser on a pc , the data in it gets updated to the sql database .
the data is now sensor1 sensor2 etc. that i will have to change still to get it asci converted and spit out by the get/post command ( was the xml message before)
should be doable with the parser function
indeed no need for an xml , only php page to update with the right data
on the sql sever side , i guess the php page will need something like
setInterval("getRequest('xml-proxy.php', 'placeholder')", 1000);
with the xml-proxy.php changing in the php code above that updates the database. so that it keeps updating at a given time
am I seeing it clear ?
You guys I realy realy apriciate your help ,Thanks, makes my day
Igor
Using the RTC on the spinneret, you can set it up to fire off the POST with the most current data whenever you want on a scheduled basis: For example 1 time every hour.
To get the latest data on demand, you could do as you've described: When the spinneret receives a GET request (or even a POST) to serve up a page, it instead (or also) automatically fires off the POST with the latest data.
Sounds to me like you're getting there. Good luck and keep us POSTED on progress! har har!
But im having some thinking about that
when i let the prop do an automated post of the php page, there should be a browser open somewhere to receive the php code and transmit the data to the DB
So in other words there will always be a brouwser open ?
EDIT: on second thouth
or I will use a proxy code and put that on the server
I guess i can make a second page kinda the interface. will be needing that in the end anyway to see the data in the db
Great guys , im finding solutions trying to explain , i love it when it goes that way
Ill definitly keep you posted
Igor
Then POSTing from the Spinneret should be very easy.
1 , when building the php in the dat section to get spit back out by the parser
i cant seem to find a way to make it work yet , the variables alligned so i can change them with another pub , like with the xll
currently looks like this , and the compiler making complaining about it . but like this i cant update the sensor1 2 3 4 variables . and the hole thing aint working
Wich is this php code that i broke down to the bare minimum to push the data to the database .
[PHP]<?php $link = mysql_pconnect("localhost", "db_user", "db_pass");
mysql_select_db("db_name", $link);
$query="INSERT INTO meeting (sensor1, sensor2, sensor3, sensor4) VALUES('sensor1', 'sensor2','sensor3','sensor4')";
$result=mysql_query($query, $link);?>
[/PHP]
problem 2 , somehow i cant seem to connect directly to the propeller
when i type the ip and say ask for the xml so example http://192.168.2.16/xml id doesnt give me that , nor with the :80 etc
stangly the html5 graph i used as underlaying does manage to talk to the prop .and get the xml variables
a proxy page does manage to connect to it by hostname , but via brouwser not . cant seem to figure out why
guess ill have to grab a fresh copy of Mike G code to start all over with the drivers , been some changes in the meantime also from what i have and what is uploaded now on google code.
Its a longg night
UPDATE ,
The second problem of not beeing able to connect directly to the propeller is kinda solved . well its a problem with my pc i guess
running ubuntu. something not right , but i guess ill have to figure it out later
Conclusion was , the Propeller is not responding to the ARP my host pc request asking to identiy itself ,
It does connect to it via my virtual server that i have running on the host pc , But it doesnt if i try from the host pc directly
After sniffing some wireshark the host pc seems that my host pc trows out an ARP request without its name in it ( marked in picture ) ,
so the arp request (Who has 192.168.2.18? Tell 192.168.2.5) for locating the prop (192.168.2.18) from the host pc is deffernt than the arp request for the propeller comming from the virtual server that is running on the same host pc
Conclusion = it has to bee some netwerk issues with the static ip set on my host pc , strangly the virtual server has almost the same settings ( but different ip of course ) but works fine
192.168.2.111 = Virtual pc running on Host (Ubuntu)
192.168.2.5 = Host pc (Ubuntu)
192.168.2.18 = ip given to propnet tru DHCP
Wireshark file of the Succesfull and Failed ARP request
wireshark.zip
I thouth id keep you posted , ,Ill be avoiding that problem for now and working on getting the data parsed out
Igor
I have a login page where you need to enter the ip adres , (and sample time interval = not working yet) so the setup proxy knows where to find the propeller
then it send the old fashion xml request , same kind used in the html5 demo te retreve the sensor readings
till here working fine ( i can display the received variables on the html page with the <span id = lvmax> </span> function
but the variables are also needed to get them posted to the db , but cant seem to figure out how to get that with javascript
Would love some help with that , its kinda the you have to know that trick problem
anyway , the pageg look like below for reference , looking pretty solid still. just a few more tweeks , and the controller is ready for logging data to the db
First connecting page where ip needs to be enterd ; ( named index2.php)
[PHP]<?php
// if "Start Data Transfer" is clicked, Check for IP, then start loop to get data in sql_db
if(isset($_POST["submit"])) {
include_once("config.php");
$_SESSION = $_POST["ip1"]. "." . $_POST["ip2"] . "." . $_POST["ip3"] . "." . $_POST["ip4"];
$time_ = $_POST["$interval"];
$times = $time_ * 1000;
$_SESSION = $times;
// and finally forward to connected page
header("Location: middleman.php");
} else {
// Not connected yet , display contect settings page
doIndex();
}
function doIndex() {
global $messages;?>
<!DOCTYPE html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />
<meta name='HandheldFriendly' content='tru'>
<meta name='viewport' content=' width=device-width, maximum-scale=1.0, user-scalable=yes'>
<meta name='description' content='An ICE Control Production.' >
<link rel='stylesheet' href='style.css' type='text/css' media='screen' />
</head>
<body>
<header id='headerbar' class='bar'>
<p class='copyright'>© 2014</p>
</header>
<div id='titlebox'>
<b><h3>Control Page </h3></b>
</div>
<div id='form2'>
<form action="<?php print $_SERVER["PHP_SELF"]; ?>" method="POST">
<div id='urlbox2'>
<table>
<tr>
<td style="padding-right:23px;padding-left:23px; ">Controller IP : </td>
<td> <input type="text" name="ip1" value="" maxlength="3" size="3">.
<input type="text" name="ip2" value="" maxlength="3" size="3">.
<input type="text" name="ip3" value="" maxlength="3" size="3">.
<input type="text" name="ip4" value="" maxlength="3" size="3">
</td>
</tr>
<tr>
<td style="padding-right:23px;padding-left:23px; ">Sample Interval (sec) : </td>
<td> <input type="text" name="interval" value="" maxlength="5" size="5">
</tr>
<tr>
<td><input name="submit" type="submit" value="Start Data Transfer" style="background-color:green; border:2;border-color:black" ></td>
</tr>
</table>
</div>
</form>
<?php if($messages) { displayErrors($messages); }?>
</div>
</body>
</html>
<?php
}
?>[/PHP]
The connected page that keeps the communication feed up (middleman.php)
[PHP]<?php
include_once("config.php");
$target = $_SESSION;
// IF Presed stop data transfer to DB
if(isset($_POST["submit"])) {
header("Location: index2.php");
} else {
doIndex();
}
function doIndex() {
global $messages;?>
<!DOCTYPE html>
<head>
<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />
<meta name='HandheldFriendly' content='tru'>
<meta name='viewport' content=' width=device-width, maximum-scale=1.0, user-scalable=yes'>
<meta name='description' content='An ICE Control Production.' >
<link rel='stylesheet' href='style.css' type='text/css' media='screen' />
<script type='text/javascript' src='graph.js'></script>
<script type='text/javascript' src='loop.js'></script>
</head>
<body>
<header id='headerbar' class='bar'>
<p class='copyright'>© 2014</p>
</header>
<div id='titlebox'>
<b><h3>Control Page </h3></b>
<h2><?php print $target; ?></h2>
</div>
<div id='form2'>
<form action="<?php print $_SERVER["PHP_SELF"]; ?>" method="POST">
<div id='urlbox3'>
<table>
<tr>
<td style="padding-right:23px; ">IP: </td>
<td> <h2><?php print $target; ?></h2>
</td>
</tr>
<tr>
<td> </td><td><input name="submit" type="submit" value="Stop Data Transfer" style="background-color:red; border:2;border-color:black" ></td>
</tr>
</table>
<tr>Connected to Controller ?</tr>
</div>
</form>
<div id='graphpanel1'>
<table border="1">
<tr>
<td colspan="2"><h1>Temperatuur</h1></td>
</tr>
<tr>
<td>Huidige</td>
<td><div style="padding-left: 6px;"><span id='placeholder'></span> ºC</div></td>
</tr>
<tr>
<td>Max</td>
<td><div><span id='tmax'></span> ºC</div></td>
</tr>
<tr>
<td>Min</td>
<td><div><span id='tmin'></span> ºC</div></td>
</tr>
</table>
</div>
<div id='benaming1'>
<p>Temperatuur ºC</p>
</div>
<div id='graphpanel2'>
<table border="1">
<tr>
<td colspan="2"><h1>Luchtvochtigheid</h1></td>
</tr>
<tr>
<td>Huidige</td>
<td><div style="padding-left: 6px;"><span id='placeholder2'></span> %</div></td>
</tr>
<tr>
<td>Max</td>
<td><div><span id='lvmax'></span> %</div></td>
</tr>
<tr>
<td>Min</td>
<td><div><span id='lvmin'></span> %</div></td>
</tr>
</table>
</div>
<?php if($messages) { displayErrors($messages); }?>
</div>
</body>
</html>
<?php
}
?>[/PHP]
the small javascript that keeps the loop alive in the middleman page (loop.js) Than it calls the proxy , same as used before in html5graph demo , but in the end it has the push to database command.
wich is suppose to push the received xml values to the db but cant get them in the php variables (loop-porxy.php)
[PHP]<?php
include_once("config.php");
$urlstart = "http://";
$urlend = "/xmltemp";
$target = $_SESSION;
$url = $urlstart . $target . $urlend;
$ch = curl_init();
curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_exec($ch);
if(!curl_errno($ch))
{
//Content type should be XML
header ('Content-type: text/xml');
}
else
{
echo 'Curl error: ' . curl_error($ch);
}
curl_close($ch);
push_data_to_db($temp, $humidity, $tmax, $lvmax);
?>[/PHP]
[PHP]function push_data_to_db($sensor1,$sensor2,$sensor3,$sensor4){ // Connect & push data to DB
$link = mysql_pconnect("localhost", "db_user", "db_pass");
mysql_select_db("groendak", $link);
$query="INSERT INTO meeting (sensor1, sensor2, sensor3, sensor4) VALUES('$sensor1', '$sensor2','$sensor3','$sensor4')";
$result=mysql_query($query, $link);
}[/PHP]
The graph.js does the parsing of the xml , seems that here i have to figure out how to get the received values in the sensor variables to push them to the db
strangly also , the ip adres that i put on the first page does go tru in the sesion , so i can use it for the url building in the proxy ,
but im having a hard time doing the same with the time interval so that can also change that on the setup page also.
same goes to the target im trying to print the ip on the middleman page , doesnt work , but the proxy somehow does have the variable to make the connection
Hope you can give me a hand with this php-javascript headache :P
(graph.js)
graph.js.zip
If you want the Spinneret to send data on a regular bases then create a timer in Spin.
May not work due to cross site scripting.
I do get the posting later from the spinret , would i be posting the xml ? . tryed making a php get out, but the formatting the data in the DAT was kinda tricky sorry im kinda confused with it all now .
with the current setup i know the page is getting uploaded with the fresh data, but i cant get that data back out . thats where im stuck
realy apriciate if you can suply an example of your proposal ,
Igor
May not work due to cross site scripting. --> that was the hole point of using a proxy in the first place right ?
Insert data
Select data
I'm sorry; I do not have the time to code an example.
the insert data etc are already working , its just that it update static values not the values retreived from the prop
[PHP]function push_data_to_db($sensor1,$sensor2,$sensor3,$sensor4){ // Connect & push data to DB
$link = mysql_pconnect("localhost", "db_user", "db_pass");
mysql_select_db("groendak", $link);
$query="INSERT INTO meeting (sensor1, sensor2, sensor3, sensor4) VALUES('$sensor1', '$sensor2','$sensor3','$sensor4')";
$result=mysql_query($query, $link);
} [/PHP]
THe $sensor1 , etc value in the statment, thats where i cant seem to update it
[PHP]push_data_to_db($temp, $humidity, $tmax, $lvmax);
[/PHP] runs as the last function in the loop-proxy.php
but here i cant seem to get the correct values in the $sensor1 etc variable
i know the javascript parser does update the values , but cant get them tru to the $php variable
Hope you can help in the little time