Can I access information from web pages on my propeller?
Private19872
Posts: 61
I had an idea of getting information (primarily weather forecasts) on my propeller and then having it display them. I was wondering if it was possible to get the information from the website onto my propeller without having the propeller connected to my computer.I would imagine that it would involve the Wiznet Ethernet board. Thanks for answering, and if anyone could suggest a website to get the information from that would greatly be appreciated.
Comments
Open this in a browser, replacing YOURLAT and YOURLONG with your latitude and logitude:
View the source and search for <font color="#FF0000" size="1"><b> . The next number will be the current temperature. The number after the nth match will be the temperature in n-1 hours. If you want a field other than temperature, change the color to be the color of whatever field you want. To make this not fail in case they change the colors, find the first color before the title of the field you want and use that for the color.
Now you just have to write a propeller program to do what you just did yourself.
EDIT: There's also https://developer.forecast.io/ that gives you results in JSON. It's free if you do less than 1000 API calls per day, which should be plenty.
Example project
http://m.instructables.com/id/ESP8266-Wifi-Temperature-Logger/
A propeller could browse the internet with one of these - we just haven't written the code yet. It should be able to pull data out of the html text stream, eg the actual temperature buried on a weather webpage.
You can test a lot of things with a simple terminal program and talk to it one line at a time, and after a few instructions it dumps out data off a web page.
BTW, we are playing with extracting information from webpages at present in Tachyon although the emphasis to date has been serving them up. I have my own compact WIZnet modules with an integrated Prop chip so that you can talk to them serially.
See this thread here.
Alternatively you can use the base IOT5500 module with your own WIZnet drivers.
You can parse it as you go line by line etc then discard or cancel what you don't need. I notice that on the local government weather site that it has JSON and CSV files updated every 10 minutes, perhaps you have something similar.
Here's the first part of the file with the latest observations so it is very easy in Tachyon to have a defined word called "air_temp": that picks this information out seamlessly.
{ "observations": { "notice": [ { "copyright": "Copyright Commonwealth of Australia 2015, Bureau of Meteorology. For more information see: http://www.bom.gov.au/other/copyright.shtml http://www.bom.gov.au/other/disclaimer.shtml", "copyright_url": "http://www.bom.gov.au/other/copyright.shtml", "disclaimer_url": "http://www.bom.gov.au/other/disclaimer.shtml", "feedback_url": "http://www.bom.gov.au/other/feedback" } ], "header": [ { "refresh_message": "Issued at 11:31 pm EST Sunday 11 January 2015", "ID": "IDQ60901", "main_ID": "IDQ60900", "name": "Brisbane", "state_time_zone": "QLD", "time_zone": "EST", "product_name": "Capital City Observations", "state": "Queensland" } ], "data": [ { "sort_order": 0, "wmo": 94576, "name": "Brisbane", "history_product": "IDQ60901", "local_date_time": "11/11:30pm", "local_date_time_full": "20150111233000", "aifstime_utc": "20150111133000", "air_temp": 25.8, "apparent_t": 30.1, "cloud": "-", "cloud_base_m": null, "cloud_oktas": null, "cloud_type": "-", "cloud_type_id": null, "delta_t": 2.1, "dewpt": 22.7, "gust_kmh": 9, "gust_kt": 5, "lat": -27.5, "lon": 153.0, "press": 1009.3, "press_msl": 1009.3, "press_qnh": 1009.4, "press_tend": "-", "rain_trace": "0.0", "rel_hum": 83, "sea_state": "-", "swell_dir_worded": "-", "swell_height": null, "swell_period": null, "vis_km": "-", "weather": "-", "wind_dir": "NE", "wind_spd_kmh": 4, "wind_spd_kt": 2Since I don'T have the code for the page download running yet, I just loaded the page on the PC and FTP'd it to a Spinnerets SD card running Tachyon.
So I could experiment with the stream parsing.
If I assume a fixed file structure - which is a good guess ...
it is easy to count the '<' start tags before the data I am interrested in i.e. the temperature values in a good editor like notepad++
So I can easily stream parse for '<' and count the '<'.
when I hit my target number I skip to the closing '>' then skip some fixed text and there I am.
Using Tachyons GETWORD NUMBER the temperature gets transfered to a variable or array ...
then continue for more or close the socket.
This needs only minimal RAM and is quite fast.
Doing it the full FORTH way using the Forth input parser and adapting it would be slower, since it requires dictionary lookups.