Problem: The Propeller Project Board USB (#32810) and the WX ESP8266 WiFi Module - SIP (#32420S)

idbruce

What I assumed was going to be a very simple task, has indeed turned out to be very frustrating. With one intention in mind, I recently purchased the Propeller Project Board USB (#32810) and the WX ESP8266 WiFi Module - SIP (#32420S). The goal is to obtain the text from a *.txt file on the internet.

I have the WX ESP8266 WiFi Module connected to the Propeller Project Board USB and it appears to be able to communicate with servers, but it is unable to display the correct value. I am currently using an altered example that was provided in a downloaded example from the WX ESP8266 WiFi Module product page. More specifically, I am using the "Text from www page with TCP.side".

The original example code, with "wifi_start" modified, is as follows:

/*
  Application circuit:
  None.
    
  Important: Your Wi-Fi module has to be connected to a
  Wi-Fi network that allows it Internet access for this 
  to work.  
    
  Programming and Wi-Fi selection circuits at:
  http://learn.parallax.com/propeller-c-wx-wi-fi 
  Make sure to change the wifi_start call in the code to 
  match the COM control circuit you choose.
  
  This application does not make the Wi-Fi module serve
  and monitor a page.  Instead, it grabs text from this
  page on the Internet: 
    www-eng-x.llnl.gov//documents/a_document.txt
  
  Note: This example relies on the 0.8 version of the wifi library.  
  Updates may change some function behaviors in later releases.
*/

#include "simpletools.h"
#include "wifi.h"

int event, id, handle;
int getFromPageId;
int val;
char str[512];
char wifi_event;

int main()
{
  wifi_start(9, 8, 115200, USB_PGM_TERM);
  wifi_setBuffer(str, sizeof(str));

  int tcpHandle = wifi_connect("www-eng-x.llnl.gov", 80);

  char request[] = "GET /documents/a_document.txt "\
                   "HTTP/1.1\r\n"\
                   "Host: www-eng-x.llnl.gov\r\n\r\n\0";

  int size = strlen(request);

  wifi_print(TCP, tcpHandle, "%s", request);
  event = wifi_event;
  
  pause(1000);

  wifi_scan(TCP, tcpHandle, "%s", str); 
  
  print("str = %s\r", str); 
}

**PLEASE NOTE: This example should be modified, because the example provides a redirect for the HTTP request.

And here is the modified code that I am attempting to use:

/*
  Application circuit:
  None.
    
  Important: Your Wi-Fi module has to be connected to a
  Wi-Fi network that allows it Internet access for this 
  to work.  
    
  Programming and Wi-Fi selection circuits at:
  http://learn.parallax.com/propeller-c-wx-wi-fi 
  Make sure to change the wifi_start call in the code to 
  match the COM control circuit you choose.
  
  This application does not make the Wi-Fi module serve
  and monitor a page.  Instead, it grabs text from this
  page on the Internet: 
    www-eng-x.llnl.gov//documents/a_document.txt
  
  Note: This example relies on the 0.8 version of the wifi library.  
  Updates may change some function behaviors in later releases.
*/

#include "simpletools.h"
#include "wifi.h"

int event, id, handle;
int getFromPageId;
int val;
char str[512];
char wifi_event;

int main()
{
  wifi_start(9, 8, 115200, USB_PGM_TERM);
  wifi_setBuffer(str, sizeof(str));

  int tcpHandle = wifi_connect("novelsolutionsonline.com", 80);

  char request[] = "GET /success.txt"\
                   "HTTP/1.1\r\n"
                   "Host: novelsolutionsonline.com\r\n\r\n\0";
                   
  int size = strlen(request);

  wifi_print(TCP, tcpHandle, "%s", request);
  event = wifi_event;
  
  pause(1000);

  wifi_scan(TCP, tcpHandle, "%s", str); 
  
  print("str = %s\r", str); 
}

The GET request is trying to obtain the contents of the file named "success.txt", located at novelsolutionsonline.com/success.txt, which simply contains the word "SUCCESS".

After many hours of frustration, the best that I can do is represented by the image below.



Even if I provide a fictious file name as a parameter for GET, I still get the same nonsense. What am I doing wrong?

EDIT: Correction.... The WiFi Module is temporarily connected to a Prop BOE USB board for testing, but I don't imagine that makes a difference, however I could be wrong

iseries

Apparently this site is picky about what it gets for a request. It requires the browser type to process the request.

This request seems to work:

char rqs[] = "GET /success.txt HTTP/1.1\r\nAccept: text/html\r\nHost: novelsolutionsonline.com\r\n"
             "Connection: Keep-Alive\r\nCache-Control: no-cache\r\n"
             "User-Agent: Mozilla/5.0\r\n"
             "\r\n";

The import piece is the User-Agent is required.

Mike

idbruce

Well thank you very much iseries. I would have never figured that all out. Okay, maybe after many more hours of frustration and research.

Thank you, thank you. thank you.

iseries

I cheated and used Wireshark to see what pieces were being sent out with a real browser.

Mike

idbruce

Well, thank you for the tip also. I am sure I will need that knowledge in the future

idbruce

@iseries - I don't suppose you have a quick method to just obtain the text contained in the file and weed out the rest of the response?

iseries

Yes, the header data which is the junk that comes back first is separated from the body with two consecutive carriage return and line feed characters. Find that pattern and you have the start of the body which should contain SUCCESS.

Mike

idbruce

To those that may be interested.....

iseries provided the information necessary to gain access to a file on my website, which contained a simple text value. However the response came back with a bunch of added mumbo jumbo, which I had to eliminate.

Depending upon the content of a file you are accessing, you can do as iseries suggested in his last reply, but if you are simply retrieving values like I am, you can use the code below to remove the added stuff. I am certain that there is a more elegant way to code this, but this works for me.

#include "simpletools.h"
#include "wifi.h"
#include <string.h>

int main()
{
  char chResponse[512];
  char chContent[512];
  
  char * pCharacterPointer;
  
  int nHandle;
  int nSizeOfResponse;
  int nBeginningOfContent;
  int nContentLength;  
  int nContentIndex;
  
  wifi_start(9, 8, 115200, USB_PGM_TERM);
  wifi_setBuffer(chResponse, sizeof(chResponse));

  nHandle = wifi_connect("novelsolutionsonline.com", 80);
             
  char chRequest[] = "GET /success.txt HTTP/1.1\r\nAccept: text/html\r\nHost: novelsolutionsonline.com\r\n"
             "Connection: Keep-Alive\r\nCache-Control: no-cache\r\n"
             "User-Agent: Mozilla/5.0\r\n"
             "\r\n";
                   
  wifi_print(TCP, nHandle, "%s", chRequest);
  
  pause(1000);

  wifi_scan(TCP, nHandle, "%s", chResponse);   
  
  pCharacterPointer = strrchr(chResponse,'\n');
  
  nSizeOfResponse = strlen(chResponse);
  nBeginningOfContent = pCharacterPointer - chResponse + 1;
  nContentLength = nSizeOfResponse - nBeginningOfContent;  
  nContentIndex = 0;
  
  while(nContentLength > 0)
  {
    chContent[nContentIndex] = chResponse[nBeginningOfContent];
    ++nContentIndex;
    ++nBeginningOfContent;
    --nContentLength;
  }
  
  print("%s", chContent);
}