DEMO: Using SNTP to sync the Spinneret's on-board RTC

Beau Schwabe

Here is a Demo that uses SNTP (Simple Network Time Protocol) to Sync and Set the time of the on-board Spinneret RTC (Real Time Clock). The W5100 is accessed in Indirect mode to open a socket in UDP mode to retrieve the SNTP data. The RTC is then set to the SNTP time and displayed using the PST (Parallax Serial Terminal)

Note1: Other SNTP time servers here to choose from: http://tf.nist.gov/tf-cgi/servers.cgi

Note2: A previous version used the W5100 in SPI mode, this version uses Indirect/Parallel mode. Not that it matters much for setting the RTC, I just thought it would be easier to transition into an HTML Web application already using the Indirect driver without loading another object.

Enjoy!

EDIT: Please see the latest BUG fix ... http://forums.parallax.com/showthread.php/145697-SNTP-Simple-Network-Time-Protocol-BUG-fix-Update


LATEST program UPDATE.... v1.0.1

Ltech

Fault at : 50 and 53

'USA Standard Time Zone Abbreviations
#-10, HST,AtST,PST,MST,CST,EST,AlST

'USA Daylight Time Zone Abbreviations
#-9, HDT,AtDT,PDT,MDT,CDT,EDT,AlDT

Is the "-" a bug?

How can I get GMT +1 ?

Thanks

Beau Schwabe

Ltech,

No, the "-" is not a bug. I'm not sure I understand your 'Fault at : 50 and 53'

The line... '#-10, HST,AtST,PST,MST,CST,EST,AlST' ... In the Propeller IDE evaluates to ...

HST = -10
AtST = -9
PST = -8
MST = -7
CST = -6 <-- I am in Central USA time zone which is GMT - 6
EST = -5
AlST = -4

...likewise for the Daylight time zones

If you want GMT + 1, then in the CON section, just make...

BST = 1 or CET = 1 or wherever you may be.


The line of code that this constant gets applied to reads ...

SNTP.GetTransmitTimestamp(CST,@Buffer,@LongHIGH,@LongLOW)

... Change 'CST' to your abbreviation or just hard code the number in its place like this...

SNTP.GetTransmitTimestamp(+1,@Buffer,@LongHIGH,@LongLOW)

LongHIGH contains the upper 32 Bits of the 64 Bit time stamp <--- Seconds
LongLOW contains the lower 32 Bits of the 64 Bit time stamp <--- Fractional seconds

Ltech

I use bst 19,3 on a ppc mac osx 10.5 .....

At line Nr 50 and 53 i get "Error at 50,7 Expected Unique Name, Constant or "(" "

If I remove the minus the program compiled

I get to find the use of " #-10, HST,AtST,PST,MST,CST,EST,AlST "

The CET part work when I use " SNTP.GetTransmitTimestamp(1,@Buffer,@LongHIGH,@LongLOW) "
Not working when I use " SNTP.GetTransmitTimestamp(CET,@Buffer,@LongHIGH,@LongLOW) "
and before riped the minus or change in line 50 & 53 " #(-10), HST,AtST,PST,MST,CST,EST,AlST "

I hope to have more time to play with the Supper Spinneret

Thanks Beau !

Ltech

Ok at work with pc win7 the file compile fine !
So it is a bst bug on osx, for me the first one.

Chip Cox

I need some debugging help. I have tried the sample code in this thread and it works wonderfully. I integrated it into my larger code body and it worked fine. Then I did something. Unfortunately I wasn't paying attention to the time while I was coding another area and I screwed up the SNTP lookup. I have tried commenting out the code I changed, I have debug messages after almost every line it seems. I am getting to the txUDP line in gettime. The data in the buffer looks ok at least based on where the module put it, it's there. I have tried every combination of byte[], @, @byte[], @byte[][0] that I could think of in the off chance that it was something about the buffer address being passed to txudp. Unfortunately with txudp being run in a separate cog, my debugging knowledge hits a brick wall. Any pointers for how to move forward debugging this. Or are there common mistakes calling txudp that you can think of that might help without posting all of my code. ( it's kind of messy right now with debug statements everywhere, it would be kind of embarrassing to post it <grin>) So in short, I'm not looking for someone to tell me what I did wrong, but for suggestions on how to better troubleshoot it so I can find and fix my own problems in the future. Thanks

Beau Schwabe

Chip Cox,

It's really hard to say without looking at your code, and even then... I have trouble sometimes looking at my own code :-)

If you want you can send it to me via E-mail, and I'll see what I can determine.

Brandon_L

FYI- The demo code as downloaded complies with Errors on Line 50 & 53. It is not taking -10. This is on a mac. I added "(" ")" and it works fine. So for Example:" #(-10), HST,AtST,PST,MST,CST,EST,AlST "

Hope this helps

Beau Schwabe

Brandon_L,

This is a BST compile error in how it handles the viewed data on the screen.

In the Propeller IDE it compiles correctly.

Benj

Thank you for posting the Beau. Yesterday I ordered a Spinneret to do exactly what your demo does. I am going to build a "time base" that will received internet time and then transmit it out via XBee to several other Prop based projects in the vicinity that need to keep time fairly accurately and handle time changes. After ordering I looked at the docs for the Spinneret and was discouraged because it all is written for people with a much higher level of understanding than I have, and they want to do much more complex things than I want. I was very relieved to find your code as it seems like exactly what I need.

I do have a question though about how the time value returned from the SNTP souce is formatted. In the CON section, you have "Zone = CST", however, in the notes of the DAT section, it says "Typically, the TIME server will report time in local time zone". If this is true, why do you need to specify the time zone you are in? Is it used to convert local time to GMT? If so, do most people set their RTC to GMT? Thank you for any help you might be able to provide.

Benj

Beau Schwabe

Benj,

"Typically, the TIME server will report time in local time zone" - It really depends on the time server you are calling. I'm sure during my research into SNTP I found that statement, but I can't find it at the moment.

Here is a list of time servers...
http://tf.nist.gov/tf-cgi/servers.cgi

Don't abuse the privledge ... "All users should ensure that their software NEVER queries a server more frequently than once every 4 seconds. Systems that exceed this rate will be refused service. In extreme cases, systems that exceed this limit may be considered as attempting a denial-of-service attack."


Here is also a Perl Script version that I wrote for testing before implementing it into the Spinneret

#!/usr/bin/perl

 use IO::Socket::INET;
$MySocket=new IO::Socket::INET->new(PeerPort=>123,  #open socket
        Proto=>'udp',
        PeerAddr=>'132.163.4.101');

$data =     chr(227) . chr(0) . chr(0) . chr(148);  #LI/VN/Mode, Stratum, Poll, Precision
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Root Delay
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Root Dispersion
$data = $data . "LOCL";                             #Reference Identifier
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Reference Timestamp
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Originate Timestamp
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Receive Timestamp
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);  #Transmit Timestamp
$data = $data . chr(0) . chr(0) . chr(0) . chr(0);

$MySocket->send($data);			            #Waiting to receive data		
$MySocket->recv($data,56);                          
$MySocket->close();                                 #received data close socket

@Chars = split("", $data);

$n1 = ord($Chars[40]);				    #Upper 32 bits of Transmit Timestamp
$n2 = ord($Chars[41]);
$n3 = ord($Chars[42]);
$n4 = ord($Chars[43]);

$Seconds = $n4 + $n3*256 + $n2*65536 + $n1*16777216; #Seconds since Jan1, 1900

$Zone =  -6;                                         #GMT - 6 time zone adjust
$Zone =  -5;                                         #use GMT - 5 during daylight savings

$Seconds = $Seconds  + 3600*$Zone;

print "Seconds since Jan1, 1900 : $Seconds \n\r";






$Days = int((($Seconds >> 7)/675)+1);                #Days since Jan1, 1900     {divide seconds by 86,400 and add 1}
print "   Days since Jan1, 1900 : $Days \n\r";
          
$DW = ($Days-1) % 7; 			             #Day of Week               

$Years = int($Days / 365);                           #Years since 1900
print "  Years since Jan1, 1900 : $Years \n\r";

$Days = $Days % 365;                                 #Days this year

$LeapYears = int($Years / 4);			     #Leap Years since 1900
$CurrentYear = $Years + 1900 ;                       #Current Year

$Days = $Days - $LeapYears;                          #Leap Year Days Correction

print "          Days this year : $Days \n\r";

#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~


$count = 1;					     #Routine to determine
 do {                                                #  - The Current Month 
$Month = 30;                                         #  - The Current Date       
$c1 = ($count & 1);                                  #
$c2 = (($count & 8) >>3);                            # from the number of Days
if ($c1 != $c2) {                                    # passed in the current year 
    $Month = $Month +1;                              
   }
if ($count == 2) {
    $Month = 28; 
   }

print "Month : $count Days in Month : $Month \n\r";

if ($Days >= $Month) {
   $Days = $Days - $Month;
   if ($Days <= $Month) {
      $CurrentMonth = $count + 1;
      $CurrentDate  = $Days;  
      $count = 12;
      }     
   }
$count++;
 } until ($count == 13);
#~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$Seconds = $Seconds - ((($Years * 365)*675) *128);   #Seconds this year
$SS = $Seconds;
$MM = int($SS / 60);                                 
$SS = $SS - ($MM*60);                                #Current Seconds
$HH = int($MM / 60);                                 
$MM = $MM - ($HH*60);                                #Current Minutes
$DD = int($HH / 24);                                 
$HH = $HH - ($DD*24);                                #Current Hour

print "\n\r";
print " Date: $CurrentMonth / $CurrentDate / $CurrentYear \n\r";
print " Time: $HH : $MM : $SS \n\r";

Note:
There is one small error in the SNTP Simple Network Time Protocol.spin file. The line that reads bytefill(BufferAddress+16,string("LOCL"),4) 'ref-id ; four-character ASCII string should read bytemove(BufferAddress+16,string("LOCL"),4) 'ref-id ; four-character ASCII string instead. This original still works for now, but I will make the correction and update the file this evening. - Error fixed

LATEST program UPDATE.... v1.0.1

Benj

In all likelihood I would only be syncing once per day, or maybe once per hour, so I wouldn't come close to the 4-second rule. If I am understanding your code and the SNTP Simple Network Time Protocol.spin correctly, it would make sense that the data actually is returned as GMT and SNTP's HumanTime method converts it to local time, based upon "Offset", which in this case would be the "Zone" constant. The only reason I ask is because, as it stands, there don't appear to be any provisions for automatically switching between Standard and DST on the proper dates. No complaints from me if this is true as I'm sure I can handle that last little 01%, I'm just seeing if I understand the code correctly.

Benj

Benj

I've had some time to poke around with this and get it working on my Spinneret, but I noticed two issues.

First, while there are provisions for switching between "Standard" and "Daylight Saving" time, it is a manual process, which requires re-upping code to your Spinneret twice a year.

Second, it appears that the GetTime method doesn't function as I had anticipated when the SNTP host can't be reached. If you use Beau's stock code and unplug the ethernet cable, you will find that after the timeout, it still sets the RTC and displays a time in the year 2036. I'm not sure why a timeout returns a -1 instead of a 0. Switching it to 0 fixes this problem.

So, instead of just listing problems, I put together some fixes. They might be a little klugey, but they do work. I wrote a method IsDST that determines whether or not the time returned should be DST or not. If it is, the offset is adjusted and then the RTC is programmed. Since I have friends in the European Union, and their rules are different, I wrote IsDSTEU for them. I also "fixed" GetTime so that a timeout returns a 0 and added some serial terminal text to indicate that SNTP had timed out.

I also added Beau's above mentioned fix of changing bytefill to bytemove in SNTP Simple Network Time Protocol.

I was not really sure where the best place was to put my new methods, but it was easiest to integrate them into SNTP Simple Network Time Protocol, so that's where they went.

This is the first code I have ever posted, so I'm excited to have some people take a look, try it out, and give me some feedback. I also am not sure I noted the additional author (me) and revision notes properly, but I'm learning.

Benj

I don't know if this is the proper place to post this, but since I have only made minor enhancements/revisions, I don't know if it warrants a new thread.

Benj

I just happened to notice today that the time is 12 hours slow. I re-ran the original code and still get the same problem. It may have always been that way and I just never noticed.

My PST output:

Mon Jun 20, 2011 05:27:43 AM


SNTP Server Sync time:
(GMT - 4:00) 06/20/2011 17:27:43

Can anyone else confirm this? I want to make sure that I didn't mess up one of the settings of my RTC or something.

Beau Schwabe

Benj,

I'll have to set this up again and double check (might be next week), I'm pretty sure I worked that out though.

When I was testing I wasn't using a Spinneret at all, I was using Perl on a Linux computer. I might have missed something in the translation into "Human readable Numbers".

Benj

I'm not sure what the problem is at this point and it doesn't happen every time. I first noticed it on Monday afternoon and made this thread. Tuesday morning it was working correctly again on 2 different spinnerets, so I thought it might be something that only happens in PM and the clock is mistakenly set for AM. At around 1:30PM I re-ran the code and the clock was set to 1:30AM (I tried on both, same result). I began to wonder if it was a 12/24 hour mode setting for the RTC or something. I set it for 24 hour mode and everything seemed to work fine (in 24 hour mode), it was setting the clock to 13 hours. I switched back to 12 hour mode and tried again, and it worked correctly, the time was set to PM (I believe some time after 2PM). I thought that perhaps the changing of the 24hourmode register on the RTC had played a role somehow, so to confirm this I connected my second spinneret (that I hadn't done anything with the 24hourmode) but it also set properly to PM. So, the problem seemed to be happening during the 1PM hour..? I am going to try again in the afternoon today and see what happens.

Benj

I think I have narrowed it down a little further. If you select a time zone where it is currently 1PM and run the code:

RTC.SetDateTime gets sent a 13 for hours
RTC.GetHour returns a 1 for hours
RTC.IsPM returns false (although it should be true)

I next selected a time zone that would set the clock to 11:xxAM and just let it run.

RTC.SetDateTime gets sent an 11 for hours
RTC.GetHour returns an 11 for hours
RTC.IsPM returns false (as it should)
at 12PM RTC.IsPM starts to return true (as it should)

So, it appears that the clock can rollover to PM, but occasionally fails when setting. This doesn't always happen, just for stints of time.

Beau Schwabe

Hmm, sounds like a logic problem in the code. I'll take a look when I get a chance.

Jeffa

Curiosity and learning - Why is _clkmode and _clkfreq set in the "s-35390A_GBSbuild_02_09_2011.spin" driver?
Is it just for testing when viewport is enabled?

CON
    _clkmode = xtal1 + pll16x
    _clkfreq = 80_000_000

skynugget

i tried the v1 and v1.1 demos, and with both packets my time sync is 1 day off
its driving me nuts!, any ideas?


Thu Sep 09, 2011 11:42:08 AM

SNTP Server Sync time:
(GMT - 6:00) 09/09/2011 11:39:51

it should be:
Thu Sep 08, 2011

Beau Schwabe

skynugget,

Hmm, that's a problem. I just tested the Perl script and it shows the same result...

[bschwabe@IC-Layout ~]$ timetest.pl
 Date: 9 / 9 / 2011 
 Time: 10 : 4 : 38 

...This means that there is an error in the way the Date is derived from the elapsed seconds since January 1st 1900.

I'll look into that and fix this

Beau Schwabe

skynugget,

What Time Zone are you in?... that could make a difference ... I think that's where the problem is in the code.

skynugget

im in the est, but i had the same problem with GMT as my zone, if it helps.

Beau Schwabe

skynugget,

Keep in mind that GMT does not observe daylight savings so, while I am in CST (GMT-6) right now to see the proper time the GMT value would be GMT-5.

That's not the problem with the wrong day, although there is some overlap around midnight with the current calculations. I'll see about addressing that.

Beau Schwabe

Found it!

Inside of 'SNTP Simple Network Time Protocol.spin' change ....

if i&1 <> (i&8)>>4      '   Month has been reached

... so that it reads...

if i&1 <> (i&8)>>[COLOR="red"]3[/COLOR]      '   Month has been reached

skynugget

*highfive! thank you sir that worked.

Beau Schwabe

Just in case anyone is curious... The logic behind ... if i&1 <> (i&8)>>3

If you Start with January as 1 and December as 12 and look at the binary nibbles .. bits A and D
With the exception of February if A and D are different then there are 31 days that month, if they are the same, then there are 30 days. ... With July and Aug being where the 'Flip' is. ...and Yes, I could have just had a lookup table, but I wanted to be different.

0001 - Jan
0010 - Feb
0011 - Mar
0100 -Apr
0101 -May
0110 -Jun
0111 - Jul
1000 -Aug
1001 -Sept
1010 -Oct
1011 -Nov
1100 -Dec

Beau Schwabe

Updated Code:

[B]'#### SNTP Spinneret DEMO v1.0.1[/B]
Revision History:
02-09-2011        - 'SNTP Spinneret DEMO v1.spin' release
09-08-2011        - optimized SNTP timeout for a more reliable server hit
                  - Added MAC ID validation (please play nice)


[B]'SNTP Simple Network Time Protocol v1.0.1[/B]
Revision History:
04-07-2011        - File created

09-08-2011        - Minor code update to correct days in Month rendering
                  - and replace bytefill with bytemove for the 'ref-id' string

Beau Schwabe

Jeffa,

Sorry I did not reply when you asked the question, I didn't see it and somehow missed it until now...

"Curiosity and learning - Why is _clkmode and _clkfreq set in the "s-35390A_GBSbuild_02_09_2011.spin" driver?
Is it just for testing when viewport is enabled?" ... I believe you are correct, and it is an artifact associated with ViewPort. This section of code is Roy's original code, all that I did was to add a routine FmtDateTime which formats the Date and Time into a human readable string

Ltech

And now the other way round ?

Give some time to RS232, and make a SNTP server ?
I know it wil not be realy sync, but ..... 1 second acuracy ?

So I have a ref time on RS422, and want to distribute it with a SNTP spinneret server on local network ?

JohnR2010

Beau, just wanted to take a sec and say thanks for the code!! It does the job!! Now I have auto time sync over SNTP every time you reboot my spinner. Love it! Just wish there was an easy way to know with daylight saving time starts and stops.