SNTP and xml time display on web page - quick question
xanatos
Posts: 1,120
I've got SNTP 2.01 running well and displaying (updating every second) on a few of the web pages on my Spinneret's SD card. What I've noticed are two small but curious items regarding the day of the week, and the seconds display.
The day of the week as shown is consistently two behind (Wed shows as Mon, Sun shows as Fri, etc). I've fixed that by just subtracting for now, but I know that's only going to be a fix for anything that doesn't go less than 0.
The seconds issue is more vexing - what happens is that when a minute is incremented, the seconds, 00 - 09, show up as 50 - 59, then continue as normal as below:
21:58
21:59
22:50
22:51
22:53....
22:59
22:10
22:11...
It looks to me like the buffer isn't clearing out the 5 and is just writing the single character 0 - 9 instead of replacing the 5 with a zero in the byte array... Has anyone seen this before and/or is there a known fix for this? I'm working my way through this, and it's not really affecting anything - but it looks dumb and I'd like to fix it - hopefully by this evening, since I'm installing tomorrow.
Also, it should be noted, that only on the xml page display is this evident - the emails I send out, which are time-stamped, have normal seconds 00 - 09.
Thanks if anyone can assist.
Dave
Update:
Looking at these two functions - which are called from XmlTime, it appears there should be an offset for this issue in HTTPServer.spin - but it's not doing it for the seconds:
The day of the week as shown is consistently two behind (Wed shows as Mon, Sun shows as Fri, etc). I've fixed that by just subtracting for now, but I know that's only going to be a fix for anything that doesn't go less than 0.
The seconds issue is more vexing - what happens is that when a minute is incremented, the seconds, 00 - 09, show up as 50 - 59, then continue as normal as below:
21:58
21:59
22:50
22:51
22:53....
22:59
22:10
22:11...
It looks to me like the buffer isn't clearing out the 5 and is just writing the single character 0 - 9 instead of replacing the 5 with a zero in the byte array... Has anyone seen this before and/or is there a known fix for this? I'm working my way through this, and it's not really affecting anything - but it looks dumb and I'd like to fix it - hopefully by this evening, since I'm installing tomorrow.
Also, it should be noted, that only on the xml page display is this evident - the emails I send out, which are time-stamped, have normal seconds 00 - 09.
Thanks if anyone can assist.
Dave
Update:
Looking at these two functions - which are called from XmlTime, it appears there should be an offset for this issue in HTTPServer.spin - but it's not doing it for the seconds:
PRI FillHttpDate | ptr, num, temp
'ToString(integerToConvert, destinationPointer)
'Wed, 01 Feb 2000 01:00:00 GMT
ptr := @httpDate
rtc.readTime
temp := rtc.getDayString
bytemove(ptr, temp, strsize(temp))
ptr += strsize(temp) + 2
FillTimeHelper(rtc.clockDate, ptr)
ptr += 3
temp := rtc.getMonthString
bytemove(ptr, temp, strsize(temp))
ptr += strsize(temp) + 1
FillTimeHelper(rtc.clockYear, ptr)
ptr += 5
FillTimeHelper(rtc.clockHour, ptr)
ptr += 3
FillTimeHelper(rtc.clockMinute, ptr)
ptr += 3
FillTimeHelper(rtc.clockSecond, ptr)
return @httpDate
PRI FillTimeHelper(number, ptr) | offset
offset := 0
if(number < 10)
offset := 1
str.ToString(@number, @tempNum)
bytemove(ptr+offset, @tempNum, strsize(@tempNum))
return
Comments
I think (not absolutely sure) you might see the same issue when the hour is incremented- the minute value will do the same as the seconds value in your example. I haven't tried it but the rtc.clockMinute/Second/etc methods don't pad single digit values with a 0 (they return decimal values, not ascii characters.) There should be two ways to fix this.
First would be to add a 0 for padding when appropriate. Something like:
PRI FillTimeHelper(number, ptr) | offset offset := 0 if(number < 10) bytefill(ptr, "0" ,1) '<---stuff an ASCII 0 for padding in it here. offset := 1 str.ToString(@number, @tempNum) bytemove(ptr+offset, @tempNum, strsize(@tempNum)) returnSecond solution would be to fill the entire httpDate buffer with ASCII 0's and then overwrite with new data as appropriate:
(This one will probably give an output in your httpdate like Wed,0010Feb02000001:00:000GMT so check closely.)
I don't remember how long the httpdate is but I think it's 29. You might want to double check this.
P.S. I'm spitballing this because I can't test it right now. Nowhere near hardware. I apologize in advance if this doesn't work!
Robert
I'll let you know if that does it, or if I have to tweak it any.
Dave
UPDATE: Option 1 worked like a charm! Thanks!!! Now I just need to figure out how that SNTP byte array gets packed... It's still Sunday :-)
Sometimes I wish we had about 98 hours in a day.
What are you trying to figure out? Something specific? Or just in general how it works?
PUB DisplayHumanTime | Yr, Mo, Dy, Dw, Hr, Mn, Sc Sc := byte[@DW_HH_MM_SS][0] Mn := byte[@DW_HH_MM_SS][1] Hr := byte[@DW_HH_MM_SS][2] Dw := byte[@DW_HH_MM_SS][3] Dy := byte[@MM_DD_YYYY][2] Mo := byte[@MM_DD_YYYY][3] Yr := word[@MM_DD_YYYY][0] Dw := Dw - 5 rtc.writeTime(Sc, Mn, Hr, Dw, Dy, Mo, Yr) 'SET THE RTC (second, minute, hour, day, date, month, year) waitcnt(clkfreq/5+cnt) returnI'm scrambling to finish up the project to install this week (was supposed to be today but I found an issue yesterday...) so unless it's a simple fix, they're just going to have to be satisfied with it being Sunday half the week! :-)
Well, the end result of the SNTP object is to get the NTP value of the transmit timestamp (unless you've decided to use one of the other timestamp methods in the object) and give that NTP value to the HumanTime method to work with. The HumanTime method does *all* of the number manipulation to convert the raw NTP value into bytes that contain decimal values that we can relate to more easily. So- if the DoW is not correct, something in that particular method is not correct in how it's determining the day of the week.
Post the exact HumanTime method you're using (remember, there's about a bazillion of them now) and I'll try to help figure it out if you'd like. I ask so many friggin questions around here the very least I can do is attempt to spit out an answer once in awhile
Here's the HumanTime Method from SNTP 2.01:
PUB HumanTime(Offset,TimeStampAddress)|i,Seconds,Days,Years,LYrs,DW,DD,HH,MM,SS,Month,Date,Year Seconds := long[TimeStampAddress] + Offset * 3600 Days := ((Seconds >>= 7)/675) + 1 '<- Days since Jan 1, 1900 ... divide by 86,400 and add 1 DW := (Days-1) // 7 Years := Days / 365 ' Number of Days THIS year and Days -= (Years * 365) ' number of years since 1900. LYrs := Years / 4 '<- Leap years since 1900 Year := Years + 1900 '<- Current Year Days -= LYrs '<- Leap year Days correction ' for THIS year repeat repeat i from 1 to 12 '<- Calculate number of days Month := 30 ' in each month. Stop if if i&1 <> (i&8)>>3 ' Month has been reached Month += 1 if i == 2 Month := 28 if Days =< Month '<- When done, Days will contain quit ' the number of days so far this if Days > Month ' month. In other words, the Date. Days -= Month { if Days > Month '<- When done, Days will contain Days -= Month ' the number of days so far this if Days =< Month ' month. In other words, the Date. quit } until Days =< Month Month := i '<- Current Month Date := Days '<- Current Date SS := long[TimeStampAddress] + Offset * 3600 SS := SS -(((Years*365)*675)<<7) '<- seconds this year MM := SS / 60 '<- minutes this year SS := SS - (MM * 60) '<- current seconds HH := MM / 60 '<- hours this year MM := MM - (HH * 60) '<- current minutes DD := HH / 24 '<- days this year HH := HH - (DD * 24) '<- current hour DD -= LYrs '<- Leap year Days correction ' for THIS year long[TimeStampAddress][2] := Month<<24+Date<<16+Year long[TimeStampAddress][3] := DW<<24+HH<<16+MM<<8+SS ' DD is redundant but I included it for completion... ' If you subtract the number of days so far this year from ' DD and add one, you should get today's date. This is calculated ' from another angle above from DaysAnd in the post above is the DisplayHumanTime Method that's in HTTPServer.spin that actually takes the data and sets it into the rtc.
I'm still looking for anything, anywhere, in SNTP 2.01 (also posted below) that actually has the Sat, Sun, Mon... etc., abbreviations listed and associated with a number in some sort of lookup table or something. I had figured that would be the place to make the change, just chaanging the day that is associated with the number... but no such table have I found yet.
EDIT: I found the correspondence table - it's in S35390A_RTCEngine.spin
EDIT 2: However, having found that, I figured that the best line to edit was the one in the HumanTime Method:
DW := (Days-1) // 7
I just changed that from a -1 to a +1 and the day is lining up now.... so, I *may* have figured it out. Can you see any way in which that'll mess anything up?
I know how you feel about asking questions and wanting to help. It's a very rewarding feeling to be able to help with something on here, and I feel like it's the least I can do to pay tribute to the people on here who have been so helpful. I honestly could never have gotten as far as I have with microcontrollers if it wasn't for this forum's people.
Thanks again for helping. After two full months now of 10 to 12 hours days in this project (to be fair, though, the last two weeks have mostly been fabricating the housing and doing all the wiring, which I never would have guessed was going to take so long) - I'm really ready to do something that doesn't require soldering irons, magnifying glasses or keyboards. I've never wanted to spend a day doing yard work more in my life! :-)
Dave
SNTP 2.01:
OBJ {{ ****************************************************************** * SNTP Simple Network Time Protocol v2.01 * * Author: Beau Schwabe * * * * Recognition: Benjamin Yaroch, A.G.Schmidt * * * * Copyright (c) 2011 Parallax * * See end of file for terms of use. * ****************************************************************** Revision History: v1 04-07-2011 - File created v1.01 09-08-2011 - Minor code update to correct days in Month rendering - and replace bytefill with bytemove for the 'ref-id' string v2 01-29-2013 - Fixed an illusive bug that caused problems around the first of the year v2.01 02-02-2013 - Logic order error with previous bug fix }} PUB CreateUDPtimeheader(BufferAddress,IPAddr) '--------------------------------------------------------------------- ' UDP IP Address - 4 Bytes '--------------------------------------------------------------------- BYTEMOVE(BufferAddress,IPAddr,4) '--------------------------------------------------------------------- ' UDP Header - 4 Bytes '--------------------------------------------------------------------- byte[BufferAddress][4] := 0 byte[BufferAddress][5] := 123 '<- Port Address byte[BufferAddress][6] := 0 byte[BufferAddress][7] := 48 '<- Header + Packet '--------------------------------------------------------------------- ' UDP Packet - 44 Bytes '--------------------------------------------------------------------- byte[BufferAddress][8] := %11_100_011 'leap,version, and mode byte[BufferAddress][9] := 0 'stratum byte[BufferAddress][10] := 0 'Poll byte[BufferAddress][11] := %10010100 'precision byte[BufferAddress][12] := 0 'rootdelay byte[BufferAddress][13] := 0 'rootdelay byte[BufferAddress][14] := 0 'rootdispersion byte[BufferAddress][15] := 0 'rootdispersion bytemove(BufferAddress+16,string("LOCL"),4) 'ref-id ; four-character ASCII string bytefill(BufferAddress+20,0,32) '(ref, originate, receive, transmit) time { leap = %11 ; alarm condition (clock not synchronized) version = %011 or %100 ; Version 3 or 4 Mode = %011 ; Client stratum = %00000000 ; unspecified Poll = %00000000 ; = 2^n seconds (maximum interval between successive messages) precision = %10010100 ; -20 (8-bit signed integer) rootdelay = 0 ; 32 bit value rootdispersion = 0 ; 32 bit value ref id = "LOCL" ; four-character ASCII string ref time = 0 ; 64 bit value originate time = 0 ; 64 bit value receive time = 0 ; 64 bit value transmit time = 0 ; 64 bit value } PUB GetMode(BufferAddress) result := byte[BufferAddress][8] & %00000111 '0 - reserved '1 - symmetric active '2 - symmetric passive '3 - client '4 - server '5 - broadcast '6 - reserved for NTP control message '7 - reserved for private use PUB GetVersion(BufferAddress) result := (byte[BufferAddress][8] & %00111000)>>3 '3 - Version 3 (IPv4 only) '4 - Version 4 (IPv4, IPv6 and OSI) PUB GetLI(BufferAddress) result := (byte[BufferAddress][8] & %11000000)>>6 '0 - No warning '1 - last minute has 61 seconds '2 - last minute has 59 seconds '3 - alarm condition (clock not synchronized) PUB GetStratum(BufferAddress) result := byte[BufferAddress][9] '0 - unspecified or unavailable '1 - primary reference (e.g., radio clock) '2-15 - secondary reference (via NTP or SNTP) '16-255 - reserved PUB GetPoll(BufferAddress) result := byte[BufferAddress][10] 'This is an eight-bit signed integer indicating the 'maximum interval between successive messages, in seconds 'to the nearest power of two. The values that can appear 'in this field presently range from 4 (16 s) to 14 (16384 s); 'however, most applications use only the sub-range 6 (64 s) 'to 10 (1024 s). PUB GetPrecision(BufferAddress) result := byte[BufferAddress][10] 'This is an eight-bit signed integer indicating the 'precision of the local clock, in seconds to the nearest 'power of two. The values that normally appear in this 'field range from -6 for mains-frequency clocks to -20 for 'microsecond clocks found in some workstations. PUB GetRootDelay(BufferAddress)|Temp1 Temp1 := byte[BufferAddress][12]<<24+byte[BufferAddress][13]<<16 Temp1 += byte[BufferAddress][14]<<8 +byte[BufferAddress][15] result := Temp1 'This is a 32-bit signed fixed-point number indicating the 'total roundtrip delay to the primary reference source, in 'seconds with fraction point between bits 15 and 16. Note 'that this variable can take on both positive and negative 'values, depending on the relative time and frequency offsets. 'The values that normally appear in this field range from 'negative values of a few milliseconds to positive values of 'several hundred milliseconds. PUB GetRootDispersion(BufferAddress)|Temp1 Temp1 := byte[BufferAddress][16]<<24+byte[BufferAddress][17]<<16 Temp1 += byte[BufferAddress][18]<<8 +byte[BufferAddress][19] result := Temp1 'This is a 32-bit unsigned fixed-point number indicating the 'nominal error relative to the primary reference source, in 'seconds with fraction point between bits 15 and 16. The values 'that normally appear in this field range from 0 to several 'hundred milliseconds. PUB{ Calling example: PST.str(GetReferenceIdentifier(@Buffer,string("----")) dashes get replaced with 4-Character Buffer contents } GetReferenceIdentifier(BufferAddress,FillAddress) bytemove(FillAddress,BufferAddress+20,4) result := FillAddress { Reference Identifier return codes Code External Reference Source ----------------------------------------------------------- LOCL uncalibrated local clock used as a primary reference for a subnet without external means of synchronization PPS atomic clock or other pulse-per-second source individually calibrated to national standards ACTS NIST dialup modem service USNO USNO modem service PTB PTB (Germany) modem service TDF Allouis (France) Radio 164 kHz DCF Mainflingen (Germany) Radio 77.5 kHz MSF Rugby (UK) Radio 60 kHz WWV Ft. Collins (US) Radio 2.5, 5, 10, 15, 20 MHz WWVB Boulder (US) Radio 60 kHz WWVH Kaui Hawaii (US) Radio 2.5, 5, 10, 15 MHz CHU Ottawa (Canada) Radio 3330, 7335, 14670 kHz LORC LORAN-C radionavigation system OMEG OMEGA radionavigation system GPS Global Positioning Service GOES Geostationary Orbit Environment Satellite } PUB GetReferenceTimestamp(Offset,BufferAddress,Long1,Long2)|Temp1 Temp1 := byte[BufferAddress][24]<<24+byte[BufferAddress][25]<<16 Temp1 += byte[BufferAddress][26]<<8 +byte[BufferAddress][27] long[Long1]:=Temp1 Temp1 := byte[BufferAddress][28]<<24+byte[BufferAddress][29]<<16 Temp1 += byte[BufferAddress][30]<<8 +byte[BufferAddress][31] long[Long2]:=Temp1 'This is the time at which the local clock was 'last set or corrected, in 64-bit timestamp format. HumanTime(Offset,Long1) PUB GetOriginateTimestamp(Offset,BufferAddress,Long1,Long2)|Temp1 Temp1 := byte[BufferAddress][32]<<24+byte[BufferAddress][33]<<16 Temp1 += byte[BufferAddress][34]<<8 +byte[BufferAddress][35] long[Long1]:=Temp1 Temp1 := byte[BufferAddress][36]<<24+byte[BufferAddress][37]<<16 Temp1 += byte[BufferAddress][38]<<8 +byte[BufferAddress][39] long[Long2]:=Temp1 'This is the time at which the request departed the 'client for the server, in 64-bit timestamp format. HumanTime(Offset,Long1) PUB GetReceiveTimestamp(Offset,BufferAddress,Long1,Long2)|Temp1 Temp1 := byte[BufferAddress][40]<<24+byte[BufferAddress][41]<<16 Temp1 += byte[BufferAddress][42]<<8 +byte[BufferAddress][43] long[Long1]:=Temp1 Temp1 := byte[BufferAddress][44]<<24+byte[BufferAddress][45]<<16 Temp1 += byte[BufferAddress][46]<<8 +byte[BufferAddress][47] long[Long2]:=Temp1 'This is the time at which the request arrived at 'the server, in 64-bit timestamp format. HumanTime(Offset,Long1) PUB GetTransmitTimestamp(Offset,BufferAddress,Long1,Long2)|Temp1 Temp1 := byte[BufferAddress][48]<<24+byte[BufferAddress][49]<<16 Temp1 += byte[BufferAddress][50]<<8 +byte[BufferAddress][51] long[Long1]:=Temp1 Temp1 := byte[BufferAddress][52]<<24+byte[BufferAddress][53]<<16 Temp1 += byte[BufferAddress][54]<<8 +byte[BufferAddress][55] long[Long2]:=Temp1 'This is the time at which the reply departed the 'server for the client, in 64-bit timestamp format. HumanTime(Offset,Long1) PUB HumanTime(Offset,TimeStampAddress)|i,Seconds,Days,Years,LYrs,DW,DD,HH,MM,SS,Month,Date,Year Seconds := long[TimeStampAddress] + Offset * 3600 Days := ((Seconds >>= 7)/675) + 1 '<- Days since Jan 1, 1900 ... divide by 86,400 and add 1 DW := (Days-1) // 7 Years := Days / 365 ' Number of Days THIS year and Days -= (Years * 365) ' number of years since 1900. LYrs := Years / 4 '<- Leap years since 1900 Year := Years + 1900 '<- Current Year Days -= LYrs '<- Leap year Days correction ' for THIS year repeat repeat i from 1 to 12 '<- Calculate number of days Month := 30 ' in each month. Stop if if i&1 <> (i&8)>>3 ' Month has been reached Month += 1 if i == 2 Month := 28 if Days =< Month '<- When done, Days will contain quit ' the number of days so far this if Days > Month ' month. In other words, the Date. Days -= Month { if Days > Month '<- When done, Days will contain Days -= Month ' the number of days so far this if Days =< Month ' month. In other words, the Date. quit } until Days =< Month Month := i '<- Current Month Date := Days '<- Current Date SS := long[TimeStampAddress] + Offset * 3600 SS := SS -(((Years*365)*675)<<7) '<- seconds this year MM := SS / 60 '<- minutes this year SS := SS - (MM * 60) '<- current seconds HH := MM / 60 '<- hours this year MM := MM - (HH * 60) '<- current minutes DD := HH / 24 '<- days this year HH := HH - (DD * 24) '<- current hour DD -= LYrs '<- Leap year Days correction ' for THIS year long[TimeStampAddress][2] := Month<<24+Date<<16+Year long[TimeStampAddress][3] := DW<<24+HH<<16+MM<<8+SS ' DD is redundant but I included it for completion... ' If you subtract the number of days so far this year from ' DD and add one, you should get today's date. This is calculated ' from another angle above from DaysDW := (Days-1) // 7
Dunno how to fix that yet- gonna watch Dexter and come back to it.
Dave
DW := (Days-1) // 7 +1
That'll do it.
Doing DW := (Days+1) // 7 will throw you off at the end of the week instead of the beginning.
However, Jan 01 1900 was a monday.
So, that means Mondays are the 1st day of the week for this line of code.
The math should interpret that as ((Days-1) // 7) +1 if I have my order of operations correct.
But then that means that the resultant 0 through 7 would now become 1 through 8, am I correct in that, or do I fundamentally not get something in the math there?
If I'm correct, then, with the days of the week corresponding to numbers 0 to 7, what does 8 yield?
Let's say Days = 6
Days-1 = 5
5 // 7 = 5
5 // 7 +1 = 6
Let's say Days = 7
Days-1 = 6
6 // 7 =6
6 // 7 +1 = 7
Let's say Days = 8
Days-1 = 7
7 // 7 = 0
7 // 7 +1 = 1 (Remember, day 8 is the first day of the second week... )
Let's say Days = 13
Days-1 = 12
12 // 7 = 5
12 // 7 +1 = 6
Let's say Days = 14
Days-1 =13
13 // 7 =6
13 // 7 +1 = 7
Let's say Days = 15
Days-1 = 14
14 // 7 = 0
14// 7 +1 = 1 (Remember, day 15 is the first day of the third week... )
The +1 is applied to the result of the modulo... modulum... modulai... moduleesis. Yeah that thing. Either way, it's always going to return a value of 0 thru 6, never seven. Since we don't have "day 0" of the week, the +1 makes the result 1 through 7.
I'm following your logic, so far, so good. So then the issue becomes which day should be 0. Here's the DAT block in the RTC Engine:
Does the change in my HumanTime line for DW mean I should change the day lineup in my DAT block so that it starts with Monday instead of Sunday? because if I use
((Days-1) // 7) +1
Then it says that today is Monday, when it's actually Tuesday... changing the DAT line to:
Does make it display the correct day of the week - but will it screw anything else up?
I was just sitting here giving myself a headache thinking through that exact change in the DAT section. I think that's the correct fix. It shouldn't mess with anything else because the only thing that uses that line of the DAT block is the getDayString method.
Here is a sample stand-alone program for playing around with this "issue" a bit. It might help if you come into more trouble with the day of week, but I think you've got it licked now.
CON _xinfreq = 5_000_000 _clkmode = xtal1 + pll16x OBJ pst : "Parallax Serial Terminal" DAT DoW byte "Mon", 0, "Tue", 0, "Wed", 0, "Thu", 0, "Fri", 0, "Sat", 0, "Sun", 0 'Each day takes up 4 bytes with the 0 at the end. { DoW byte "Sun", 0, "Mon", 0, "Tue", 0, "Wed", 0, "Thu", 0, "Fri", 0, "Sat", 0 } pub Start pst.Start(115_200) delay(3000) Main pub Main|TotalDays,dayofweek repeat TotalDays:=1 '<------- enter any number of days since Jan 01, 1900 here. dayofweek:= (TotalDays-1) // 7 +1 pst.str(String("Modulo thingy: ")) pst.Dec(dayofweek) pst.newline pst.str(String("Day of week: ")) pst.Str(@Dow+(dayofweek-1)*4) pst.newline pst.newline delay(1000) pub Delay(Time) | AutoTime, Cycle 'Wait amount of time before continuing Cycle:=cnt AutoTime:=clkfreq/1000 * Time waitcnt(Cycle+=AutoTime)It's nice to be down to little tweaks - especially since the (new) install date in Thursday...
Dave