Algorithm Request: Create YYYY/MM/DD from seconds since Jan 1, 1900 and vice-v

Timothy D. Swieter

To test the Brilldea UDP W5100 driver I thought I would create a demo program that implements the Network Time Protocol (NTP). Attached is a very, very rough demo code of sending a single packet to an NTP server and getting a packet response. Note this code violates all recommended practices for the NTP specification so please use it cautiously. For instance the time sever address is hard coded (based on a query from pool.ntp.org). I will provide a more complete demo but first I have a challenge.....

The NTP protocol has timestamps. The timestamp is two 32-bit numbers. The first 32-bit number is the number of seconds since January 1, 1900. The second 32-bit number is the fractions of a second. For the moment lets only focus on the first 32-bit number. I started researching and trying to create an algorithm that goes from YYYY/MM/DD format to number of seconds and from number of seconds back to a YYYY/MM/DD format. However my head started spinning while working on it and so I need to go do something else for a while. I thought this would be an interesting code challenge for the forum to try out while I am focused on other tasks.

Anyone want to try and create the two algorithms?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com

pullmoll

Timothy D. Swieter said...
To test the Brilldea UDP W5100 driver I thought I would create a demo program that implements the Network Time Protocol (NTP). Attached is a very, very rough demo code of sending a single packet to an NTP server and getting a packet response. Note this code violates all recommended practices for the NTP specification so please use it cautiously. For instance the time sever address is hard coded (based on a query from pool.ntp.org). I will provide a more complete demo but first I have a challenge.....

The NTP protocol has timestamps. The timestamp is two 32-bit numbers. The first 32-bit number is the number of seconds since January 1, 1900. The second 32-bit number is the fractions of a second. For the moment lets only focus on the first 32-bit number. I started researching and trying to create an algorithm that goes from YYYY/MM/DD format to number of seconds and from number of seconds back to a YYYY/MM/DD format. However my head started spinning while working on it and so I need to go do something else for a while. I thought this would be an interesting code challenge for the forum to try out while I am focused on other tasks.

Anyone want to try and create the two algorithms?

I have done this in x86 asm a while ago and believe me: it's difficult. You have to take care of an awful lot of special cases. I attached the asm code for fun and inspiration, but I wouldn't want to try to write this in PASM

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
He died at the console of hunger and thirst.
Next day he was buried. Face down, nine edge first.

Timothy D. Swieter

Hmm...is there a creative, yet technically sounds way to perhaps reduce or eliminate the oddity with past year conditions? For instance, perhaps the algorithm could start from something like year 2005 and a lookup table, but disregard any values before that since obviously this code will only be used going forward in time. Of course it may not be a strict implementation of NTP, but something that works for our projects and products going forward.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" LCD Composite video display, eProto for SunSPOT
www.tdswieter.com

Leon

I think the C ctime function does seconds to date conversion, you should be able to find the code in the appropriate gcc compiler library source files. The reverse function should be available.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Leon Heller
Amateur radio callsign: G1HSM

Post Edited (Leon) : 3/21/2010 11:20:34 AM GMT

Cluso99

The only issue is that 1900 is not a leap year. Otherwise, all other /4 years are leap years. I do not expect your code will run in 2100 which is not a leap year. You see years divisible by 4 are leap years, yet those not are not leap years.
I would suggest you calculate the seconds for a leap year, and those for a non leap year, and the number up to 2000 since we are there and you will not be using less than that in your app.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz

pullmoll

Cluso99 said...
The only issue is that 1900 is not a leap year. Otherwise, all other /4 years are leap years. I do not expect your code will run in 2100 which is not a leap year. You see years divisible by 4 are leap years, yet those not are not leap years.
I would suggest you calculate the seconds for a leap year, and those for a non leap year, and the number up to 2000 since we are there and you will not be using less than that in your app.

Yup, that should work. I don't know for how long an unsigned 32 bit will suffice to count the seconds since 1900. The signed 32 bit overflow on the Unix epoch is sometime 2038 IIRC, but by then we will all be doing 64 bit or more math.
FWIW there are 17 leap years and 53 non leap years from 1900 to 1969 inclusive. From 1970 to 1999 I haven't the numbers handy, sorry.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
He died at the console of hunger and thirst.
Next day he was buried. Face down, nine edge first.

Cluso99

It's easy to calculate the leap years..
1900-1999 = 100/4 -1 =24 leap and 76 non-leap because 1900 was NOT a leap year.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Links to other interesting threads:

· Home of the MultiBladeProps: TriBlade,·RamBlade,·SixBlade, website
· Single Board Computer:·3 Propeller ICs·and a·TriBladeProp board (ZiCog Z80 Emulator)
· Prop Tools under Development or Completed (Index)
· Emulators: CPUs Z80 etc; Micros Altair etc;· Terminals·VT100 etc; (Index) ZiCog (Z80) , MoCog (6809)·
· Prop OS: SphinxOS·, PropDos , PropCmd··· Search the Propeller forums·(uses advanced Google search)
My cruising website is: ·www.bluemagic.biz·· MultiBlade Props: www.cluso.bluemagic.biz

Harrison.

There is an excellent object on the OBEX that converts between seconds and standard human readable date/time. I use it for time sync using NIST's daytime service (which uses TCP instead of UDP, since PropTCP doesn't support UDP yet).

Object: http://obex.parallax.com/objects/36/

PJAllen

[noparse][[/noparse]If the year ends with a "0"·then it's not a Leap Year.]

Propability

pj allen said...
[noparse][[/noparse]if the year ends with a "0"·then it's not a leap year.]

··· 2000 ?

Beau Schwabe

There is a catch to the ending with "0"

If the year is divisible by 4 then it is a leap year, UNLESS the year is divisible by 100 then it is not a leap year. The catch or exception is if the year is also divisible by 400 in that it takes precedence over the year being divisible by 4 or 100. If the year is divisible by 400 it is a leap year.


en.wikipedia.org/wiki/Leap_year

if (year modulo 4 is 0) and (year modulo 100 is not 0) or (year modulo 400 is 0)
       then is_leap_year
else
       not_leap_year

What's interesting is that if you ask someone what the shortest Month in recorded history is due to the introduction of the Gregorian Calendar ... it is NOT February.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Beau Schwabe

IC Layout Engineer
Parallax, Inc.

Tracy Allen

I had a routine for the Stamp, converted to Spin:

PUB julianMinute(year, month, day, hour, minute)   ' date and time as flat binary (not BCD)
    result := (month-1)*30+((month/9+month)/2)-(((month <# 3)/3)*(((year//4) <# 1) +1))+ day  ' day of current year, 1 to 365 (366 in ly)
    result := result + (((year // 100) - 1) * 365) + ((year // 100 - 1)/4) - 1    ' days since 1/1/2001, minus 1 (through "yesterday")
    result := result * 1440 + hour * 60 + minute    ' days thru yesterday to minutes, plus minutes from "today"
    return    ' result contains minutes elapsed since midnight, 1/1/2001, covers thru 12/31/2099

This is in minutes since midnight on 1/1/2001. There are 36890 days from Jan 1, 1900 to Jan 1, 2001, so the offset minutes to add in order to reference that year would be 53121600. I'm not sure how to do the seconds. Of course, it is 60 seconds per minute, but occasionally the wizards in the tower add a leap second to account to keep in sync with the stars and wow and wobble of mother earth. Are leap seconds are documented somewhere? (caveat-I typed the above off a hard copy)

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

Brian Carpenter

Timothy,

Did tracy's routine work for you?

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔


It's Only A Stupid Question If You Have Not Googled It First!!

Timothy D. Swieter

I haven't gotten time yet this week to work on software. There are a couple great suggestions and examples here to review. I hope maybe Thursday or Friday to find time.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" 16:9 LCD Composite video display, eProto for SunSPOT, PropNET, PolkaDOT-51
www.tdswieter.com

Rayman

You might want to take a look at this function:

PRI GmTime(secs,pTime)|year,i,yday, hour, minute,p, month,mday
  'convert unix time (#seconds since epoch (jan 1, 1970)) to GMT time structre @pTime

In my PropTime code posted here:
http://www.rayslogic.com/propeller/Programming/Programming.htm



▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

My Prop Products:· http://www.rayslogic.com/Propeller/Products/Products.htm

Beau Schwabe

We do this because a year is not exactly 365 days, it's actually 365.2425 days ... Since we can pretty much nail 365 days we are left with a fraction of a day at the end of each year of 0.2425 days, so we add a 'leap year day' to the calendar every 4 years to compensate. Since this adjustment of 0.25 every 4 years overshoots a day we are left with an error of ... 0.0075 ... (.25 - .2425 = 0.0075) causing us to modify the blanket equation that a leap year happens every 4 years.

... so now to correct this error, we subtract a 'leap year day' every 100 years. This creates an error that now undershoots a a day by 0.0025 which now can easily be 'fixed' every 400 years by adding a 'leap year day' to the calendar.

The resulting equation is:

365 + 1/4 − 1/100 + 1/400

...Which means:

Every 4 years add a day
Every 100 years subtract a day
Every 400 years add a day


Keep in mind though with the passing of time the value of a year becomes increasingly longer and longer... with events such as the earthquake in Chili alone, the rotation of the earth will miss it's mark at the equator by some 8 inches compared to where it should have been had there not been an earthquake at all.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Beau Schwabe

IC Layout Engineer
Parallax, Inc.

Tracy Allen

Here is a table of leap seconds, with the reasoning behind them:
tf.nist.gov/pubs/bulletin/leapsecond.htm.
The most recent leap second was added at midnight of 31-Dec-2008 going into 2009. There was also one added in 2005. The table shows 24 leap seconds added going back to 1972, all positive to account for lengthening years. As of Feb. 25, 2010, UTC was 61 milliseconds behind UT1 (astronomical time measured by transit of stars).

The Chilean 8.8 earthquake shortened the day by a little over 1 millionth of a second, according to calculations. That is too small to measure, though, the margin of error being around 20 microseconds. The margin of error given on the above web site for the difference between UTC and UT1 is +/- 5 milliseconds.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Tracy Allen
www.emesystems.com

DynamoBen

I started an (S)NTP object last year. While its not finished I believe where I left off allowed me to retrieve the time via SNTP, but not much else (notice the unfinished/used portions of the code).

This project was a big help:
tuxgraphics.org/electronics/200710/avr-ntp-clock.shtml

Hopefully this helps some.

Post Edited (DynamoBen) : 3/24/2010 5:17:33 PM GMT

DynamoBen

I just found the time conversions for the tuxgraphics time project I posted earlier, this should help a lot.

Timothy D. Swieter

Great resources here!!! My other time problem (lack of being able to get back to the code because of work) is now killing me. I don't suppose the forum can help with that problem though!

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" 16:9 LCD Composite video display, eProto for SunSPOT, PropNET, PolkaDOT-51
www.tdswieter.com

Timothy D. Swieter

Thanks for the responses from everyone. There is a treasure chest of resources here. I started using Harrison's obex.parallax.com/objects/36/. I added a constant for the epoch date of January 1, 1900 which is what NTP is based upon. After some initial testing it appears the the algorithms are wrapping around and I think it is because of the note regarding 68 years.

For the moment I am going to bookmark this and come back to the topic in the future. I need to focus on my original task of getting the WIZnet W5100 driver working and then return to certain protocol parsing. Some many great distractions though.

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Timothy D. Swieter, E.I.
www.brilldea.com - Prop Blade, LED Painter, RGB LEDs, 3.0" 16:9 LCD Composite video display, eProto for SunSPOT, PropNET, PolkaDOT-51
www.tdswieter.com

Rayman

Could be...· UNIX time suffers from the Y2K38 problem and it starts in 1970...

http://en.wikipedia.org/wiki/Year_2038_problem

▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
My Prop Info&Apps: ·http://www.rayslogic.com/propeller/propeller.htm

My Prop Products:· http://www.rayslogic.com/Propeller/Products/Products.htm