Proposed Clock format for propeller programs...
Cluso99
Posts: 18,069
General 32bit Date/Time format proposal for propeller programs...
A modified FAT16/32 Date/Time format...
FYI: seconds in 4yrs = 126,230,400. So unsigned 32bits can hold 132 years.
From 1904 to 2096, every 4 years are leap years. So, no need to worry about 100 year leap years.
If we also hold a 32bit value of seconds, I suggest we use a base of year 2000. I am unsure if this is necessary as well as the date/time value.
1. It is quite likely that Kye's SD driver could be modified to maintain the clock value in hub between SD usage (Dumb RTC using prop). Lonesock does this in his fsrw routines (using just a seconds counter in hub).
2. If a real RTC is used, then an object could be used to read and set a real RTC clock chip, and update the SD RTC counter value(s). Or it could be done via a separate cog and object.
3. Any program could obtain the seconds/time at any time by simply reading a live hub location. No need for a request because it will always be updated by the RTC cog in the background.
Any ideas or comments on...
* The format ??
* The best way to implement this??
P.S. I have asked similar questions relevant to the Catalina implementation. It is also relevant to my OS and others. I hope this does not offend the moderators for asking similar questions on other respective threads.
A modified FAT16/32 Date/Time format...
[FONT=Courier][SIZE=2][FONT=Courier][SIZE=2]Year20xx Month Date Hours Minutes Seconds (00-63) (1-12) (1-31) (00-23) (00-59) (00-59) 000000___0000____00000___00000____000000____000000 [/SIZE][/FONT] Above uses year base 2000 providing up to year 2063, and accurate to the second.[/SIZE][/FONT] [FONT=Courier][SIZE=2]We can update the whole 32bit value at once, so no possibility of reading between a partial update.[/SIZE][/FONT] [SIZE=2] [/SIZE] [FONT=Courier][SIZE=2]Conversion to FAT16/32 file format is simple. Shift >>1 then add 16<<25[/SIZE][/FONT] [FONT=Courier][SIZE=2](i.e. Shift right 1 place, then add 16 to the year because base is 1984 instead of 2000) [/SIZE][/FONT] [FONT=Courier][SIZE=2]Year1984+ Month Date Hours Minutes Seconds*2 (00-127) (1-12) (1-31) (00-23) (00-59) (00-58) 0000000___0000____00000___00000____000000____00000 [/SIZE][/FONT] [FONT=Courier][SIZE=2]Above FAT16/32, the seconds are only counted every 2 seconds because it is 1 bit short! This gives years 1984-2111, but note 2100 is not a leap year. The time is held in 1 16bit value and the date in another 16 bit value.[/SIZE][/FONT]
FYI: seconds in 4yrs = 126,230,400. So unsigned 32bits can hold 132 years.
From 1904 to 2096, every 4 years are leap years. So, no need to worry about 100 year leap years.
If we also hold a 32bit value of seconds, I suggest we use a base of year 2000. I am unsure if this is necessary as well as the date/time value.
1. It is quite likely that Kye's SD driver could be modified to maintain the clock value in hub between SD usage (Dumb RTC using prop). Lonesock does this in his fsrw routines (using just a seconds counter in hub).
2. If a real RTC is used, then an object could be used to read and set a real RTC clock chip, and update the SD RTC counter value(s). Or it could be done via a separate cog and object.
3. Any program could obtain the seconds/time at any time by simply reading a live hub location. No need for a request because it will always be updated by the RTC cog in the background.
Any ideas or comments on...
* The format ??
* The best way to implement this??
P.S. I have asked similar questions relevant to the Catalina implementation. It is also relevant to my OS and others. I hope this does not offend the moderators for asking similar questions on other respective threads.
Comments
-Phil
You really want to do that in the propeller?
I would tend to keep it as simple as possible, which is: If you need a full date-string, use an I2C RTC attached to the EEPROM-pins and read it at least once a day to get the date-part freshly - which then includes leap year changes correctly. If you need the date very often - like when you need periodic logging - I think it makes sense to convert it into the format proposed by Cluso99, as it's
a) easier to update - see the quoted statement above
b) easier to convert to a string
c) gives us another 25 years on top of unix time until we have to switch to 64 bit ;o)
1. It is almost totally usable with the fat format... 1 shift and 1 add. We should use this in the os.
2. It has 1 sec accuracy where fat is only 2 sec. This was most likely because of word alignment that we do not require in the prop.
By using a base that is a leap year, it is simpler to calculate. I re fer to unix here.
Don't forget we ned to convert to/from fat time for file updates.
add an RTC
add a RF-time receiver
add a network-time request
or have a "never switch off-system"
.. well ... you also have the possibility to ask for the date/time with each switch-on ;o)
For me option 4 and 5 are most unlikely! And all other 3 already do the handling of leap-years. You have to read their date anyways if you switch on the propeller! So, re-read at 00:00 is enough to deal with the leap-year problem.
There is also a routine to format a string for display purposes. And routines to convert to/from FAT date/time format.
There are routines (commented out and untested) that can be called at least once per second. Well, actually it can be less often, but the seconds will jump. As long as the CNT does not overflow (~50 seconds @ 80MHz or ~40 seconds @ 104MHz).
Next is to add code to initially read from the RTC (I use a DS1340C) or if no RTC, then ask the user.
Here is an extract of some portions of the code...
' Date/Time Format: Year20xx Month Date Hours Minutes Seconds ' (00-63) (1-12) (1-31) (00-23) (00-59) (00-59) values ' 000000___0000____00000___00000____000000____000000 ' 3 2 2 2 2 1 1 1 1 0 0 0 \ bits ' 1 6 5 2 1 7 6 2 1 6 5 0 / PUB CountSeconds | Timer, i Timer := cnt date_time := 12<<dt_years_shift | 3<<dt_months_shift | 26<<dt_days_shift { } | 18<<dt_hours_shift | 44<<dt_minutes_shift | 56 repeat waitcnt(Timer += clkfreq) ' accurate 1 second delay UpdateTime PRI UpdateTime | flag, days, mths date_time++ ' secs +1 if (date_time & DT_SECONDS_MASK) => 60 date_time += DT_MINUTES_INC ' mins +1 date_time &= !DT_SECONDS_MASK ' secs =0 'next: the shift is on the rhs because it evaluates at compile time so it's faster if (date_time & DT_MINUTES_MASK) => (60 << DT_MINUTES_SHIFT) date_time += DT_HOURS_INC ' hour +1 date_time &= !DT_MINUTES_MASK ' mins =0 if (date_time & DT_HOURS_MASK) => (24 << DT_HOURS_SHIFT) date_time += DT_DAYS_INC ' day +1 date_time &= !DT_HOURS_MASK ' hours =0 days := (date_time & DT_DAYS_MASK ) >> DT_DAYS_SHIFT ' extract the days mths := (date_time & DT_MONTHS_MASK) >> DT_MONTHS_SHIFT ' extract the months flag~ if days == 0 '(i.e.=32) ' 32: it is 0 because we only have 5 bits date_time -= DT_MONTHS_INC ' so remove the rollover to the month mths-- ' " " flag~~ ' 32: Jan/Mar/May/Jul/Aug/Oct/Dec: mth+1 elseif days => 31 ' 31: test for 31 days in mth if mths => 8 ' if mth Aug/Oct/Dec +1 to make odd; Sep/Nov become even mths++ ' Jan/Mar/May/Jly are already odd if (mths & 1) == 0 ' now test for an even month which has 30 days flag~~ ' even= Apr/Jun/Sep/Nov: mth+1 elseif days => 29 if mths == 2 ' February? if (date_time & DT_LEAPYEAR_MASK) <> 0 ' not leap year? flag~~ ' not leap year & 29: mth+1 elseif days => 30 flag~~ ' leap year & 30: mth+1 if flag ' inc month? date_time += DT_MONTHS_INC ' months +1 date_time &= !DT_DAYS_MASK ' days =0 date_time += DT_DAYS_INC ' days +1=1 if (date_time & DT_MONTHS_MASK) => (13 << DT_MONTHS_SHIFT) date_time += DT_YEARS_INC ' year +1 date_time &= !DT_MONTHS_MASK2 ' months =1 (months already odd) PRI ConvertToFAT(dt) Return (dt >> 1) + (16 << 25) PRI ConvertFromFAT(dt) Return (dt - (16 << 25)) >> 1Propeller_RTC - Archive [Date 2012.03.27 Time 01.02].zip
We had a discussion about logging, and it seems that an RTC is only good to seconds, and we can get very good time sync over the internet, and great time from GPS. (So we're thinking RTC is not going to be a mandatory part for most applications).
The thought was most things would be hooked to the time source that supplies the proper accuracy, IE apps for GPS or internet.
The exceptions would be sort duration apps that are not connected to GPS or internet, these would have to be (manually) sync'd to something external anyway. For these short duration apps, one could just count clock cycles, and use a second cog register or hub word to count overflow. A cog would be good for up to 2,668,799 days, or about 7,311 years. A hub word would be good for 40 days. Being short term, they by definition wouldn't run long enough to drift too much before restart or resync. This would cover 80-90% of of cases.
Based on this, it looked like linux time would simplest. But we didn't commit to any method yet. And this was focused on logging rather than FAT compatibility.
The internet is no where near as accurate as you cannot determine the latency of the packets received. And of course, this presumes that you have an internet connection to the prop - more expense. So a lot really depends on the apps as to what is required.
What I have is the capability of a cog, or a part of a cogs shared code, to do a soft RTC which may (or may not) have a real RTC in the background to initialise the date/time. One of the circuits I am working on has 3 props, 512KB SRAM, microSD and an RTC, amongst other things. We need to be able to keep (relative) accuracy down to 1 second.
As for the format of the time, I am running a Prop OS on the SD card and FAT16/32 is used. So I already have to deal with the FAT time format. I just did not want to use the exact format because of the 2 second ticks. I may well keep a seconds counter too. If so, I will start it at 00:00:00 on 1Jan2000. BTW I am ignoring any leap seconds.
If you are only really interested in file timestamps then 1 second resulution and a limit of 32 bits might be enough.
If you are doing calculations involving time you prefer a format that is linear - the proposed format is discontinuous.
For general use I don't think this is flexible enough - once you start talking calendars then you may have to represent timezone, deal with historical dates, keep track of the day of the week etc.
I don't think one type is general enough to be "the clock format". Given constraints of memory space and the highly domain-specific applications on microcontrollers people's requirements won't always overlap.
-Phil
Precision logging can easily be done using this proposed time-format plus the cnt-value. You'd simply log the start-time first. With the start-time/cnt you can even correct the time later on if updating the time-format value is to slow, as the seconds are redundant for up to 52 seconds.
After that initial start timestamp all the times you log are as precise as cnt. If it also has to be high-speed, it's of course better to output raw longs.
So, the only question is how precise can you setup the time variable when starting up the propeller. A longwave atomic clock signal would propably be the best if you are close enough to a sender.
Unix handles high resolution logging by appending a microsecond resolution timer to the epoch seconds. You can see this easily while using tcpdump. The kernel stamps every packet with a high resolution timestamp, so you get very good resolution. It would be possible to emulate this with the cnt variable, just some clever coding is required to handle overflow.
I would think that once per second you calculate a modulus and offset value to be applied to the system counter, so you get the proper value. I haven't thought it out more than a few moments, so it could be simpler than that.
The Posix standard really messed up in its handling of leap seconds (by leaving them out it made calculating elapsed time very difficult, which was kind of silly given that the base "count of seconds" format is otherwise very convenient for elapsed time). It is a widely implemented and used standard, though, so it's a logical choice.
Eric
Updating is of course a little more complex especially because you can use a counter for the 32 bit integer solution, but converting to string should be much faster. So, actually it depends on the usage whether one is better or the other.
But this is programming ... it's always about the yin and yang - and you can't have both at the same time ;o)
I am always looking to have my cake and eat it too
Actually, I have pretty much decided to keep both my modified FAT version and a seconds version starting with year 2000. I wont be worrying about leap seconds - someone can add a table if they want, but the future leap seconds are not decided, so they cannot be inbuilt anyway. This method will allow 132 years = 2132 but I am not taking 2100 into account as it is really not a leap year. If anyone wants UTC, just add "126,230,400 * 7 + 31,536,00 * 2" = 157,766,400 (30 years in seconds from 1970 to 2000 excluding leap seconds) to the long.
The reason I am using 2000 as the base is because it makes the maths a little easier, and because I need something that uses 2000 as a base for tables. Also it keeps the 2 longs in sync.
Maths is easy... 4 years = 126,230,400 seconds, so keep subtracting this while it is > seconds, accumulating years as +4
Then subtract 31,622,400 for 1 year (leap year), and then 31,557,600 for up to the next 3 years
Once you have the leftover seconds, they will be < 1 year, so now subtract as follows...
- 2,678,400 for Jan
- 2,419,200 for Feb (non leap year) or 2,505,600 (leap year)
- 2,678,400 for Mar
- 2,592,000 for Apr
- 2,678,400 for May
- 2,592,000 for Jun
- 2,678,400 for Jul
- 2,678,400 for Aug
- 2,592,000 for Sep
- 2,678,400 for Oct
- 2,592,000 for Nov
- 2,678,400 for Dec (you will not get here unless you made an error above)
Once you have a smaller remainder than the next months subtraction, subtract as follows...
- 86,400 per day
- 3,600 per hour
- 60 per minute
and your seconds are left (and should be <60)
Dont forget to start you month and day at 1, not 0.
This is a lot quicker using interger maths than divides!!
Because we can update the hub by writing longs, it will not be possible for another cog to read it in the middle of a rollover.
It is also possible to do this with 64 bit values, as long as all COGs follow the same protocol. If they always write the low word first, followed immediately after by the high word (in the next hub window), and similarly read the low then high word in 2 consecutive hub windows, then all the COGs will see consistent values for the 64 bit value. The only downside to this protocol is that not all high level languages are able to guarantee that the 2 longs will be read in consecutive hub windows, so some PASM glue code may be required.
-Phil
As for unix, 2032 is now shortsighted. It wasnt back in 1970, and look at how much code is still running. But all is not lost, because by the time we get near 2032 they can use unsigned date and squeeze another 68 years out of it, instead of using -ve to go back from 1970. And we will be on 64 bit processors before you know it anyway.
-Phil
The algorithm is the same as calculating Julian Day except of course the base year is 1/1/4713BC.
I am going to run a time test on the conversion and I will post the results.Perhaps I can avoid holding two formats after all. Just holding seconds is way easier if the conversion doesn't take too long.
BTW Nice job on the web docs!