My first Y2038 Bug encounter.
Heater.
Posts: 21,230
I just hit my first year 2038 bug!
Perhaps not many Parallaxians are interested in or affected by this but I just hit my first Y2K38 or Y2038 bug as it is known elsewhere. All users of 32 bit systems should be aware of this impending catastrophe.
In short my issue was this: I was creating some keys and certificates with openssl to use for secure client server communication over the net. This is actually part of a project to get my Propellers on the WEB so you will be interested one day:)
Anyway, when you create public keys/certificates you can specify a time span, in days, for which they are valid. After that they expire and can no longer be used. I set the time span to 9999 days which means they should expire some time in 2040.
The certificates created by openssl did not work. Eventually I discovered the certs were set to expire in 1904. WTF!
I slept on the problem and then the light went on. You see many computer systems have been patiently counting seconds since Jan 1, 1970, the "epoch" as it is known. That is how they know what date and time it is and how to calculate differences in dates etc. Problem is a 32 bit machine will store that count as a signed int which will overflow at 2:14:07 AM GMT on January 19, 2038. BOOM all hell will break loose.
So, it turns out that openssl on my 32 bit Linux box is using 32 bit ints to calculate those certificate expiry dates and in my case they rolled over to 1904! When I repeated that certificate creation on a 64 bit machine the certs worked fine. When I recreated them with lesser expiry dates on the 32 bit machine the certs worked just fine.
There is a lot more about Y2038 issues here: http://www.y2038.com/
Note: Linux itself has been using 64 bit ints for time keeping for a long while now so should be no problem. Some apps on the other hand, like openssl, may have issues. Notice how this has the possibility to cause problems in the future. Even if the app is fixed before 2038 it can be embedding erroneous date/times in data now!
I guess Windows and Mac have been fixed already but we are going to need a 64 bit Propeller soon:)
Perhaps not many Parallaxians are interested in or affected by this but I just hit my first Y2K38 or Y2038 bug as it is known elsewhere. All users of 32 bit systems should be aware of this impending catastrophe.
In short my issue was this: I was creating some keys and certificates with openssl to use for secure client server communication over the net. This is actually part of a project to get my Propellers on the WEB so you will be interested one day:)
Anyway, when you create public keys/certificates you can specify a time span, in days, for which they are valid. After that they expire and can no longer be used. I set the time span to 9999 days which means they should expire some time in 2040.
The certificates created by openssl did not work. Eventually I discovered the certs were set to expire in 1904. WTF!
I slept on the problem and then the light went on. You see many computer systems have been patiently counting seconds since Jan 1, 1970, the "epoch" as it is known. That is how they know what date and time it is and how to calculate differences in dates etc. Problem is a 32 bit machine will store that count as a signed int which will overflow at 2:14:07 AM GMT on January 19, 2038. BOOM all hell will break loose.
So, it turns out that openssl on my 32 bit Linux box is using 32 bit ints to calculate those certificate expiry dates and in my case they rolled over to 1904! When I repeated that certificate creation on a 64 bit machine the certs worked fine. When I recreated them with lesser expiry dates on the 32 bit machine the certs worked just fine.
There is a lot more about Y2038 issues here: http://www.y2038.com/
Note: Linux itself has been using 64 bit ints for time keeping for a long while now so should be no problem. Some apps on the other hand, like openssl, may have issues. Notice how this has the possibility to cause problems in the future. Even if the app is fixed before 2038 it can be embedding erroneous date/times in data now!
I guess Windows and Mac have been fixed already but we are going to need a 64 bit Propeller soon:)
Comments
Doesn't the Propeller 2 have a 64 bit CNT register? That's almost like a 64 bit counter from January 1, 1970 (except for running 150 million times faster!).
Interesting problem, though. I was too young to enjoy the Y2K spectacle.
So, let's see. The max count as a signed integer is 2 to the power 63 minus 1 or 9,223,372,036,854,775,807.
Well that's not so long. Python tells me:
9223372036854775807 / (150 * 1000000) / 60 / 60 / 24 / 365
Is only 1949 and a bit years. (Assuming a 150MHz clock).
So we can expect problems in the year 3919.
Still with the P2 the issue is that when doing these date type calculations you still only have a 32 bit engine and would need to take special care.
Here in Taiwan, the year is 102.. not 2013. All banking is done in the years since the founding of the Republic of China. Thus the Y2K bug was not a potential problem within their system. Think about it, there are other nations with other calendars... not everyone is locked into Christian era.
True, but I bet many of them whatever calendar they use are still using 32 bit time ticks on their computers and the same epoch as Unix:)
The thing about this bug is that despite the fact that 2038 is probably a date I'd be lucky to see it was causing me problems now!
Fortunately the result of this bug was an immediately non-working system and I could look into fixing it. But what if the wrongly calculated result were buried in some data only to cause something to fail when its in production? There might be some odd surprises for some people.
I'm very interested in the openssl stuff you're working on. Is the Prop handling the encryption and the CA handshake? If so, how larger is the code footprint?
Loopy, that made me bust out laughing! Since I should be retiring in 2037, I am choosing not to care about the 2038 bug as well. :thumb:
The plan here is that the Prop or Props take care of the hard real time stuff. The Rasperry Pi or whatever embedded ARM board takes care of the networking, including encryption, authentication etc.
I cannot imagine that the Prop or Prop II has enough memory to handle all of that.
I wonder... My bank books, my telephone and ADSL bill, my water bill, and my cell phone bill are all dated in the Year 102.
If you go to Thailand, you will find that they have yet another system of years. The really big companies may use Windows or something at the desktop, but the mainframe is likely to be a different system. And often it is a Japanese computer system in the banks here.
Also my driver's licence, my driver's insurance, and my health insurance are all in statements in Republic of China years. The postage stamps and the money are too.
Yes, the point is it does not matter what representation us humans finally see the dates and times as. What matters is what is the representation inside the machine.
So far, I see no evidence that machines in Asia or wherever would be using anything other than the Unix epoch of January 1st 1970. Especially, as you have said, most of Asian business runs on pirate copies of Windows:)
I addressed that concern at an age when most people would have been looking to blow every penny they had each weekend. My retirement portfolio does not depend on 401k, Social Security, or any of the other "risky" topics discussed today. I also started a very nice life insurance package when I was 20 years old. Looking back, I don't know why I did, but very glad that I listened to whomever told me to do so. My wife and daughter are very well secured in the event that I leave first.
I am saying that the computers in Taiwan's government and banking and communications all use Republic of China years. You may think it is all Unix epoch, but these facilities pre-date the advent of Unix. Unix was and is mainly for educational institutions world-wide. There were computers before 1970 and Unix.
There is another world of computer hardware out there and IBM is a huge player is services for government computing. This world is not visible to you or I.
Saying that most of Asia'sbusinesses run on pirated copies of Windows may be somewhat true, but NOT most of Asia's governments, banking, commuications, and infrastructure.
Thailand for instance has the 4th largest cash foreign reserves in the world and may be operating on a completely separate calendar. People's Republic of China may not be using Christian era, and so on.
You seem to think that the world is manged from desktops, rather than mainframes.
All my Taiwanese income tax records are in Republic of China years and I suspect all the data of my living in Taiwan since 1994 is on a mainframe in R.O.C years. And since it was an early Chinese language system, I doubt that Unix was involved. It likely evolved out of IBM developing a national computer system for occupied Japan.
You are right, I have no idea about those old mainframes or their modern descendents, it's a world not visible to me very often. Apart from all those Android devices and, oh yeah, let's not forget the Apple products. Oh, and a big chunk of the servers on the internet. Oh and all those super computers. Well, OK, I'm counting Linux as Unix, I think that's fair enough. Not at all.
The thing is the Y2K threat was a huge attempt to fear monger people into buying a new upgrade of Windows. Your Y2038 bugs has a few years to be sorted out. And the Y2K had many years to be resolved, but the fear mongering went on until it was obviously the non-event it was.
And then there is the real world of computing rather than the world of MS, Apple, and Linux. There is a whole different world of governmental computers that are fire-walled from the craziness of the internet. There are also specific cultural barriers, historical political relations, and language differences that have created some systems that are hidden from view. Just consider what Oracle sells in servers and all the Sun Systems software that it now owns. Hewlett-Packard and IBM have hardware platforms and OSes that are not intended as consumer products and never will be. Toshiba may be providing something just for Japan as it certain has some very interesting leadership in public domain database software. Samsung in Korea is likely to be the provider of choice for the South Korean government.
I am not sure what the PRC does for a calendar for recording governmental data, but as I pointed out Taiwan and Thailand have their own eras. Saudi Arabia is another one and it certainly holds a huge position in world wealth. Japan was part of the Trilateral Commission that was set up by President Truman at the end of WWII to rebuild both Japan and Germany. And the Tril-ateral Commission still exists to this day. Early on, Japan was encourged to use computers in government. But if you study Japanese history, they never did develop a written language of their own. They adopted Chinese characters to the Japanese language as an administrative tool in about 700 AD.
So, any computational representation of Japanese was also useful to represent Chinese unless you add in the Japanese phonetic representations.
I suspect Saudi Arabi was also an early adopter of computers with attempts to represent their own language rather than use the English alphabet.
And of course Russia had to use Cryllic, not a Roman alphabet with their independent cold war use of computers.
None of this came together until rather late, with Unicode. Windows 98 was certainly not as Unicode friendly as it hoped to be. So I suspect that governments being what they are just continued to be slow adopters. And with the Y2K threat being hyped everywhere, why would any government give up a system that was isolated from it by incompatibilty?
Android, Windows, Apple, and LInux are all for the consumers to play with and not disturb the powers that be.
I'm not sure why you want to dump Linux in the consumer bin or call it a play thing. There are some monster businesses running a lot of Linux from Google downwards. IBM does a lot of work with Linux. Then there are the super computers. And so on. In fact consumers are very Linux resistant to Linux unless you hide it out of sight and rename it like Android or Chrome Book.
They brought in certification of separate production lines and separate inventories - one without lead or heavy metals and the other with.
I believe Parallax, like just about everyone else finally got rid of the consultants and decided to have just one production line with no lead or heavy metals. Of course others may have just decided to stay with one production with lead and heavy metals.
But the consultants, the planning meetings, the separate certifications were pretty much a lot of expensive nonsense. Duplication is costly. Duplicate inventories are very costly. Duplicate production lines with limited markets are also extremely. But it was all great spin for highly paid consultant to create billable hours. It has been said, A consultant is a guy that borrows your watch to tell you what time it is.
Linux and Unix are not merely consumer items, and the Unix file system started as multi-user.. so security in much better than Windows that started from a single user OS. Also, Linux is 70% of the functioning backbone of the Internet. So it is a serious OS, but the Internet is a huge network that is primarily for advertising and entertainment to the public these days with related businesses. I suppose that since the network itself was designed to survive a nuclear war, it is quite durable.
You may be right, Linux seems to crop up everywhere. And it does a very good pro-active job of swatting down security threats before they become huge problems.
I spent a few years teaching English language articles of fear and loathing about the coming of Y2K, but the threat never made sense in Taiwan. So the classes just picked it apart and joked about it. I guess that jaded my point of view about how dire a roll over of years is to an OS.
These certifications that you mention are likely to go through several changes for other reasons before 2038. Encryption may be better, new standards evolve, 64 bit machines may dominate before then.
It all may be that I've just had too many lattes today.
Anway as far as I can tell a lot of work has gone into future proofing the system as it does allow for different crypto systems and key lengths etc. New ones can be added I'm sure.
The issue I had was to do with expiry dates. You see, as a root certificate authority I can sign a certificate for you which then allows you to sign certificates for others. This can go on down a long chain until someone ends up signing keys that people use.
Now as a root CA or other CA down the chain I might want to only give you permission to do that for a limited time. Hence the expiery date in the certificates. Once it expires you can no longer sign certificates and niether can anyone below you.
Well this seems like quite a big deal and I wonder how that bug has been allowed to persist so long. Except of course real CA's. like the ones you have to pay, will be using 64 bit machines I guess.
How can a bug exist so long? Take a look at the bug reports for the software.. Some bugs are tough to solve and have been known for years, others get knocked down in a day.
But Linux does have a system that allows you to monitor the progress that is being made. And that is much better than waiting for Apple or Windows to get around to something without telling you.
When I hit Google I find it has been reported and fixed in Ubuntu a year or so ago and mentioned in a few places. In the first page of links found is a link back to this thread!
So it's still a mystery...
I was just trying to generate keys for use in embedded systems that I literally will never physically have access to again in my lifetime.
So I set it for a very long expiry date. Same problem. I could only generate keys that expire in 2038.
Nowadays there's some fork? (if that's the right term) of OpenSSL that does use the 64-bit time. Of course it's too late as my root authority key that I'm making other keys with will expire by then. But of course this was never announced as a big thing, as they like to sweep bugs under the rug, and inquiries are met with "we know that already" responses.
Unfortunately, barring nuclear war, the world will be the same old boring place in 2038 as it is now.
I say this from the perspective of how the houses in towns look, what the roads are paved with, what fuels cars run on, and such. In a lot of places there still will be ancient embedded linux 100 mbps routers still sending IPv4 packets. I guess 2038 is when they will all go silent.
What I'm saying is that I can't possibly believe, even on let's say a crappy 8 MHz system, that anyone would be so short-sighted as to implement 32-bit time that measures ticks in seconds. As if timestamps and seconds based time comparisons are such a common thing used so many times a second (sarcasm) that it really needs to be optimized that much.
I guess I'm jaded in the fact that there is no real mainstream operating system and no manufacturing company that will commit to building a device that will act the same, use the same file structure, the same user interface, (same everything), for hundreds if not thousands of years. Why is it always this inexorable race to obsolete things that are presently functional? What I'm getting at is the guts of the device could change from hundreds of chips and discretes, and ultimately become one single chip on a tiny circuit board in a huge empty case as technology permitted, but everything about the first device to the very latest device would, from a human perspective, act the same, perform the same, configure the same, so any guides written and any backups made would just work. Individual devices need not last a hundred years, just how the series acts and looks.
LCD wristwatches, with their mode and set buttons, seem to be the only closest thing that generally acts the same from the 80's to now.
What I'm saying is I wish things would slow down a little. Every time it seems we get all the bugs worked out of something and its features and limitations are perfectly understood, and there are no surprises, then boom, something new and shiny (and bug-infested) appears. Change for the sake of change, and things built in such a way to fail shortly thereafter the developer's death, as a sort of "stick it to them" mentality.
Don't tell anyone but all this software and technology churn is just a job creation scheme for engineers:)
Anybody remember the 16 bit roll over? And the disaster that paved the way for the Y2K hoax?
On a smaller scale I have seen (back in the eighties) December->January transitions with time handling bugs that were literally life-threatening when they happened.
-Tor
Sorry to hear the Bug Report Process has been difficult to track down, but at least there is one. And there are alternatives to OpenSSL if it is not being well supported by an active community.
I would go to www.distrowatch.com and see what alternatives to OpenSSL are being used by the top 5 or more distributions. It really helps to stay with the mainstream.
I do see that Mint is using OpenSSL 1.0.1e ... so I wonder if you are getting the latest version's info. Ubuntu is using 1.0.1c Maybe everyone is using OpenSSL, but various versions.
But see the bottom of this link for alternatives.
https://wiki.linuxfoundation.org/en/OpenSSL
Yes indeed, you are guessing about Windows and Mac fully anticipating a bug that will cause issues 25 years from now. I really have my doubts that they think that far ahead. After all, it is likely that even Windows8 and whatever Apple currently selling will no longer be supported by then.
I was surprised at Debian for not having a fix for it in place as this is a security related issue which they like to be hot on.
Rather than debate, I should just admit that there are some problems that everyone complains about but no one has identified what the real problem is or the best solution. I have one with my Firefox Browser and Chrome Brower not working right in my local Starbuck Wifi hotspot.
Ironically in trying to figure out your OpenSSL issues, I may have found that I can fix this by switching to another browers, Opera as it supported the leading edge security in Linux, as well as other OSes.
And so, I do admit that there are times that Linux just doesn't have everything one wants available today. I am still very happy with it. It costs nothing and I learn ever so much more. MicroSoft has beaten me up very badly with various need releases of OSes that both failed and were costly. Vista was the most recent excursion into such. Any OS has warts and bugs, but the ones that charge money are willing to pretend to be superior.
I just can't see why you cannot accept an shorter time period to expiry date. But I really am not well versed in OpenSSL, SSL, and all it implies.