kwinn you are right in the case of bitmap if you did it with jpg altering the fft values after compression then it would not be noticeable because jpg are already not compresable
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
mctrivia said...
kwinn you are right in the case of bitmap if you did it with jpg altering the fft values after compression then it would not be noticeable because jpg are already not compresable
That's interesting.· I know nothing of the internal format of JPG files -- for all I know, that lsb might be essential to the format.· Do the JPG files still work as JPGs after you do what I now know to call steganography?
Thanks to Kwinn for giving me a new word.· Very few people can do that.
in jpg you could not just replace the lsb and have the image look correctly but you could cleverally alter wich number is used.
basics of jpg:
1) take a group of pixels
2) do fast forie transform to convert to frequency domain
3) multiply f domain value by matrix that squashes resolution in frequencies human eye does not see well but keeps resolution in ranges that are seen well.
4) use hufman compression algorithm to make values that show up more often have less bits then those that show up not as often.
because of the last step you can't just change the lsb or you would be picking a completely different color. but if you sellectively chose a color from the list that was similar that had a lsb that you wanted you could do it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
Doubly interesting.· I've been reading about, and playing with -- sometimes working with -- cryptology and cryptanalysis for many decades, starting with H. F. Gaines's early textbook Cryptanalysis and progressing through many others.· Of that book, I own the only copy I've ever seen.
And I don't recall encountering the word "steganography".· I don't forget words.
Googling that word, I find I have missed a lot.· There's an excellent Wikipedia article on steganography,
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
for OTP would the real random number generator in the obex work for generating keys? or is there a problem with its randomness?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
mctrivia said...
for OTP would the real random number generator in the obex work for generating keys? or is there a problem with its randomness?
I've looked at it a bit.· Not having NSA's ability to find subtle failures of randomness, I couldn't say whether its randomness has problems or not.· Randomness is a famously difficult thing to test.
The fact that I can't tell is a sufficient problem in itself.· I wouldn't trust it for cryptological purposes.
I did a lot of testing on strings I thought were random.· In addition to standard things, like character frequencies, digraph frequencies, and so forth, I converted them to WAV files and listened to them.· I've found nonrandomness (detectable tones and repetitions) in stuff that seemed really random when tested any other way.· So I'm very distrustful.
Maybe I ought to make bitmaps out of them too, and look for visual stuff.· Hmmm --
And even then I can't hope to be as good as NSA at finding subtle departures from randomness.
So, every bit in one of my random strings is an XOR of several hundred individual bits (that "several hundred" is also as random as I can make it, with a minimum and a maximum) derived from measurement of thermal noise.· That's slow, but it may be good enough.· I hope.· But I can't know.
I also have stuff in there that attempts to make truly random the time between samplings of the random noise.· As I say, it's very slow, but I can set a PC doing that and ignore it for weeks at a time while it makes keyfiles.· I have lots of extra PCs, so it doesn't matter.
When the Rand Corporation generated "real" random numbers for their book, they discovered that the numbers were biased. So they removed the bias with, I believe, Von Neumann's algorithm, which is explained here.
Public private key encryption was first broken in 1943 by the British. They got their post office (telephone department) to build the world’s second computer (the first electronic computer) to do it. This was not the Enigma, Heath Robinson device used against German mobile encryption units but a top secret project. The computer was a 5 bit device called Colossus. The British only released information about the project in 2001. A lot of the decrypted messages will remain secret until 2021. There must be some damning stuff against the British Government there…
Their decryption jobs were made much easier because the disciplined German communications staff always started their messages in exactly the same way. The Germans also believed that the code was impossible to crack, this despite clear evidence that the British had done so.
Encryption can always be broken. Your NIA/CIA do it regularly and often with evil intent. A personal example - the CIA decoded messages from President Samora Machelle of Mozambique and conveyed the information to the Aparthate Government with the suggestion that the South Africans shoot down his plane. No one denies that Machelle was a committed socialist but everyone is equally convinced of his greatness. Note how his wife, now married to Mandela, has carried on his great and noble work. His sins, a refusal to give up his socialist dream and a refusal to allow CIA to operate from Mozambique.
When encryption is weak, fear not those criminals and thieves. Fear rather amoral organs of state who justify their actions with slogans like “defending the nation” and "National Security".
kwinn you are right in the case of bitmap if you did it with jpg altering the fft values after compression then it would not be noticeable because jpg are already not compresable said...
mctrivia, you are correct, however jpg is already a compressed format, and changing any bit in that file (or any compressed format file) will make the file decompression impossible (indecompressible???, uninflatable???,) which would also be very easy to detect.
Phil Pilgrim (PhiPi) said...
When the Rand Corporation generated "real" random numbers for their book, they discovered that the numbers were biased. So they removed the bias with, I believe, Von Neumann's algorithm, which is explained here.
-Phil
Thanks, Phil.· Bias (in general, not just binary bias) is really easy to test for, though.· Simple character counts will do it.· It is cyclicity that's a bear.·
Suppose, for example, that a sequence is composed of eight-bit bytes, and they are all truly·random except that every nth byte, where n·is some·prime, is influenced by the (truly random) byte that precedes (or perhaps follows) it by m bytes, where m is some other prime?· Such a structure could be used to convey information in a signal that would pass every test for randomness (except perhaps NSA's tests, who knows?); and a one-time-pad system that relied on the randomness of such a scheme might be breakable because it isn't really random.
That kind of nonrandomness is a bear to test for.· Not impossible, but very expensive in processor time.· Notice that you've got to test separately for each pair of primes n and m, and for each n you've got to test the whole file·n*m times to detect it.· That's tough to do.
My OTP system doesn't use any such structure (which would multiply the original message size by n, very inefficient), but it does rely on the true randomness of the keyfile.· Randomness is tough to test for.· Code-breaking specialists have to be inhumanly good at testing for prime-related nonrandomness -- and therefore I assume that they are inhumanly good at it.
John Bond said...
Public private key encryption was first broken in 1943 by the British. They got their post office (telephone department) to build the world’s second computer (the first electronic computer) to do it. This was not the Enigma, Heath Robinson device used against German mobile encryption units but a top secret project. The computer was a 5 bit device called Colossus.
Heath Robinson wasn't an Enigma-cracker either.· It was directed against the German radioteletype encryption, and it worked by comparing long punched paper tapes against each other with various offsets (the same type of problem I mention in a recent post).· Colossus, which was not actually a computer in the modern sense (no stored program), was an all-electronic version of Robinson, and contained the world's first electronic memory.· The devices developed by GC&CS (Bletchley Park), by OP-20-G (US Navy), and by Arlington·Hall (US Army) in WW2·were essential parts of the technology that led to the first genuine stored-program computers -- but they weren't stored-program computers themselves.
One writer has remarked that among the greatest breakthroughs at Bletchley Park was the discovery that punched paper tape could be run at 30 mph without breaking.
To eliminate any whiff of cyclicity, one could generate an OTP with a detector sensing nuclear decay events. These are considered to be "fundamentally random" (whatever that means) and independent with no long- or short-term correlations.
Phil Pilgrim (PhiPi) said...
To eliminate any whiff of cyclicity, one could generate an OTP with a detector sensing nuclear decay events. These are considered to be "fundamentally random" (whatever that means) and independent with no long- or short-term correlations.
-Phil
But there are pitfalls.· You have to detect the decay events, for example.· Any detector will have a threshold between "there is a pulse right now" and "there ain't a pulse right now".· Such a detector may be subject, for example, to 60-Hz hum.· It may be in a room with loudspeakers.· It may be in a building that vibrates (or has·inconstant magnetic fields)·from air-conditioning blowers.· I'm sure you can imagine other sources of cyclic noise in the stuff between the decaying element and the bit recorded on disk.· And even if the noise isn't cyclic it can still be nonrandom.· Life is a witch.· Something like that.
The physicist R V Jones at the University of Lancaster worked on seismic detectors.· At one point he was puzzled by a cyclic effect in these.· He·wrote, Had I not been there at the time I would have been very puzzled by a rhythmic tilt of period about one minute, which at first looked like an unusual effect of a distant earthquake.· The building was gently rocking to and fro with an amplitude of a few parts in a hundred million, owing to the shift of weight as a conscientious cleaner moved herself and her floor polisher backwards and forwards progressively along the building.
In 1874, in Life on the Mississippi, Mark twain wrote, There is something fascinating about science.· One gets such wholesale returns of conjecture out of such a trifling investment of facts.
well both giger counter and nuclear fusion reactor are relatively easy to make
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
mctrivia said...
well both giger counter and nuclear fusion reactor are relatively easy to make
Umm, that would be very interesing to the Department of Energy and various university consortia, who've poured billions into fusion reactors without success. (BTW, it's "Geiger".)
You're right about detection bias and cyclicity, of course. I guess it's just a question of the timescale in which these measurement defects occur, and whether the defects can be filtered out by combining measurements that span a longer timescale.
-Phil
Where's Rich? He started this whole discussion with an extraordinary claim. We've yet to see the requisite extraordinary proof — or even evidence — to back it up.
Post Edited (Phil Pilgrim (PhiPi)) : 4/7/2009 5:06:19 PM GMT
the reactor I know how to build fuses duterium and can not be used for power generation as it requires a lot more power to fuse it then you get out. it is however useful for generating radioactive materials.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
Where's Rich? He started this whole discussion with an extraordinary claim. We've yet to see the requisite extraordinary proof — or even evidence — to back it up.
Maybe Rich is out of town for a Judo tournament.· That's where I usually am when I vanish for a while.· Gotta go watch my students compete, which is all one can do at my age.
The idea of counting fission events to generate random numbers in an interesting one.· I'm not quite sure I know how to do it, although I have experience in processing information from similarly random "shot" events.· I have a patent (expired) on a system that counted and processed pulses from a blood counter.· A blood sample diluted in saline was passed through a very small orifice.· When an erythrocyte (red cell) passed through the orifice (tiny hole in a thin glass membrane), the electrical resistance of the hole spiked.· We counted spikes to count red cells.· These were the same kind of random events one gets from atomic decay.· Sometimes the orifice gets blocked and you get a bad count.· We ran three orifices in parallel to get three counts.· My patented device took the three counts, computed the expected variation among them, compared them, and either calculated a best-estimate answer or threw away the result.· It did this as follows:· if all three counts are within one standard deviation (sigma)·of each other, take the mean of all three.· If two agree within one sigma but the third is goofy, take the mean of the two that agree.· If no count is within sigma of any other, turn on a red light and a buzzer and produce no result at all.
The statistics of random events like those are interesting.· In any time interval, one will get some discrete count.·How many is a measure of the overall rate (which in our case indicated the patient's red count).· The longer you count, the better your estimate of the average.· The standard deviation is (sqrt(n-1))/n, so it's easy to calculate the probable error in your result.
But how do you generate an unbiased, flatly distributed random number from counting such events?· It's probably simple, but I'm not statistician enough to know how to do it.· Will the time between events be flatly distributed, for example?· I have no idea.
odds are the lsb of the time is random don't know about distribute
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
The number of independent random events (such as radioactive decays) that occur in a known time interval follows a Poisson distribution, whose density function is given by
····f(k) = pke-p/k!, where
k is the number of events, and p is a scaling factor that depends on the process at hand and the length of the time interval and is usually determined by curve fitting observed data.
You can convert any random variable to a uniform random variate on the interval [noparse][[/noparse]0, 1] by inverting its distribution function (the integral of the density function over the random variable, from 0 to k). This process is more precise with continuous random variables (such as the time interval between decays) than with discrete random variables (such as the count per given time interval), especially where sparse events are being recorded.
If one wanted to use radioactive decay to produce random numbers a reactor is not required. You could measure the time between pulses from a scintillation detector or Geiger counter caused by background radiation and cosmic rays.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
I've been following this thread with interest, but I have absolutely no practical knowledge of cryptography. Does anybody know of any books that would make a good introduction? Something with both history and modern practice would be good, especially if it had code examples in a HLL or pseudo-code algorithms.
Simon Singh's The Code Book is pretty good for a history of cryptography up through quantum methods. For a look at steganogrophy, try Peter Wayner's Disappearing Cryptography. Both are entertaining and approachable reads; neither will make you an expert. (For that you'd need a Ph.D. in number theory. )
SRLM said...
I've been following this thread with interest, but I have absolutely no practical knowledge of cryptography. Does anybody know of any books that would make a good introduction? Something with both history and modern practice would be good, especially if it had code examples in a HLL or pseudo-code algorithms.
Here's a random selection from my shelves.· Few of these are really modern in a technical way, because I'm an old guy retired for a long time (and never more than a dilettante in cryptography anyway).
Double-Edged Secrets· W. J. Holmes.· United States Naval Institute, 1979.· My copy has no ISBN.· This is an account of USN code-breaking in the pacific, by a major participant.· Interesting to the historian but less so to the cryptologist.· I'm a history buff, too.
Code Breaking· Rudolf Kippenhahn.· The Overlook Press, 1999.· ISBN 0-87951-919-3 (hardcover); ISBN 1-58567-089-8 (paperback).· Some history, some technical explanation.··Information about pre-computer-age codes and ciphers, but also an excellent discussion of RSA and similar methods based upon large primes.· Includes public key encryption, digital signatures, etc.·
Cryptanalysis· H. F. Gaines.· American Photographic Publishing Company, 1939.· Dover Publications, N.Y. 1956.· No ISBN.· Original title Elementary Cryptanalysis.· From the 1939 date you will guess there's nothing modern here.· Useful background knowledge abounds, though.· Helen Gaines was an early practitioner in U. S. government service.
Algebraic Coding Theory· Elwyn Berlekamp.· McGraw-Hill, 1968.· Library of Congress number 68-17175.· No ISBN that I can find.· Not a cryptographic text at all, this is concerned with error-correcting codes and the like, and is intended as a graduate-level textbook.· Much useful mathematical treatment of codes in general, especially their information content (useful to know for cryptographic codes too) and error-correcting methods.· Possibly out of date, and the mathematics is now beyond me (once, it wasn't).
Codes, Ciphers and Computers· Bruce Bosworth.· Hayden, 1982.· ISBN 0-8104-5149-2.· Includes treatment of RSA, DES and public-key systems.
Battle of Wits· Stephen Budiansky.· Simon & Schuster, 2000.· ISBN 0-684-85932-7 (paperback 0-7342-1734-9).· Fascinating history of code-breaking in WW2.· If you really want to understand how it was done and who did it, this incomparable book is the first to seek.· I should have bought three copies; mine is wearing out, even though not cheaply bound.· Yours will, too.· Besides the historical interest, there is much technical information here also.· Get this book first and it will assure your continued interest in the field, while also providing a rich fund of background knowledge.
This book is absolutely a must in understanding encryption and methods of attacking encryption.
Everything from plain letter replacement, OTPs, the Enigma, Public keys is discussed, with the hows, the why's and the fallacies.
The reason the Enigma was cracked was because the Germans sent a short code TWICE in the beginning of each message. Oh, and it was the Polish intelligence who found the clue after the French got hold of the manuals and - thinking that it was impossible - passed them on.
Just having a 'code book' wasn't enough to crack the cipher.
It was thought you also had to have the machine itself.
(One of the reasons the Germans added new code wheels to it several times, and why the different branches used slightly different models. )
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Don't visit my new website...
The guys developing the Colossus didn’t know of the Heath Robinson until their success with Colossus. After that, they significantly refined Heath Robinson. Colossus was a computer in terms of Turing’s definition. It could compute and perform both Do-Loops and Conditional loops.
The two programs were completely compartmentalised and Jones (Boss of Enigma), when he wrote his rather poor book in +-1972 did not know that Colossus existed!!! (Enigma was the name of the program as well as the English name used to describe the German Mobile Encryption machine – the name was probably of Polish origin). Heath Robinson (Look at my Avatar/picture on the left It’s my favourite Heath Robinson picture) was the mechanisation of pattern spotting.
Colossus was used on the more heavily encrypted main line information and attempted to tease out numeric sequences and, because the first 50 characters of each message were often similar, they were amazingly successful. Interestingly, even Turing, when he committed suicide appears to have been unaware of Colossus.
The two programs were completely different, Colossus wasn't part of the Bletchley Campus
The guys developing the Colossus didn’t know of the Heath Robinson until their success with Colossus. After that, they significantly refined Heath Robinson. Colossus was a computer in terms of Turing’s definition. It could compute and perform both Do-Loops and Conditional loops.
The two programs were completely compartmentalised and Jones (Boss of Enigma), when he wrote his rather poor book in +-1972 did not know that Colossus existed!!! (Enigma was the name of the program as well as the English name used to describe the German Mobile Encryption machine – the name was probably of Polish origin). Heath Robinson (Look at my Avatar/picture on the left It’s my favourite Heath Robinson picture) was the mechanisation of pattern spotting.
Colossus was used on the more heavily encrypted main line information and attempted to tease out numeric sequences and, because the first 50 characters of each message were often similar, they were amazingly successful. Interestingly, even Turing, when he committed suicide appears to have been unaware of Colossus.
The two programs were completely different, Colossus wasn't part of the Bletchley Campus
Kind regards from Darkest Africa
John Bond
Well, John, of course I have no personal knowledge of these events, and must rely upon published accounts.· The one I rely on most (Budiansky's Battle of Wits) gives a pretty complete account.· I would hesitate to contradict you, but Budiansky's narrative is very different from yours.
I type fast, so here's the appropriate account from Battle of Wits.·Budiansky is discussing the decryption of the "Fish" system, which was a five-bit encrypted radioteletype code:
····What was required, then, was to prepare two long loops of teletype tape.· One would contain the cipher text added to itself with a shift of one; the othe would contain, for every·possible starting position of the Χ wheels, the Χ wheel output added to itself with a shift of one.· The two tapes would then be compared, character by character, and the number of coincidences counted.· The correct Χ wheel starting position ought to result in the greatest number of hits between the two tapes. ···· The first machine built to do this was called Heath Robinson (the name was the English equivalent of "Rube Goldberg"), which arrived at the "Newmanry" in May 1943.· [noparse][[/noparse]The Newmanry was a section at Bletchley Park headed by Max Newman.· It was a short distance from Hut 8, of which Turing was the head -- Carl]· Jack Good, a statistician who formed part of Newman's formidable team of mathematicians, said that one of the greatest secret inventions of the war was the discovery that ordinary teletype tape could be run at thirty miles per hour without tearing.· It did, however, tend to stretch -- especially the Χ wheel tape that was used over and over.· That was of course fatal, because absolute synchronization between the two tapes was essential for a correct count.· Turing served as a more or less informal consultant to the Fish project, but he suggested that Newman summon Tommy Flowers, an electronics specialist from the Post Office Research Station.· Flowers, before the war, had designed and built experimental telephone switches that used vacuum tubes in place of any moving parts, and he at once suggested that the solution was to design circuits that would generate the Χ wheel patterns electronically.· The resulting Colossus Mark I was completed and installed at the Newmanry in February 1944; it contained fifteen hundred vacuum tubes (later models had twenty-five hundred) and proved far more reliable than Robinson, which, Good said, did have the one redeeming feature that it was usually possible to diagnose a fault by the type of noise it made -- and sometimes by the smell it made, as there was one particular recurring problem that caused the machine to overheat and attempt to catch fire.· The Colossus was the first computing device with a substantial electronic memory and it was programmable by switches and patch cords; it was even capable of some conditional logic, adjusting a calculation according to data accumulated in the course of a run.· By the end of the war ten Colossi were in service.· Like the American RAM equipment, the Colossus contained many innovative elements of the modern digital computer without itself being in the direct line of descent that led to the computer; other wartime machines, especially the American ENIAC,which had punch-card input and output, circuits to perform arithmetical calculations and square roots, and some ability to store programs electronically, have a far better claim to having given birth to the computer age. ···· But there is no doubting that the Colossus spawned an intelligence bonanza.· The German radioteletype circuits connected Berlin directly to the headquarters of theater commanders and army groups, and revealed German intentions and strategy at the highest levels.
So of course it's possible that Turing knew nothing of Colossus, but it's pretty clear that the Robinson and Colossus were related and were installed at the Newmanry in Bletchley Park.· All this is true if we accept Budiansky over your sources.· How about sharing your sources, so that I can seek a copy of my own?· I find these things very interesting.
Comments
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
Thanks to Kwinn for giving me a new word.· Very few people can do that.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
basics of jpg:
1) take a group of pixels
2) do fast forie transform to convert to frequency domain
3) multiply f domain value by matrix that squashes resolution in frequencies human eye does not see well but keeps resolution in ranges that are seen well.
4) use hufman compression algorithm to make values that show up more often have less bits then those that show up not as often.
because of the last step you can't just change the lsb or you would be picking a completely different color. but if you sellectively chose a color from the list that was similar that had a lsb that you wanted you could do it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
And I don't recall encountering the word "steganography".· I don't forget words.
Googling that word, I find I have missed a lot.· There's an excellent Wikipedia article on steganography,
http://en.wikipedia.org/wiki/Steganography
and other stuff to look at.· So why didn't I know that word?·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/6/2009 11:36:35 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
The fact that I can't tell is a sufficient problem in itself.· I wouldn't trust it for cryptological purposes.
I did a lot of testing on strings I thought were random.· In addition to standard things, like character frequencies, digraph frequencies, and so forth, I converted them to WAV files and listened to them.· I've found nonrandomness (detectable tones and repetitions) in stuff that seemed really random when tested any other way.· So I'm very distrustful.
Maybe I ought to make bitmaps out of them too, and look for visual stuff.· Hmmm --
And even then I can't hope to be as good as NSA at finding subtle departures from randomness.
So, every bit in one of my random strings is an XOR of several hundred individual bits (that "several hundred" is also as random as I can make it, with a minimum and a maximum) derived from measurement of thermal noise.· That's slow, but it may be good enough.· I hope.· But I can't know.
I also have stuff in there that attempts to make truly random the time between samplings of the random noise.· As I say, it's very slow, but I can set a PC doing that and ignore it for weeks at a time while it makes keyfiles.· I have lots of extra PCs, so it doesn't matter.
But I still can't really know.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/7/2009 4:41:54 AM GMT
-Phil
Their decryption jobs were made much easier because the disciplined German communications staff always started their messages in exactly the same way. The Germans also believed that the code was impossible to crack, this despite clear evidence that the British had done so.
Encryption can always be broken. Your NIA/CIA do it regularly and often with evil intent. A personal example - the CIA decoded messages from President Samora Machelle of Mozambique and conveyed the information to the Aparthate Government with the suggestion that the South Africans shoot down his plane. No one denies that Machelle was a committed socialist but everyone is equally convinced of his greatness. Note how his wife, now married to Mandela, has carried on his great and noble work. His sins, a refusal to give up his socialist dream and a refusal to allow CIA to operate from Mozambique.
When encryption is weak, fear not those criminals and thieves. Fear rather amoral organs of state who justify their actions with slogans like “defending the nation” and "National Security".
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Suppose, for example, that a sequence is composed of eight-bit bytes, and they are all truly·random except that every nth byte, where n·is some·prime, is influenced by the (truly random) byte that precedes (or perhaps follows) it by m bytes, where m is some other prime?· Such a structure could be used to convey information in a signal that would pass every test for randomness (except perhaps NSA's tests, who knows?); and a one-time-pad system that relied on the randomness of such a scheme might be breakable because it isn't really random.
That kind of nonrandomness is a bear to test for.· Not impossible, but very expensive in processor time.· Notice that you've got to test separately for each pair of primes n and m, and for each n you've got to test the whole file·n*m times to detect it.· That's tough to do.
My OTP system doesn't use any such structure (which would multiply the original message size by n, very inefficient), but it does rely on the true randomness of the keyfile.· Randomness is tough to test for.· Code-breaking specialists have to be inhumanly good at testing for prime-related nonrandomness -- and therefore I assume that they are inhumanly good at it.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
One writer has remarked that among the greatest breakthroughs at Bletchley Park was the discovery that punched paper tape could be run at 30 mph without breaking.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/7/2009 3:34:54 PM GMT
-Phil
The physicist R V Jones at the University of Lancaster worked on seismic detectors.· At one point he was puzzled by a cyclic effect in these.· He·wrote, Had I not been there at the time I would have been very puzzled by a rhythmic tilt of period about one minute, which at first looked like an unusual effect of a distant earthquake.· The building was gently rocking to and fro with an amplitude of a few parts in a hundred million, owing to the shift of weight as a conscientious cleaner moved herself and her floor polisher backwards and forwards progressively along the building.
In 1874, in Life on the Mississippi, Mark twain wrote, There is something fascinating about science.· One gets such wholesale returns of conjecture out of such a trifling investment of facts.
Perfection is difficult to achieve.·
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
-Phil
You're right about detection bias and cyclicity, of course. I guess it's just a question of the timescale in which these measurement defects occur, and whether the defects can be filtered out by combining measurements that span a longer timescale.
-Phil
Where's Rich? He started this whole discussion with an extraordinary claim. We've yet to see the requisite extraordinary proof — or even evidence — to back it up.
Post Edited (Phil Pilgrim (PhiPi)) : 4/7/2009 5:06:19 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
The idea of counting fission events to generate random numbers in an interesting one.· I'm not quite sure I know how to do it, although I have experience in processing information from similarly random "shot" events.· I have a patent (expired) on a system that counted and processed pulses from a blood counter.· A blood sample diluted in saline was passed through a very small orifice.· When an erythrocyte (red cell) passed through the orifice (tiny hole in a thin glass membrane), the electrical resistance of the hole spiked.· We counted spikes to count red cells.· These were the same kind of random events one gets from atomic decay.· Sometimes the orifice gets blocked and you get a bad count.· We ran three orifices in parallel to get three counts.· My patented device took the three counts, computed the expected variation among them, compared them, and either calculated a best-estimate answer or threw away the result.· It did this as follows:· if all three counts are within one standard deviation (sigma)·of each other, take the mean of all three.· If two agree within one sigma but the third is goofy, take the mean of the two that agree.· If no count is within sigma of any other, turn on a red light and a buzzer and produce no result at all.
The statistics of random events like those are interesting.· In any time interval, one will get some discrete count.· How many is a measure of the overall rate (which in our case indicated the patient's red count).· The longer you count, the better your estimate of the average.· The standard deviation is (sqrt(n-1))/n, so it's easy to calculate the probable error in your result.
But how do you generate an unbiased, flatly distributed random number from counting such events?· It's probably simple, but I'm not statistician enough to know how to do it.· Will the time between events be flatly distributed, for example?· I have no idea.
Back to the important stuff:· where's Rich?
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/7/2009 7:41:26 PM GMT
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
····f(k) = pke-p/k!, where
k is the number of events, and p is a scaling factor that depends on the process at hand and the length of the time interval and is usually determined by curve fitting observed data.
You can convert any random variable to a uniform random variate on the interval [noparse][[/noparse]0, 1] by inverting its distribution function (the integral of the density function over the random variable, from 0 to k). This process is more precise with continuous random variables (such as the time interval between decays) than with discrete random variables (such as the count per given time interval), especially where sparse events are being recorded.
-Phil
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Need to make your prop design easier or secure? Get a PropMod has crystal, eeprom, and programing header in a 40 pin dip 0.7" pitch module with uSD reader, and RTC options.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
-Phil
Double-Edged Secrets· W. J. Holmes.· United States Naval Institute, 1979.· My copy has no ISBN.· This is an account of USN code-breaking in the pacific, by a major participant.· Interesting to the historian but less so to the cryptologist.· I'm a history buff, too.
Code Breaking· Rudolf Kippenhahn.· The Overlook Press, 1999.· ISBN 0-87951-919-3 (hardcover); ISBN 1-58567-089-8 (paperback).· Some history, some technical explanation.··Information about pre-computer-age codes and ciphers, but also an excellent discussion of RSA and similar methods based upon large primes.· Includes public key encryption, digital signatures, etc.·
Cryptanalysis· H. F. Gaines.· American Photographic Publishing Company, 1939.· Dover Publications, N.Y. 1956.· No ISBN.· Original title Elementary Cryptanalysis.· From the 1939 date you will guess there's nothing modern here.· Useful background knowledge abounds, though.· Helen Gaines was an early practitioner in U. S. government service.
Algebraic Coding Theory· Elwyn Berlekamp.· McGraw-Hill, 1968.· Library of Congress number 68-17175.· No ISBN that I can find.· Not a cryptographic text at all, this is concerned with error-correcting codes and the like, and is intended as a graduate-level textbook.· Much useful mathematical treatment of codes in general, especially their information content (useful to know for cryptographic codes too) and error-correcting methods.· Possibly out of date, and the mathematics is now beyond me (once, it wasn't).
Codes, Ciphers and Computers· Bruce Bosworth.· Hayden, 1982.· ISBN 0-8104-5149-2.· Includes treatment of RSA, DES and public-key systems.
Battle of Wits· Stephen Budiansky.· Simon & Schuster, 2000.· ISBN 0-684-85932-7 (paperback 0-7342-1734-9).· Fascinating history of code-breaking in WW2.· If you really want to understand how it was done and who did it, this incomparable book is the first to seek.· I should have bought three copies; mine is wearing out, even though not cheaply bound.· Yours will, too.· Besides the historical interest, there is much technical information here also.· Get this book first and it will assure your continued interest in the field, while also providing a rich fund of background knowledge.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/7/2009 9:08:57 PM GMT
http://www.amazon.com/Code-Book-Science-Secrecy-Cryptography/dp/0385495323/ref=sr_1_1?ie=UTF8&s=books&qid=1239179537&sr=8-1
This book is absolutely a must in understanding encryption and methods of attacking encryption.
Everything from plain letter replacement, OTPs, the Enigma, Public keys is discussed, with the hows, the why's and the fallacies.
The reason the Enigma was cracked was because the Germans sent a short code TWICE in the beginning of each message. Oh, and it was the Polish intelligence who found the clue after the French got hold of the manuals and - thinking that it was impossible - passed them on.
Just having a 'code book' wasn't enough to crack the cipher.
It was thought you also had to have the machine itself.
(One of the reasons the Germans added new code wheels to it several times, and why the different branches used slightly different models. )
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
Don't visit my new website...
Hi Carl
The guys developing the Colossus didn’t know of the Heath Robinson until their success with Colossus. After that, they significantly refined Heath Robinson. Colossus was a computer in terms of Turing’s definition. It could compute and perform both Do-Loops and Conditional loops.
The two programs were completely compartmentalised and Jones (Boss of Enigma), when he wrote his rather poor book in +-1972 did not know that Colossus existed!!! (Enigma was the name of the program as well as the English name used to describe the German Mobile Encryption machine – the name was probably of Polish origin). Heath Robinson (Look at my Avatar/picture on the left It’s my favourite Heath Robinson picture) was the mechanisation of pattern spotting.
Colossus was used on the more heavily encrypted main line information and attempted to tease out numeric sequences and, because the first 50 characters of each message were often similar, they were amazingly successful. Interestingly, even Turing, when he committed suicide appears to have been unaware of Colossus.
The two programs were completely different, Colossus wasn't part of the Bletchley Campus
Kind regards from Darkest Africa
John Bond
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
I type fast, so here's the appropriate account from Battle of Wits.· Budiansky is discussing the decryption of the "Fish" system, which was a five-bit encrypted radioteletype code:
···· What was required, then, was to prepare two long loops of teletype tape.· One would contain the cipher text added to itself with a shift of one; the othe would contain, for every·possible starting position of the Χ wheels, the Χ wheel output added to itself with a shift of one.· The two tapes would then be compared, character by character, and the number of coincidences counted.· The correct Χ wheel starting position ought to result in the greatest number of hits between the two tapes.
···· The first machine built to do this was called Heath Robinson (the name was the English equivalent of "Rube Goldberg"), which arrived at the "Newmanry" in May 1943.· [noparse][[/noparse]The Newmanry was a section at Bletchley Park headed by Max Newman.· It was a short distance from Hut 8, of which Turing was the head -- Carl]· Jack Good, a statistician who formed part of Newman's formidable team of mathematicians, said that one of the greatest secret inventions of the war was the discovery that ordinary teletype tape could be run at thirty miles per hour without tearing.· It did, however, tend to stretch -- especially the Χ wheel tape that was used over and over.· That was of course fatal, because absolute synchronization between the two tapes was essential for a correct count.· Turing served as a more or less informal consultant to the Fish project, but he suggested that Newman summon Tommy Flowers, an electronics specialist from the Post Office Research Station.· Flowers, before the war, had designed and built experimental telephone switches that used vacuum tubes in place of any moving parts, and he at once suggested that the solution was to design circuits that would generate the Χ wheel patterns electronically.· The resulting Colossus Mark I was completed and installed at the Newmanry in February 1944; it contained fifteen hundred vacuum tubes (later models had twenty-five hundred) and proved far more reliable than Robinson, which, Good said, did have the one redeeming feature that it was usually possible to diagnose a fault by the type of noise it made -- and sometimes by the smell it made, as there was one particular recurring problem that caused the machine to overheat and attempt to catch fire.· The Colossus was the first computing device with a substantial electronic memory and it was programmable by switches and patch cords; it was even capable of some conditional logic, adjusting a calculation according to data accumulated in the course of a run.· By the end of the war ten Colossi were in service.· Like the American RAM equipment, the Colossus contained many innovative elements of the modern digital computer without itself being in the direct line of descent that led to the computer; other wartime machines, especially the American ENIAC,which had punch-card input and output, circuits to perform arithmetical calculations and square roots, and some ability to store programs electronically, have a far better claim to having given birth to the computer age.
···· But there is no doubting that the Colossus spawned an intelligence bonanza.· The German radioteletype circuits connected Berlin directly to the headquarters of theater commanders and army groups, and revealed German intentions and strategy at the highest levels.
So of course it's possible that Turing knew nothing of Colossus, but it's pretty clear that the Robinson and Colossus were related and were installed at the Newmanry in Bletchley Park.· All this is true if we accept Budiansky over your sources.· How about sharing your sources, so that I can seek a copy of my own?· I find these things very interesting.
▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔▔
· -- Carl, nn5i@arrl.net
Post Edited (Carl Hayes) : 4/8/2009 8:06:26 PM GMT