A Riddle wrapped in a Mystery inside an Enigma…
“I cannot forecast to you the action of Russia. It is a riddle wrapped in a mystery inside an enigma.” – Winston Churchill (1939)
Perhaps this post will only appeal to byronius who, along with me, is a student of the World Wars of the 20th century. If this is the case, I offer my apologies in advance. But if you like secrets and how to steal them, then by all means read on!…
(BTW: I commented on this in Cat-eyes’ What’s that up in the sky? post, but decided the subject merited its own post.)
But this remarkable story isn’t about weapons or strategy or bloodshed (at least not directly), but about codes and ciphers. From the clever way the British intercepted a German diplomatic cable message (the Zimmerman Telegram) in 1917, decoded it and (realising its importance), funnelled it (wiped discreetly of British fingerprints) to Washington, forcing a reluctant America into WWI… on through the famous (and devilishly clever) German ‘Enigma’ machine and the Japanese ‘Purple’ and JN-25 ciphers.
The urgent (even existential) requirement for the Brits as WWII loomed to break the German Enigma cipher drove the formation of ‘Station X’, better known as ‘Bletchley Park’, tucked away on an estate in the pleasant English countryside of Buckinghamshire… The men and women at Bletchley DID succeed, and brilliantly; at the very least shortening the war by about two years and perhaps even preventing England from losing the war in the dark days when England stood alone against Hitler before America’s entry into the conflict.
Oh and BTW, Alan Turing and Tommy Flowers also designed and built the very first digital computers at Bletchley Park to speed their decoding efforts.
If you like puzzles, Enigma was a real humdinger, and one might forgive the Germans for believing it was ‘unbreakable’ with roughly 3 x 10114 (courtesy NSA) possible combinations (rotors+plugboard+start positions). Factorials get big fast! Enigma was self-decoding. If a cipher output is fed back into the machine (or one set up exactly like it), the original plain text is retrieved. A ‘reflector’ disk essentially reverses the encoding process. Pretty damn slick. Just to make sure, an operator had ‘extra’ pre-wired rotors to choose from (8 was typical later in the war)… so 3 rotors were chosen ‘randomly’ and inserted in ‘random’ order and set to a ‘random’ starting position. A 3-letter ‘random’ initiator sequence started each message. Adm. Karl Dönitz, the Kriegsmarine (German navy) commander requested (and got) a 4-rotor version of Enigma after he started losing more U-boats in the North Atlantic than he thought could be put down to chance. But the method still seemed completely foolproof… and the Germans (to their grief) believed in it utterly.
I emphasised the ‘random’ factor in setting up an Enigma machine for operation. This was Enigma’s weakness that the Bletchley teams recognised and exploited. I could do a whole post on the nature of ‘randomness’ alone, but here it suffices to say that if you use truly ‘random’ choices for setting the machine, it IS pretty much ‘unbreakable’. However, human beings are NOT capable of producing random sequences. Neither are computers. Such numbers are called ‘pseudo’-random for a REASON. Truly random sequences are surprisingly difficult to produce. If you need truly random numbers, monitoring radioactive decay is the easiest way to go.
Any cipher system also suffers from repetition in messages. The Germans blew it here as well. Overconfidence in Enigma (and a lack of understanding in coding basics) led many Enigma operators to end messages with flourishes such as ‘Down with England’ or ‘Heil Hitler’, and initiator sequences were often a girlfriend’s initials or similar repeated sequences. Major error. Failure to ‘randomise’ the rotor starting positions properly also aided the code breakers.
Fortunately for us, these operator blunders allowed Bletchley Park to detect PATTERNS. And even better, German arrogance (and British sneakiness) kept them from EVER snapping to the fact that their cipher system had been fatally compromised.
A 3-rotor Enigma machine in operation (1943)
The surviving German high-command were shocked white and speechless when informed after the war that Bletchley Park had been reading their most intimate ‘mail’ through almost the whole war!
I can’t leave this subject without a nod (and a wink) to my Dad (USN Lt. Roy Howard [code name: 'Hot Pants' ? ;-}]), who spent a good part of his naval career working on just these sorts of problems in the late 1940's-early 50's. As far as I know (not too far I'll admit), neither US diplomatic nor military operational ciphers have EVER been broken in the modern era. Our guys are pretty damn good. Dad could probably reveal further details, but of course then he'd have to kill me... 
However, this should remind us that NSA certainly has capabilities (some of which Cat-eyes wrote about some time back in Big Brother is here) that most American citizens would (and should!) be pretty uncomfortable about... There are two sides to every coin... Just sayin'.
Finally, if you'd like to play with a virtual Enigma machine, here's a fun Enigma simulator.
“Hot Pants”?
[chortle] KNEW that would get your attention, Pop!
Comment on April 22, 2012 @ 5:06 am
My favorite WWII story, period.
BTW, who picked Turing out of the masses and set him up as the Main Bletchleyan?
The Bulldog, of course. Always the Bulldog.
Comment on April 22, 2012 @ 11:48 am
byronius: Thanks for scanning my post. Weird about Turing, isn’t it?
The ‘Colossus Mk II’ (“the world’s first digital computer”) is often credited to Turing. In fact, that’s what *I* thought until I looked into it more deeply. ‘Colossus’ was actually designed and built by the brilliant electrical engineer Tommy Flowers.
Wikipedia: Colossus
Turing’s own machine, called the ‘Bombe’, was a sort of computer, but really was an Enigma rotor simulator, designed to test a lot of possible rotor combinations simultaneously.
Wikipedia: The ‘Bombe’
Alan Turing was well known as a brilliant eccentric. In reality, he probably ‘suffered’ from Asperger syndrome, a ‘high functioning’ type of autism. Isaac Newton is also suspected to have had it, and I would add Wolfgang Mozart (and several others) to that list. Characterised by poor interpersonal skills, but also capable of intense concentration on very complicated matters… and understanding them.
Famously, Turing was also homosexual… at a time when such ‘deviant’ behaviour was illegal in Britain (and the US).
If Churchill chose to highlight Turing’s contribution to the Bletchley Park ‘ULTRA’ effort (I didn’t know of this), he surely didn’t do him any favours!
In 1952, during a routine interview following a home burglary, Turing let slip that he had a male lover. Due (probably) to Asperger’s, Alan Turing was incapable of guile. And given the sexual attitudes of the day, he was arrested and charged with ‘Gross Indecency’… and was convicted.
Given the choice of prison or a course of female hormones (chemical castration), Alan Turing chose the hormones and (predictably) fell further and further into depression as the side-effects added to his public ‘dishonour’. In 1954, Turing laced an apple with cyanide and ate the ‘poisoned apple’… dying in his bed. He was just 41 years of age. (the obvious ‘Snow White’ parallel to his chosen method of suicide has always disturbed and intrigued me)
So added attention to himself was NOT a good thing for Alan M. Turing.
But he left us these two key papers, which became the foundations of both Computer Science and Artificial Intelligence:
‘On Computable Numbers’ [PDF] 1936
and
‘Computing Machinery and Intelligence’ 1950
Comment on April 22, 2012 @ 4:00 pm
@ Sky> I was wondering if you knew whether collections of humans could simulate true randomness – if each believed they were acting independently and were selected on a purely temporal basis? In other words, how much of our inability to simulate true randomness is hard wired into every human brain so that even independently tossed into a pot and stirred our choices will leave a distinctly human signature. And how much of this is cultural -so isolating cultures aside – tossing in all humans of all cultures and all ages – could we find the “human” pattern. Think sci-fi intergalactic coding. Could we find each other that way???? Hope this makes sense to you. I tend to think definitely so. We are after all limited beings. If we found it would it be the program that runs us on a simulator??? For an interesting but easy read that might yield a few unknown anecdotes I recommend reading “CODE: The evolution of secrecy from Mary, Queen of Scots to quantum cryptography” by Simon Singh.
@ Byron > too bad the bulldog – so very much like real bulldogs – was so OCD on his own end goals that he didn’t scare off the witch hunt that made Alan commit suicide.
I always liked codes since reading about them in a book in the fifth grade, but not so much anymore – it seems it has become so much computing power for number crunching and not much else these days. Effective, but boring.
Comment on April 22, 2012 @ 4:26 pm
@Cat-eyes: An interesting query… the short answer is I don’t know. Don’t know if it’s ever been tried! It’s actually a pretty deep question, both theoretically and philosophically… And the more I think about it, the more unsure I get!
The concept of our ‘reality’ being merely a simulation being run on an intergalactic super-computer (a program by some genius-ET undergrad for her(?) semester final-exam?) is uneasy making to say the least!
What if you could monitor a group of ‘unrelated’ (another pitfall!) computers and at intervals sample the last key typed?.
What about the SETI folks (and SETI@Home)? They’re trying to detect a NON-random signal. How do they tell? Will their software know it when they see it?
I guess the key question is ‘what makes a given sequence random’? The corollary being ‘what is deterministic’?
I worked for a couple of computer security outfits for a while, and would occasionally run an FFT on data segments to get a frequency analysis… thinking (probably naively) that a 1/f ‘white noise’ spectrum would be a pretty good indicator of randomness. I could tell if a sequence was definitely NON-random, but it was difficult/impossible to measure degrees of randomness. Is it??
A Number/Information Theoretic problem it seems to me. And I’m sure this very case has been tackled more than a couple of times! But now you’ve got my head buzzing on it… I need more research!!
Hmmm… maybe it’s a thermodynamics question. LESS entropy means more order. MORE entropy means more DIS-order. Does ‘truly’ random = ‘maximum entropy’???
Your question is a GREAT one the more I think about it! I need a copy of Singh’s ‘Code’. I’ve seen/heard several favourable references to it.
Cat-eyes fries Sky’s brain once AGAIN!! Way to go!
Comment on April 22, 2012 @ 5:14 pm
A fascinating (and often entertaining) ‘Bletchley Park Tour’ with lots of info on what went on there during the War. (9 part playlist – should run automatically)
Comment on April 22, 2012 @ 6:13 pm
@Cat-eyes> Check this out: Random.org
Comment on April 22, 2012 @ 6:22 pm
Yet another flaw in the Enigma design was the fact that no letter could EVER be the same as the plain-text. So if the message contained a ‘C’ for instance, Enigma would NEVER send a ‘C’ in cipher for that letter…
The Germans considered this a very clever ‘feature’ of the Enigma when in FACT it was a major BUG!
But Hitler had long since chased all the good Jewish mathematicians out of Germany… Any good statistician would have quickly caught this error.
Once again, pride goeth…
I haven’t mentioned the ‘Lorenz machine’ which Hitler used personally to communicate his increasingly insane orders of battle … sort of an enhanced ‘Super-Enigma’ with 12 (count ‘em!) rotors and other clever enhancements. But again, due to experience with Enigma and Lorenz operator errors (massive ones, like sending a lengthy message twice with the same settings!) – Lorenz was pretty quickly cracked with the help of Flowers’ ‘Colossus’ computer.
With these videos, there is a good deal of duplication or overlap of information, and a couple of discrepancies, but enough is unique in each (IMHO) to warrant a viewing.
Comment on April 23, 2012 @ 4:47 am
re: #6 – thanks Sky. Didn’t mean to fry your brain, was looking for more of a slow sizzle.
Comment on April 23, 2012 @ 6:03 am
slow sizzle achieved… please pass the barbeque sauce!
Comment on April 23, 2012 @ 6:10 am
and here’s you reward: check out these vids (granted they are aimed at a younger audience) they are very fun.
http://www.youtube.com/user/Vihart?feature=watch
Comment on April 23, 2012 @ 6:34 am
Thank you. Like the Fibonacci stuff. And reciprocally:
Music by Philip Glass. Really nice video work.
Comment on April 23, 2012 @ 2:29 pm
@ sky and all the Neal Stephenson fans: see this
http://itre.cis.upenn.edu/~myl/languagelog/archives/000313.html
Comment on April 25, 2012 @ 6:30 pm
That was cool, Cat.
Julian Jaynes really did give me new perspectives, freaked me out. And now that that fellow mentions it, I see it in Stephenson’s books.
No wonder I love him so much.
Now I have to figure out this ‘Banbarismus’ stuff. WTF?
I’m not going to live long enough or be half as smart as I need to be, am I.
Pbbbhhht. Primate. Just an early-issue primate.
I need a good alien learnin’ machine. Without the anal probe, please.
I have just been informed by my highly intelligent offspring that I know nothing of military history. His contention is that modern war is much more destructive and painful than Old Wars, like One and Two. Those wars were nothing compared to Vietnam and Iraq, he insists, and I’m just ignorant about all this, and those thousands of volumes I absorbed on the subject are meaningless tripe.
Now, that’s gotta hurt. But it often does when the young buck tries out the new horns, am I right?
Them things is sharp. And maybe he’s on to something.
I hear that soil a’callin me home. Sigh. Fertilizer. That’s me.
Flowers a’sproutin’ in my loam. I’m gonna go do something to hurry up the process. Something that should be legal.
“Ignorant.”
Gearifjsadgh. I must take comfort in the fact that I ALONE CAN BRUSH THE WHITE TIGER.
There is that.
Comment on April 25, 2012 @ 11:52 pm
Très boss Cat-eyes & byronius!
Briefly, before I am forced to admit my abysmal ignorance of ‘Banburismus’ (which I am attempting to rectify), please allow me to address the ‘levels of awfulness’ disagreement between byronius and his son:
I don’t know byronius’ son’s name or age, so I will use ‘BB1‘ (byronius ‘prime’) and make the rash assumption of his being his being around 16 years of age. I could use Bayes’ Rule to measure my level of ‘belief’ in this, but I digress…
Ahem. WWI and WWII are ancient history to the youth of today. You might as well be talking about the Battle of Hastings or indeed, the Punic Wars… they are MYTHS of the dim and misty past and have no bearing on the here and now. This, I assume, is BB1‘s general position. It may also be relevant that America came out BOTH WW1 and WW2 smelling of ROSES with our power and influence greatly enhanced. Korea (1950) was a ‘police action’ and doesn’t ‘count’ to today’s kids. The ‘Cold War’ (1946-1989) was ‘cold’ and doesn’t ‘count’ either. Just a half-century long ‘malaise’… irrelevant.
I’m NOT slamming the kids – this youthful lack of perspective is true of all generations. F’rinstance, I was ONLY concerned with Vietnam… since I was eligible for conscription in THAT particular misguided ‘adventure’ which finally killed nearly 60,000 Americans and well over a million Vietnamese. WWII from THAT teen’s perspective seemed like a ‘no-brainer’ ‘good war’ (as opposed to Vietnam) to my younger self… but I’d grown up watching all those heroic (propagandistic) war movies… War stories from the 1970′s on are generally told from a very different (and more cynical/realistic) POV…
The history of American wars in BB1‘s time horizon go back no further than the ancient days of Vietnam (1964) [when dinosaurs roamed free]. Grenada (1983) and Panama (1989) were mere ‘hiccoughs’.
It is probably the wars of recent history: Iraq 1 (1991), Afghanistan (2001) and Iraq 2 (2003) that have the most bearing on BB1‘s perspective on war.
The probable takeaway from the history of all of these conflicts is that we either LOST (Vietnam) or have come to ambiguous outcomes at best, and in NONE of these conflicts does America come out smelling in the least ‘rose-like’.
It is the limited perspective of the young BB1 (and his confreres [I almost typed 'creche-mates'! ;-}]) makes it very difficult to grasp the ENORMITY of the World Wars which indeed shaped and enabled the very world in which he lives.
Since WWII concluded (with a Bang!) several years before my own entrance onto the world stage (read “mewling, crying, pissing, shitting, always hungry pain in the ass” for ‘entrance’)… it assumes a bit of a ‘mythical’ aura even to me.
But there ARE those rare occasions when war is not an ‘option’ (as were/are ALL of the wars BB1 is familiar with), but a NECESSITY… real honest-to-gaia life-or-death stuff. National SURVIVAL. America’s entry into WWI broke the stalemate and shortened that horrible meat-grinder substantially. America’s entry into WWII really DID in all probability ‘save the world’ or at LEAST saved a world which would be recognisable today.
Sit the kid down and show him ‘Tora Tora Tora’ and maybe ‘Patton’, although I bow to your knowledge in the European theatre. And maybe read/watch “All Quiet On The Western Front” to get the gist of the waste and horror of real WAR.
WAR IS HELL.
No matter how you slice it… Period.
Whatever the era or setting… Period.
Bronze sword or Greek ‘fire’ or Panzer or M16 or IED or Predator drone… PERIOD.
Dead is DEAD. Maimed is MAIMED.
Today’s wars are NOT any more horrible than those of the past… Indeed, I could effectively argue just the opposite. Just look at the casualty figures. Today’s wars are ‘small potatoes’ in that regard. WWII cost us almost half a MILLION young American lives (and WE got off LIGHTLY). Perhaps ‘Saving Private Ryan’ and ‘Platoon’ should be added to your father-son viewing list.
War should NEVER be entered into lightly. But there ARE RARE times when we MUST. The very BAD examples since 9/11 should NOT colour your boy’s attitude too much. I understand your kindly paternal frustration/concern.
There are precious few rewards to getting old, but ‘perspective’ is one of them. I would advise your son to attempt to take a ‘longer view’ here.
Ummm, did I say ‘briefly’? Sorry… I got a bit carried away! But I stand by it…
Now, onto ‘Banburismus’… I really had never even heard of the term until I read of it in Cat-eye’s link just this (last?) evening… Showing my ‘depth of knowledge’ (or rather lack of same) on the Bletchley Park saga…
I’ve since learned that:
(1) Turing coined the word after a proximate village to Bletchley, ‘Banbury’. ‘imus’ is from Latin and implies a certain ‘alchemical’ aspect. In other words, ‘magic’.
(2) It was used to help ‘guess’ which ‘cribs’ to test in his ‘Bombe’ computers. It worked.
(3) It was based on ‘Bayes’ Rule’. Updating probabilities as sequential data is updated… Levels of ‘belief’, knowing what you know NOW. Seems like common sense to me and was important to Shannon’s later Information Theory. So it’s not surprising to me. ‘Exact’ answers are rare in statistics. But…
(4) ‘Bayes’ was actively attacked by most ‘serious’ academicians and especially by statisticians. R.A. Fisher was rabid on the topic. He clearly had no grasp of Quantum ideas! Einstein would have no doubt sided with Fisher. They were both WRONG.
(5) Bayesian methods are used extensively in Computer Science. A good guess can be most helpful if you don’t have an exact answer! I’ve used it for ‘fun’ in building classification ‘trees’… (I clearly don’t get out much)
And that’s about IT.
Sadly, the actual Gold/Shadlen paper is not freely available, so I’ll have to make do with the review (pdf) which I have downloaded and am just now scanning. I’ll have to ‘get back to you’ on that…
It’s been a long time since I read my copy of Jaynes’ ‘Bicameral Mind’ (likely lost over the years… need to check storage), but his basic idea made a LOT of sense to me. It wasn’t till I got down in the weeds that a few ‘yes, buts’ and even some doubts started creeping in…
I also love most of Stephenson’s stuff. I need to re-read ‘Cryptonomicon’ (also ‘maybe’ in storage).
The only video I could find on ‘Banburismus’. I like to play lectures in the background while doing other stuff. If something pricks up my ears I can stop and pay attention…
This is surprisingly relevant to our root topic:
We’re getting into some interesting stuff here n’est ce pas?
My compliments! More please.
Comment on April 26, 2012 @ 2:08 am
@byronius: this also my be relevant to war as it’s fought today. When told all at once like this, it makes me really sad and angry at the ‘powers that be’… the lying COWARDS. Just mark it ‘classified’… and meet ‘the girls’ for cocktails at five. [GRRRR]
Sadly, war is ALWAYS a mess… but this stuff is CLEARLY overkill… in the original sense of the word.
Comment on April 26, 2012 @ 3:46 am
Excellent riposte, I’ll have to digest it later.
My son was informed last night that his application for a super-exclusive writing program was accepted, and the stories he submitted have won him the Governor’s Medal as an Outstanding California Arts Scholar.
So he was feeling his oats, and not about to accept any moderating perspectives from his old man.
Really, I found it humorous. I remember myself at his age — the absolute arrogance of a little knowledge — and was pleased that I could serve as the gentle foil.
My old man would have murdered me.
Progress.
Comment on April 26, 2012 @ 10:19 am
Good for him! Please convey my congrats!
Simple. You’re a better man and father than the one you got.
Progress indeed!
Comment on April 26, 2012 @ 10:47 am