The Cycle of Gaining and Losing Knowledge

“There are known knowns; there are things we know we know. We also know there are known unknowns; that is to say we know there are some things we do not know. But there are also unknown unknowns – there are things we do not know we don’t know.” — Donald Rumsfeld

In a press briefing a little over 10 years go, Rumsfeld popularized the concept of known knowns, known unknowns, and unknown unknowns. I hope my figure below communicates the problem with “unknown unknowns.”

In general you want the small green slice of “known knowns” to be as big as possible; this represents your knowledge of the universe. For example,

I know that I know how to program in the Ruby programming language.

You also want the blue slice of “known unknowns” to be as big as possible.  Once you are aware of ignorance about a particular thing, you can then decide to educate yourself (if the benefit of learning outweighs the cost of learning).

I didn’t always know how to program in Ruby, but I knew Ruby existed and had an idea of its value proposition.

Moving from “known unknowns” to “known knowns” is therefore relatively straightforward. However, that big red slice of “unknown unknowns” is really scary. You have no idea what’s in there and you have no idea how big it is.  You can’t really decide on your own to move from “unknown unknown” into a “known unknown,” because, well — you don’t even know what knowledge is in that category.

The best you can do with “unknown unknowns” is be aware that the category exists and maintain an open mind. This way when information presents itself to you, you can cognizantly realize that it was an “unknown unknown” and then you’ll either be in the “known knowns” or “known unknowns” category.

I didn’t always know that Ruby existed. One day someone told me about it, and I became aware of its value proposition.

These transitions of knowledge are presented in the figure below.

I’ve already explained the blue and green arrows, but what the hell is that red arrow?  The point of this blog post to describe the arrow  from “known knowns” to “unknown unknowns;” the transition from knowledge to ignorance.

Becoming ignorant again

It’s often a long road becoming an expert on a topic.

I have been programming for 17 years.  I’ve programmed in Ruby for severals years and I know Ruby pretty well now.

Becoming an expert was hard. I remember what it was like to be a beginner — the perplexing error messages and the mystery of source code.

But today some information came my way that made me realize I don’t actually know what it’s like to be a beginner. I thought I knew what it was like; in actuality though, I didn’t know that I didn’t know what it was like to be a beginner.

I was reading an article in Slate about a journalist’s efforts to teach herself programming in Ruby.  This is the line that shocked me out of the “unknown unknown” category.

“I started off by saving [my first Ruby program] as a Word document” — 

I never would have imagined someone would try that. From 17 years of experience, I know that you need to save your programs in text file format, but I had no idea that a beginner wouldn’t know that.

Even worse than being ignorant, I had been deceiving myself into thinking that I remembered what it was like to be a beginner.

I do remember bits and pieces of learning to program, which fooled me into thinking I remembered what it was like to be a beginner. For instance I remember that when I first started programming, the error messages overwhelmed me.  Perhaps the only reason these memories stand out is because sometimes they still overwhelm me. Maybe I forgot about the text-file issue because it was one particular issue I learned 17 years ago; and once I learned it, I moved on and never thought about it again.

I can’t relate to what its like to be a beginner programmer.  But at least now, I know that I don’t know.

What the flock?

Today, through the acquisition of my company Dasient, I have joined the flock at Twitter (they love bird metaphors). I am a proud new member of Twitter’s revenue-security team, and I’m excited!

Until I heard about Twitter’s interest in acquiring Dasient, I was only a light Twitter user. I didn’t really get it. And when I talk to some of my friends about Twitter, they don’t get it either. Let me share what I learned about Twitter that enlightened me.

Twitter helps me find and follow information I can’t get any other way

Just by looking at my Tweets, you can’t tell I’m a power user, because I don’t Tweet much. But I am a power user because I use Twitter to access information I can’t get any other way. Here’s two personal examples of how Twitter helped me within the past couple of weeks:

(1) ATM outage

Last week at Bank of America every ATM was out of order and the bank lobby was closed. Strange.

In the days before smartphones I would normally drive around town looking for another ATM. But now I have a smartphone and understand the value of Twitter. Searching for “bank of america outage” on Twitter revealed 4 tweets from the past hour reporting Bank of America ATM outages all across the United States.

Bank of America outage on Twitter search

No need to look for another ATM; the entire system was down.

Now that I get Twitter, I know how to find that kind of real-time information. But what would I have done if I hadn’t been a Twitter user? When I got back to the office, I did a little experiment. I tried looking up the ATM outage on Google. I searched the news, blogs, BofA’s own site, and the Web at large—and found nothing.

Twitter was the only medium that had the real-time information I needed.

(2) Algorithmic-complexity attacks in the wild

The primary topic of my graduate research at Harvard and MIT Lincoln Laboratory was algorithmic-complexity attacks (AC attacks), a particular type of denial-of-service (DoS) attack, which I have blogged about before.

While surveying the literature, I searched far and wide for public discussions of these attacks occurring in the wild. I found nothing. I felt sure they actually happened in the wild, because of their appealing features from an attacker’s perspective. I just couldn’t find any discussions. In my research paper describing my novel defense against such attacks, I couldn’t mention the occurrence of any real-world AC attacks in the wild because I was unaware of any.

My negative experiences researching AC attacks without Twitter sharply contrasts my recent positive experiences researching AC attacks with Twitter. Specifically, several AC vulnerabilities were recently announced on the web. These particular vulnerabilities represent a subgenre of AC vulnerabilities, known as HashDoS vulnerabilities. Discussions of the vulnerabilities emerged on Twitter using the #hashDoS hashtag. There is even a @HashDoS account on Twitter, which moderates and promotes such discussions.

Within minutes of beginning my search on Twitter, I found the gem I’d been searching for for years: a report of an AC attack against StackOverflow.com by an employee.

From Geoff Dalagas, @SuperDalagas. #StackOverflow just saw somebody attempt to use the #hashdos 42 mins ago - patch your servers folks (yes, we have) bit.ly/tKmTFO

Keep in mind: I am an expert on AC attacks, I am a Google power user, and I am a newcomer to Twitter.

To top it all off, now that I know who the other researchers are in this area, I don’t have to search anymore. I just follow them and their insights get pushed to me automatically.

Flock

I think most people are like me; they consume information more than they produce it. Subsequently, thinking of Twitter as just a microblogging platform grossly understates its value. If you are not already, become a Twitter user and have the world’s real-time status at your fingertips.  Join the flock and follow me @MichaelNGagnon.

Spontaneous radial symmetry

Minutes into January 1, 2009 my wife’s family and I spontaneously jumped together for a group photo.  Several of us threw our hands in the air independently.

Amazingly, our arms made exact 45-degree angles.

Original photo

As soon as I saw the photo I immediately announced it was going to be the best picture of 2009. I still stand by that statement. I also knew that some image manipulation with gimp would transform this photograph into fine art.

Spontaneous Radial Symmetry

I hope it doesn’t scare you.