DEFCON 18

The session titles are tinged with cloak and dagger, anarchy and freedom of expression: "We don't need no sticking badges: hacking electronic door access controllers", "Your ISP and the Government: best friends for ever", "Practical cell phone spying".

My personal impression of the some-what cult-like DEFCON security conference can be characterized as  smart people instinctively driven to share knowledge and unadulterated research for a greater good. Whether its the protection of civil liberties, revealing stupid security vulnerabilities and flaws of products or unabashedly calling out vendors on incompetent engineering.

A smattering of speaker comments offer's a peak into the topics for this year's conference:

• There is no patch for stupidity 
• 15 year device life-time == long tail for bad decisions 
• Clever does not mean secure 
• What appears secure is not
• Privacy is a subtle thing
• The warm, fleece-y Snuggie of Obscurity
• Software moves power on the grid
• Cute smart meter is cute
• The dumbest lock design ever
• Assumed to be trustworthy - 543 million devices shipped in first half of 2010
• Download games at your own risk
• An attack on any one node of an electric grid could take that entire grid down
• My life as a spyware developer and why I'm probably going to Hell 
• There is no such thing as privacy. It is dead. Get over it.
• Malware scanner's are mostly stupid
• What can you do with Twitter that is utterly evil? Lots and lots of things 
• There are 155, 693 public water systems - serving 286 million American's
• I don't think you need a sophistical exploit, there will always be a certain number of people that will click "yes" no matter what
• Social engineering has a long history and works just fine on the Internet

The Impact of Scale

The topic of scale is best illustrated through an analogy. 

If you live in a mega-city, then city planning and transportation come to mind. With so many people living in one city, responsible governments must deal with public safety and creating livable structures. Individual buildings, transportation routes, water supply systems - are part of the city. Overtime we got clever and started to architect and build vertically to deal with scarce physical space. As the number of cars on the streets zooms upwards, we study traffic patterns, congestion and invest in alternative modes of transportation. All told, all these "parts" of the city are a system.

The dimensions we use to characterize whether a "system" is large includes number of elements (people, hardware, "things"), tasks, relationships, policies, domains of interest, and enforcement points etc.

If we are investing in smart buildings, smart cities, smart transportation, smart grids, healthcare infrastructures etc. we are talking large scale. We are talking about "system of systems" that is complex and in fact ultra-large.

The theme of ultra-large scale IT systems is explored in "Ultra-Large-Scale Systems: The Software Challenge of the Future" here, a report, published in 2006.

Ask the statisticians, and they would agree, big is in: 

• Number of Cell Phones Worldwide Hits 4.6B in 2010
• 4,000: The number of lines of code in MSDOS 1.0 - Microsoft's first operating system
• 50 million: The number of lines of code estimated to be in Microsoft Vista
• Data storage requirements for smart meters will increase at a rate that resembles a natural logarithmic rate
• The 2009 movie Avatar is reported to have taken over one petabyte of local storage for the rendering of the 3D CGI effects
• The US DoD has 3 million desktops, with just one data-center housing 18 terabyte storage

Of course big things are broken down into manageable parts in an attempt to make sense of the pieces. Cars, routes, traffic control, toll plaza's etc. 

However, we are still left with menacing problems:

• Emergent properties; (or in plain English) stuff will happen that we can't cope with because it is brand new and never happened before. (e.g. the impact of mobile phones on driving behavior)
•  We cannot completely define let alone measure the properties of all components a-priori i.e. somethings are just out of our view-lens
•  It is impossible to update all elements of the system (as each changes) without leaving some window of vulnerability or ambiguity 
• There is continuous evolution of our surrounding environment with unknowable outcomes and inconsistent (changing) states (e.g. new devices, new personalities, new enemies)
• There is no clear ownership, possession and boundaries (e.g. cloud computing)

The Internet and the typical IT infrastructure of an enterprise and government agency is very complex. We think about external networks, web sites, data flows, applications and users, insiders, hardware/software, transactions and supply chains. The Internet and everything in it and connected to it, is very much an ecosystem: a dynamic community of interdependent and competing organizations in a complex and changing environments. The elements have an intrinsic adaptive behavior and we can measure the health and sense troublesome indications. 

To deal with that sort of large-scale complexity, rich research is needed (and going on in) to:

• Understand the biological metaphore and it's applicability to tech-centric views
• Use heuristics and learn how to better predict behaviors 
• Learn how to respond to external stimuli with keener intelligence gathering 
• Assure the software and hardware components that are used to build systems 
• Survive hostile intentions and operate in a continuously changing system

Its a Messy World

We live in a world that is rather messy. 


Russell Ackoff defined "a mess" as  "interacting problems or issues that are not easy to appreciate as a whole" (Flood & Carson, 1993).  You are in  mess, if you can't put any structure to the situation.  


Organizations are not closed systems. You can't measure everything because things change so quickly. The Internet is a vexing source of "unknown, unknowns". We depend on software, that if you look closer, is made up of piece parts whose source is ambiguous at best. Its "Office 2.0" for most employees with no definite way for the employer to tell what's good or bad behavior. 


It is safe to say, that when it comes to IT we are dealing with ... an unstructured situation. 


The growth in terms of volume, speed and diversity of data, devices and threats is non-linear. So our thinking about IT Security must also be non-linear. 


For example, the correlation analysis that is used to flag a security incident or track an impending threat is very much divorced of cause-and-effect accuracy. We are left with false emergencies, lots of noise and inaction to priorities. We are still building "naive" applications instead of making them "street smart" and engineered with the knowledge they will be broken into and constantly attacked. Finally there is a tolerance to live with broken links. Yet to make informed security decisions we need to connect the rationale and outcomes of our policy decisions, cultural expectations, counter-intelligence activities and the lessons learnt from incidents. 


We have to shift from securing an environment to surviving an ever changing ecosystem.

A Wicked Web We Weave

How do you get in the middle of plans or actions of a knowledgeable employee who intends to inflict damage to your company? 

It’s very difficult to control the flow of information within today’s work-place. In any normal “business” day there could be foreign national interaction, USB key exchanges, work from home, contractors –all greatly increasing opportunity for espionage and data theft. There is plenty of room for mischief and the amount of harm done by a well-informed saboteur is non-trivial. It is an order of magnitude more damaging than a stranger.

Definition of an insider by CERT: A current or former employee, contractor, or business partner who

• Has or had authorized access to an organization’s network, system, or data and

• Intentionally exceeded or misused that access in a manner that negatively affected the confidentiality, integrity, or availability of the organization’s information or information systems 

Trouble with an "insider" is they have legitimate access. They are working within regulation, aware of policy and  unlikely to break rules. So access control won't work nor will "intrusion detection". There is no intrusion.

Most of the time an employee will be reported for unusual behavior by a co-worker or an audit. There are also personnel screening processes that have to be in place before hiring. Training and awareness helps employee's notice unusual activity.

That's old-school stuff - and it works and is crucial. In addition there are technical approaches that have gained a rightful place in the "must-have" list. Most companies should turn on network and application activity logging, file integrity checks and data loss monitoring. Stitching together the alerts that are generated by "big brother" helps spot things that are "out of the norm".  For example if a file changes, or an email is sent with a sensitive attachment, a user or behavior can be marked as "bad" and declared suitable for further surveillance.

Unfortunately there is still plenty of noise obstructing an accurate and clear reading of what is good, bad or ugly behavior. We hop on and off social networks, plug and unplug cables, head to work late, forget to submit expense reports, make travel arrangements out of policy, skip virus updates and get overly zealous downloading information. The list goes on. Human behavior can seem hopelessly impossible to predict. 

Statistics brings the magic of mathematics to make sense of data and tell us "what's up". It is a science that fills in blanks in our memory, keeps us honest about the present and paints a rough approximation of our future. There is good research literature on the topic of user behavior analysis. We are working to take tried and true algorithms such as root cause analysis, propensity analysis, link analysis and econometric forecasts - and then rigoursly apply them to hard to solve cyber security prolems.

If you are interested in assessing threats across transactions, individuals and groups that are unobservable to the naked human eye, please contact me. While tackling the problem of employee betrayal can be hard, we can turn to “big data” and analytics to help trip up the enemy within. 

At the NIST Cloud Computing Forum it was Baby Steps for Baby Feet, or so it Seems

I attended the Cloud Computing Forum & Workshops hosted by the National Institute of Standards and Technology (NIST). Venue: United States Department of Commerce, Washington DC. NIST’s mission in life is to work collaboratively with the public and private sector on standards and technology. Vivek Kundra, United States Chief Information Officer, was the key note speaker.

The workshop atmosphere (it was really just one long day of presentations and panels) was naturally US government centric. Patrick Gallagher, Director of NIST, peged the event as a launch pad for a “robust dialogue” to define how the US government will procure and use cloud computing.

Here is a snippet of NIST’s widely circulated a definition of Cloud Computing:

“Cloud computing is a model for enabling convenient, on-demand network access to a shared pool of configurable computing resources (e.g. networks, servers, storage, applications, and services) that can be rapidly provisioned and released with minimal management effort or service provider interaction”

As the day unfolded it became clear that NIST is faced with a catch 22. Agencies need standards to move forward. At the same time NIST is shying away from standards lest they get it wrong too soon.

History, markets and economics tells us that buyers and sellers alike benefit when there are global norms that remove ambiguity, build confidence, establish best practices and clarify legal ramifications. It’s like a traffic signs and speed limits.

As the world’s largest purchaser of information technology – with 10,679 IT systems (lots of duplication) the US government spends $76+ billion annual budget. These impressive stats are a small indication of the high stakes that are being cast around. [The government serves on a daily basis over 300 million “customers” (students, employees etc.) and 1.9+ million federal employees.]

Vivek, while a big fan of cloud computing, reiteratted that the initiative that he launched last September 2009 is still mostly about “rethinking investments in IT budgets” through data center consolidation across agencies.  A report on the state of the union was just released here.

There is meaningful progress towards cloud computing within state and federal agencies and some of the  illustrative use-cases can be found on page 12/38 in the report. Here are a few examples:

• The SEC uses Salesforce to shorten the time to handle investor advocacy cases. It took 30 days to close a case. Now it takes less than 7 days.

• For the 2010 count, 2010census.com was launched by US Census. It is very media rich and there was no means to evaluate the traffic volume. The US Census did not want to stand up a litany of servers for the 2010 count. Instead they used Akamai to delivery content “from the edge”.. The census also selected Everbridge used for mass notifications to the 1+ million temporary census employees.

• Recovery.gov moved to the Amazon Cloud. Recovery.gov is a tool that “shines light into Federal spending that must handle million of visitors”. The shift to Amazon EC2 led to a savings of $750,000 – redirected funding to help identify fraud, waste and abuse.

• State of Utah is moving to a “hybrid cloud” (70% complete) going from 1800 physical servers to 400 virtual servers – with a $4 million annual savings out of a $140 million IT budget. The cloud solution includes Salesforce, Google Earth and Wikispaces.

• The city of Los Angeles moved to Gmail from an antiquated communications system. They calculate a savings of $5.5 million in direct costs and an ROI of 20-30 million.

• The US Department of Interior is in mid-stream of a consolidation effort of multiple email systems with 80,000mail-boxes users into a cloud model. It was not unusual to have a single email server to manage only 6 email accounts.

• NASA is re-evaluating a $1.5 billion data center effort in the context of “cloud first” strategy.

A promising initiative was “revealed” so to speak by Katie Lewin, Director of the GSA Cloud Computing PMO. The Federal Risk and Authorization Management Program (FedRamp) is the official name for what Microsoft and Google are involved in to get their cloud “FISMA-compliant”. The program is still in formation phase.

What I gleamed about the spirit and purpose of FedRamp:

• The program lives under the Federal CIO and seeks to ease tough questions of security controls, authorization procedures and continuous monitoring for large outsourced and multi-agency systems. It would work in the context of the NIST Risk Management Framework (NIST 800-37 revision 1).

• Instead of working with each vendor independently to ensure compliance, an agency would depend upon stamps of approval that would be given to a cloud provider.

• An agency would review the authorization given to a vendor and if convinced with safe-guard assurances, they tie the knot with the vendor

• It is unclear what sort of base-lines are going to be published and applicability across international borders and strict (high sensitivity) agency requirements. The agency would still hold the responsibility to determine suitability

• FedRamp will follow-up with participating cloud vendors to ensure “continuous monitoring” and flag any drift from the original certification.

All  told the tone in the hallways of the US federal government is one of optimism about the future of cloud computing. There is a "cloud first" mind-set. It will be a long-haul though. John J. Shea Director in the DoD, Office of the CIO reminded us that cloud is a 2-10 technology. 2 years of hype and 10 years of adoption. Plenty of time to get rid of the "sins of the past".

General Hayden - Words of Wisdom

I was privileged to listen to one of my all-time favorite presentations by retired General Michael Hayden, former director of NSA and CIA. The venue: TTI / Vanguard Conference on CyberInsecurity in Washington DC. He was reflecting on the current "bogey man" known as Cyber security. 

Now to most of us, Cyber space is a Hollywood (sci-fi genre) conjured term used synonymously with the Internet. It's origins are actually from a 1984 novel called Neuromancer.  In the National Security discourse the term cyber-space is the "5th domain" with implications far removed from a Hollywood set. There is complex language of "mutually assured dependence" and Cyber warfare. Why the 5th domain? Well the other four are: land, sea, air and space. General Hayden artfully distringuished cyber-space as the only domain that was created by man, the rest of which was "created by god" - and natural events. Adding: "we did not do such a good job". Simply-put the Internet is inherently untrusted, and was designed for anonimyity and information movement. Not for security.

In the US we are still grappling with the ownership of the "cyber thing" which Hayden points out is very much fractured. There are voices that seek to frame the problem as a matter of commerce to be shepherded along led with the private sector. On the other, cyber is a matter of national security with bills that promote a nation navigating both times of cyber peace .. and war. Clearly its not a trivial topic. The private sector owns a lot of critical infrastructure from our energy supplies, power grids and financial markets. And the networks and systems are not entirely isolated from this thing called cyber-space. For ill-doers this is a prize. 


The Internet was designed and engineered for friend not foe.


The growing voices that suggest cyber space should be regulated and done so by government, immediately starts to run chills down the spine of civil libertarians and privacy mavens. They fear as Hayden put it: that the US government will make the dubious offer that it cannot protect its citizens unless, it is allowed to monitor them on the Internet. A fear reinforced when General Hayden gave a wink to his own prior employers (NSA) "abridged sense of privacy".


His message was one of a sense of urgency and not alarmist. He encouraged industry to spend energy into thinking about security and privacy and not just the ease of use of the Internet. And the need for doctrine about what can and should not happen in cyber space. (think about the lack of norms against a a cyber attack on a hospital's patient management system).

As John Negroponte put it (also a presenter) in cyber space "proceed with caution"

In Cyber Space, It Pays ... To Pay Attention

The ability to imagine the future and do so rather inaccurately is a uniquely human quality. Lest we forget the faulty real-estate asset valuations and risky gambles some financial services firms undertook. That inability to "get it right" in the midst of plenty of relevant information saw us enter into one of the worst economic downturn. 

Now, while we do imagine creatively (flying machines, submarines, Internet, smart-phones and electric cars to name a few) it is still difficult for us humans to get the future right, because of some familiar limitations. First we are locked in the present as we try and predict the future, or in other words the future almost always looks like a different version of the present, at least for most of us. And second we are very subjective in our forecasts. We can get stuck into believing that our own point of view reigns supreme, and that when we evaluate our claims against those of others – we will doubt those of others. 

Let’s say you regularly drive your car down a route home-bound and we'd like to evaluate your driving behavior. For this test we've created 3 driver proficiency categories. You are either someone that drives in “auto-pilot”, you are a directionally challenged driver (and a lost cause) or you are an individual that is extremely in-tune with your surroundings. 

If you find yourselves in the last category – you are a near perfect driver.  You know the distance of your car to the next. You observe the erratic behavior of a truck 20 feet ahead and two lanes across. You are tracking the changing weather conditions. You are aware. Actually there is a term you don’t fall into: Driving Without awareness (DWA): someone in a state where there is no active attention to the task of driving. 

Congratulations, you’ve managed to free yourself of simply focusing on the precise task of driving. You are pretty good at making forecasts because you are not totally centered around your-self, and instead you are actively absorbing (and filtering) information from your environment. And in relationship to the introduction of this blog, you are someone that does not ignore subtle cues and signals.

If you were to program all those keen skills into a next generation drive-assist system it would have features such as defensive driving heuristics, map-based reasoning and use your own experience to predict traffic flows. Moreover it would be smart enough to respond to changing situations with more acuity with or with-out you in the loop. 

And so it is also true that the goal of better understanding our surroundings exists all around us: air-line traffic control, supply chain management, in the battle-field, doctors and other critical decision makers must all maintain some level of situational awareness in dynamic and tricky environments.

The process of raising that situational awareness barometer starts with differentiating status (of something) from events and thus relying heavily on surveillance (more passive monitoring) and reconnaissance (actively targeting someone or something) to recognize errant behavior, the terrain and environmental conditions, track targets and sense indicators and early warning signs. 

Think of an air traffic controller and the tools they need to get and maintain the right attention to track fast moving objects from colliding with each other in mid-air. 

It is increasingly apparent that in Cyber space (as in land, air and sea) there is virtual terrain and dimensions of time and space. To conduct commerce, serve citizens and communicate without some sort of handle on one’s surroundings is akin to walking in a dark alley with no perception whatsoever. It’s out of the question. 

Organizations of course rely on intrusion detection systems, event monitoring, incident response and readiness teams, anti-virus scanners and well managed applications and operating systems. Hopefully that pristine infrastructure or application is under a digital microscope where anything that is out of place or odd will be observed. 

The challenge is that observation or witnessing an event, is again different and harder than forecasting or predicting an outcome.

For example, consider an trusted insider that is observed downloading sensitive files for an extended period of time - after hours. On the surface there may have been no reason to doubt any misuse of privileges. There may have been no “rule-breaking" behavior. With some  projection and connection of the dots, there may be an opportunity to prevent a serious incident of data theft. Consider, if that same individual 4 months earlier was placed on administrative leave and 1 year earlier had visited a web site that is known to distribute malware. 

The point is that most of today’s IT security systems that help gleam what’s happening, what has happened and what is about to happen are either disconnected or most likely not in place at all. 

As we live, work and play in cyber space - organizations and all of us must raise our own situational awareness and in different ways. Whether it’s changing passwords on a regular basis, updating anti-virus definitions or avoiding that tempting link in our emails. 

Organizations and government agencies must also up the ante in terms of accurately detecting suspicious behavior, putting in place credible deterrents and automating responses that will minimize the impact of a potential threat actually occurring - whether that threat is known or unknown. They must also get better at working with a wider latitude of information that originates in cyber-space and must be correlated to the physical world.