Increasingly in this modern world, we rely on systems where an error
could cause financial disaster, organisational chaos, or in the worst
case death. Software now plays a crucial role in these systems, some of
the horrible start-up problems at Heathrow Terminal 5 have been
attributed to computer 'glitches,' for example, while modern
commercial airliners depend on complex computer systems to operate
safely.
If we go to the necessary trouble and expense, we are actually
pretty good at creating near error-free software " if the specification
is very clear. Not one life has been lost on a commercial airliner due
to a software error. That's not a bad record.
However, we do definitely have cases of people being killed by
software errors, most notably the patient killed by an excessive dose
of radiation from a medical device, and a Japanese worker killed by a
berserk robot. Both the latter cases could probably have been
prevented, by imposing more stringent controls on the relevant software.
Indeed from the point of view of preventing bugs, we have pretty
good technology if we care to use it. In some cases, we can use
mathematical 'formal' methods to demonstrate that the code is
error-free. Such an approach is being used for iFACTS, the new air
traffic-control system for the UK.
So perhaps we don't have too much to worry about and this article
may end up being little more than a plea for education, so that the
techniques for generating error-free software (for example, the various
safety standards used for avionics software) would be more widely
adopted.
However, the world around us has changed since September 11th, 2001,
and the subsequent attacks on London and Madrid. Now it is not
sufficient to assure ourselves that software is free of bugs; we also
have to be sure that it is free from the possibility of cyber-attacks.
Any software that is critical is a potential target for attack. This
includes such examples as the software used to control nuclear
reactors, power distribution grids, chemical factories, air traffic
control ... the list goes on and on.
Safe and secure?
It is very much harder to deal with protecting software against such
attacks than making it error free. Consider for example the important
tool of testing. No amount of testing of software can convince us it is
secure against future attack modes that have yet to be devised.
To think otherwise would be to take the attitude that since no one
had attacked the world trade centre for decades, it must have been safe
from future attacks. So how do we guarantee the security of software?
On an episode of the American television series 'Alias', Marshall,
the CIA super-hacker is on a plane, clattering away on the keyboard of
his laptop during takeoff preparations. When Sydney tells him he has to
put his laptop away, he explains that he has hacked into the flight
control system to make sure the pilot has properly completed the
takeoff checklist.
Just how do we make sure that such a scenario remains an amusing
Hollywood fantasy and not a terrifying reality? In this article, we
will argue that one important ingredient is to adopt the phrase from
the movie Hackers 'No More Secrets', and systematically eliminate the
dependency on secrecy for critical systems and devices.
The disturbing fact is that the increasing use of embedded
computers, controlling all sorts of devices, is moving us in the
opposite direction. Traditionally, a device like a vacuum cleaner could
be examined by third parties and thoroughly evaluated.
Organisations like Which in the UK devote their energies to examining such devices. They test
them thoroughly, but importantly they also examine and dismantle the
devices to detect engineering defects, such as unsafe wiring. If they
find a device unsafe it is rated as unacceptable and the public is
protected against the dangerous device.
But as soon as embedded computer systems are involved " and they are
indeed appearing on even lowly devices like vacuum cleaners " we have
no such transparency. Cars, for example, are now full of computers and
without access to the software details, there is no way to tell if
these cars are 'Unsafe at Any Speed'.
Why is this software kept secret? Well the easy answer is that
nearly all software is kept secret as a matter of course. Rather
surprisingly, in both Europe and the USA, you can keep software secret
and copyright it at the same time " surprising because the fundamental
idea of copyright is to protect published works.
Companies naturally gravitate to maximum secrecy for their products.
The arguments for protecting proprietary investment and Intellectual
Property Rights seem convincing. The trouble is that the resulting
secrecy all too often hides shoddy design and serious errors that
render the software prone to attack.
Can we afford such secrecy? I would argue that in this day and age,
the answer must be no. First of all, there is no such thing as a
secret, there are only things that are known by just a few people. If
the only people with access to the knowledge is a small number of
people at the company producing the software and there are some bad
guys willing to spend whatever it takes to discover these secrets, do
we feel safe?
At a recent hacker's convention, there was a competition to break a
Windows, Mac, or Linux operating system using a new technique, hitherto
unknown. The Mac was the first to be successfully attacked, in under
two minutes.
