(This post currently requires client-side JS from
Constant Flawless Vigilance, or CFV, is a meme which occurs in several contexts across software engineering. CFV preëmpts normal risk analysis by endorsing exactly what it says on the tin: Never-ending observation by humans of complex systems to detect faults. If the observation is sufficiently continuous, and the humans are sufficiently trained, then CFV claims that faults will not occur.
We note CFV as a generalization of several hazardous memes which are common throughout software engineering. By isolating and documenting CFV, we hope to remove its paralyzing venom.
Linus’s Law, named for the creator of Linux, is traditionally stated as:
Given enough eyeballs, all bugs are shallow.
In other words, with enough people looking at a given piece of code, it is likely that eventually all bugs will be found. Or is it?
Let us consider the case of code review. One reviewer may find some bugs. Another reviewer may find other, different bugs. Linus’s Law suggests that there is a number of reviewers which will find all bugs. How many reviewers? The coupon collector’s result can be applied here; presuming that any given code reviewer is likely to find bugs on every reading of some code, it will take code reviewers to find all the bugs! This isn’t a guarantee, either; some bugs might remain hidden for a long time.
Constant Flawless Vigilance works here, but only statistically and at a cost of many engineer-hours.
My First IRC Bot
Let us talk of novice programmers. We shall take Alice to be such a novice. A common exercise for novice programmers, like our friend Alice, is to implement an IRC bot. Alice, like many novices, thinks that a code-evaluating feature is fun and easy to implement safely, and so she adds a naïve code evaluator to her bot. It works correctly, computing simple arithmetic facts and displaying weather information retrieved from the Web.
However, suppose Alice finds herself aggressed by Mallory, who sends the bot snippets of code which cause the bot to misbehave. Alice does not yet know about confused deputies nor ambient authority, but after a few searches on a 2017 search engine, she learns about the concept of input validation and begins implementing some validators for her bot.
Alice knows that trying to manipulate the text of incoming code is a fool’s errand, so her first attempt is to validate that no harmful syntax comes through, by pruning powerful nodes from the AST. While not directly aware of ambient authority, she intuitively forbids access to extremely powerful objects, and by constant dialogue with Mallory, Alice systematically eliminates more and more avenues of ingress.
However, eventually Mallory comes up with a flourish which finishes off
Alice’s dreams of safe evaluation. Perhaps it’s a careful example of coloring
within the lines, as in this series of Python
or perhaps it is a timeless eldritch classic like . Alice sighs and
starts learning how to implement a resource-limited sandbox.
Generalizing Little Bobby Tables
SQL injection has become well-known and popular, with themed websites discussing how to mitigate it. Unfortunately, this isn’t enough to reveal the deeper pattern of why injection happens. Injection from a host language into a DSL is caused by the host-DSL bridge lacking the ability to mix host and DSL values directly, and instead attempting an assembly-line series of fixups. A typical pipeline in such a system might have phases which:
- Rejects non-strings
- Lowers input to common encoding
- Strips invalid characters
- Escapes quote characters
- Quotes escape sequences
- Looks for and rejects Mallory’s PoC from last week
Does this work? Sometimes. But “sometimes” isn’t good enough when Constant Flawless Vigilance is being applied as an axiom; it’s all-or-nothing.
Avoiding Undefined Behavior in C
The C and C++ languages have undefined behavior, or UB. Following Robert O’Callahan’s example, we would like to point out that the task of writing correct C is an exercise in applying Constant Flawless Vigilance. Pascal Cuoq and John Regehr have a detailed and daunting survey of UB.
More generally, any feature or quirk in a programming language which requires Constant Flawless Vigilance to avoid accidentally provoking nasal demons or other similarly-devastating security compromises should be considered extremely dangerous. Some examples from popular programming languages:
- Pointer arithmetic
- String manipulation
- The entirety of Cuoq and Regehr, as already linked
- Everything listed for C
- Exception handling
- Operator overloading
- Object life cycle: implicit constructors and destructors, copy constructors, move semantics, structural sharing
- Automatic Semicolon Insertion
- Equality semantics with
- In the browser:
- Browser-specific bugs
JSONPfinally recognized as a Bad Idea
- Origin policies
- Importing code via
- Closure semantics capturing by name instead of by value
There are many flaws and warts not covered in this listing, but we have tried to focus on precisely those misfeatures which are so fragile that nothing short of the most Flawless and Constant of Vigils will protect programs from them.
Constant Flawless Vigilance is the concept that a perfect defensive posture, applied consistently and without fail, guarantees safety. We should reject it in favor of fault tolerance, error analysis, and correctness bounds.