How Do Test Engineers Decide What’s Good Enough?

The statistical reasoning is sound, even if the verisimilitude of the digestion test is a bit uncertain. Bird populations change; Migration patterns, or what experts call “biomass waves,” are variable. In nearly half of the bird strike cases investigated by the Federal Aviation Administration, the turbine vault is so messy that no one can predict the type of culprit. (Davies, who has a wonderful ear for niche jargon, is pleased to say that the bird slur is called a “lie.”) We can be pretty sure that no traps can be traced back to flightless chickens. From 1990 to 2019, Davies writes, “the FAA recorded more bearded seal attacks (one) than in-flight chicken encounters (zero).” In the absence of a solid supply chain for osprey, testers are resorting to chicken gunning.

The use of perfect chickens reflects the compromise between processability and similarity. Experimental results are reliable only if the procedure is practical, reproducible and standardized. We don’t fly airplanes just because someone flew them once. At the same time, results are informative only if the procedure is a useful approximation of the actual conditions we expect to encounter. Most of the time some basic tricks will be enough. Manufacturers evaluate the usability of medical equipment by placing it in the hands of intended users, sometimes in fake hospital rooms filled with real beeping noises where nurses are interrupted by some idiot asking if they have “the blue Chevy that went off the alarm.”

Other examples are more challenging. Rocket sled’s John Stapp viewed the body in a distinct light—he once remarked, “To the human engineer, the human being is a thin, flexible sack of skin filled with thirteen gallons of fibrous and gelatinous material, inadequately supported by an articulated bony framework”—but most of us view flesh with greater sensuality. For example, safety features in cars have generally been evaluated with cadavers that are not at risk of further harm. Yet Davies observes that the dead “have their weaknesses.” First of all, they are not impact resistant and often suffer from pre-existing conditions. They are also expensive and difficult to source ethically. Crash test dummies are easier to produce and their uniformity allows for better experimental control, but they are imperfect substitutes. Produced in the image of the “average” man of a leaner age, they are intended to be one size fits all. They are women, children, and poor stand-ins for today’s “average” man.

Davies emphasizes that the testing industry is made up of trade-offs from top to bottom. While some products introduced are unnecessarily dangerous (there was never any reason for the Little Miss Hot Stove, a post-war toy, to reach six hundred degrees or for contemporary science kits to contain real uranium) other products are useless unless they can harm you. The ideal balance is not always simple. In the 1960s and 1970s, sixty thousand people a year were reaching into lawnmower drains to unclog them and ending up in the emergency room. In 1974, technocrats at the newly formed Consumer Product Safety Commission campaigned on behalf of the nation’s yard workers, eventually advocating for a dead man’s key to be found in the handle of every lawn mower. In four studies, hundreds of volunteers moved their hands from the handle of the lawnmower to the discharge chute; The results were collected in approximately three seconds. Three seconds became standard after engineers confirmed that an affordable mechanism could stop the blade within this range. Industry lobbyists thought the rule was too strict; safety advocates thought the knife should have stopped faster. The CPSC made a middling call, weighing minor additional costs against modest safety gains, and prevailed.

Table saws have gone the other way. Davies tells the story of an amateur woodworker named Steve Gass who invented an electrical method that would prevent America’s thirty thousand annual shop accidents. (Needing a replacement for his own meats, he settled on hot dogs.) The mechanism worked very well, but was badly defeated by the Big Saw, as it added hundreds of dollars to the price of the device.

Children’s medicine cabinet poisoning was a similar problem that had an ingenious and effective solution. Just as lawnmowers and saws cannot work with rubber blades, medicines cannot be replaced by placebos. These could be stored in bottles that were difficult to open, but if the bottles were too challenging, people made matters worse by leaving the caps open. The cure appeared in the form of the Palm N Turn container, which charted a path between the Scylla of a child’s ability to access pills and the Charybdis of an adult’s inability to access pills. Such containers are meticulously calibrated. Even after children are clearly shown the winning technique, they must surprise at least eighty percent of a panel of fifty young children within ten minutes. Then, within five minutes, they must submit to the will of at least ninety percent of a panel of one hundred adults between the ages of fifty and seventy. Since the early 1970s, deaths from unintentional childhood poisoning have fallen by four-fifths.

Leave a Comment