Take-home message:
-Elizabeth Holmes’ company Theranos tried to create a machine that could do hundreds of medical tests on a single drop of blood
-John Carreyrou’s book Bad Blood reports on the lies and secrecy that allowed so many people to believe the machine was real
In 1983, a software company named gave a demo of its latest product suite at a news conference. It was meant to be the Microsoft Office of its day. Behind the veneer of this demonstration, however, was a non-existent product. The demo, which should have been interactive, was . Eventually, the company went under when it couldn’t deliver the product it had pretended it had had all along.
Nobody died when Ovation failed to materialize in the end. The same cannot be said with complete certainty when it comes to the molecular diagnostic miracle machine that was promised by Silicon Valley startup Theranos.
Theranos (a combination of “therapy” and “diagnosis”) was the brain child of Elizabeth Holmes, who dropped out of university to create a machine that could accurately perform hundreds of clinical tests on a single drop of blood. If your doctor wants to test your potassium levels or check your thyroid hormone titres, they have to request a blood draw and, often, multiple tubes, as specific tests require the preservation of different components of your blood. Holmes, having a fear of needles, decided to simplify the process through technological innovation. What if, instead if this invasive venipuncture, you could give one drop of blood from one of your fingers?
The story of Theranos, as described in punctilious detail by Wall Street Journal reporter John Carreyrou in his book , is a study in deception. Early demonstrations in front of potential investors relied on pre-recorded computer screens, much like 1983’s Ovation, because their proprietary machine wasn’t reliable. When asked to compare their technology to the standard used in clinical labs, Theranos showed an amazing correlation… except that they had used actual standard lab equipment in lieu of their own technology, because the promised miracle still hadn’t materialized. The layers of pretence reached their peak when Vice President Joe Biden visited the company and was shown a fake lab.
Even Holmes’ own voice, often described by Carreyrou as “deep”, appears to have been manufactured to appeal to Silicon Valley’s “boys’ club” mentality.
The harm with this non-existent device, which was meant to be the iPod of diagnostics, comes from the fact that Theranos expanded beyond its security-heavy headquarters and into grocery stores and pharmacies. Safeway created wellness centres in select locations in the United States to house Theranos’ malfunctioning technology. When suspicious healthcare professionals started sending for the same blood work to established companies as well to compare, they quickly realized that Theranos’ results were wildly inaccurate. Carreyrou reports on a pregnant patient whose thyroid hormone level was declared by Theranos to have increased even higher than usual. The patient’s dose of medication was about to be unnecessarily raised… except her doctor had her retested by a trusted lab. Her hormone level was normal. An increase in the patient’s medication could have jeopardized her pregnancy.
Reading Carreyrou’s fascinating book, I was constantly reminded of how easily non-experts in positions of power can be fooled by sleight-of-hand and charisma. Holmes’ confidence and the tight secrecy enforced inside the company by her right-hand man and secret boyfriend, Sunny Balwami, were enough to cast a spell on four-star general James Mattis (who joined the company’s board), President Obama (who appointed Holmes as U.S. ambassador for global entrepreneurship), Rupert Murdock (who became the #1 investor in Theranos), and the mainstream media (who lavished Holmes with praise).
Investors and media personalities are not good judges of scientific validity. In the case of Theranos, it was former employees who realized that the game was rigged and that their bosses were committing fraud. It was scientists and medical professionals like Alan Beam, Tyler Shultz, and Erika Cheung who explained to a curious and unrelenting journalist how Theranos was deviating from good lab practices and building a smokescreen to fool nearly everyone.
The investigative reporting detailed in John Carreyrou’s Bad Blood makes for an occasionally repetitive read, especially in the beginning, as chapter after chapter introduces a new employee who quickly gets sacked. Lather, rinse, repeat. But once the story gains momentum and breadth, it reveals layers of absurdity and degrees of media spin that make us beg for its villains’ comeuppances. Bad Blood serves as a powerful reminder that science does not thrive in secrecy; it is a collaborative process that requires transparency and open criticism. Especially when our health is on the line.
For those who want to delve deeper into “How to artificially boost the accuracy of your test”, click here
There was a very easy way to cheat when reporting on the accuracy of their clinical tests, and Carreyrou describes the process on page 187 of his book.
Theranos’ machine, the Edison (which wasn’t working very well), was supposed to be able to detect syphilis in a drop of blood. The test needed to be validated, meaning results from the Edison had to be compared against results from a reliable, commercial machine.
One of Theranos’ scientists and soon-to-be whistleblower, Tyler Shultz, was assigned to test the validity of this test, and he told Carreyrou that the Edison could only identify 65% of the known positive blood samples as positive. This is a terrible batting average, only slightly better than flipping a coin to determine if you have syphilis or not. However, Theranos reported a figure of 95% instead of 65%. How was this possible?
A separate incident involving vitamin D testing reveals the answer. When another scientist, Erika Cheung, ran known samples on the machine prior to running actual unknown samples (what we would call a “quality control run” or “running controls”), two of her 12 data points were wrong. This should have resulted in the machine being taken offline and inspected. However, a fellow employee decided to mark these two data points as “outliers”—results that are significantly above or below the next nearest data point and may or may not be dismissible. With these “outliers” out of the way, it looked as if the machine performed admirably.
If a particular sample is positive for syphilis but your machine says it’s negative, simply mark this data point as an “outlier” and take it out of your calculations, seems to be the company’s policy. By cherry-picking the data they liked and ignoring the mistakes, the heads of Theranos allowed themselves to portray their technology as much more accurate than it really was.
Want to engage with this content?
