A Healthy Dose of Skepticism
A Healthy Dose of Skepticism
Finding the Signal in the Noise
If something seems too good or too bad to be true, it probably isn’t true.
I was reminded of this in the past few days as I came across (yet another) CEO giving a cringe, company-wide call. (We’ll need to address this at some point. Why are so many CEOs out-of-touch? Completely cut off from reality? But that’s another post.)
The video below is from the CEO of Clearlink. I’m familiar with Clearlink, and this is pretty on-brand for them. Much of its leadership has always been like this, though I’ve never worked with this particular CEO. But clearly cut from the same cloth.
The CEO was praising an employee who got rid of their family pet so they could come back to the office. And then made a claim that seems too far-fetched to be true:
Tiktok failed to load.Enable 3rd party cookies or use another browser
If you haven’t watched the whole video, he claims that 30 people at Clearlink haven’t even opened their laptops for a month. He singles them out as “remote employees,” including the manager.
My first reaction to that was “nice.” But then a healthy dose of skepticism kicked in. Would 30 employees, including their manager, not work for an entire month? That’s possible, but feels incredibly unlikely. Even if you were fully checked out of a job and could legitimately step away for a month, wouldn’t you at least maintain some appearance of work?
If I were the CEO and was presented with this information, I would ask for the team to follow up and dive deeper. Is there a problem with our activity tracking? Are we sure we accounted for machines that are actually being used and not ones that have been swapped out for newer models? What other possibilities exist to explain a highly unbelievable data point?
Calling Bullshit
In our most recent book review (found below), Carl T. Bergstrom and Jevin D. West discuss this idea of data skepticism.
In the book, the authors discuss a study that found a correlation between people who had email addresses and those who were more likely to file insurance claims, especially in the early days of email. The insurance company saw this and highlighted it as a big risk. Anyone who has an email address is likely to file a claim!
At first glance, this finding seems to suggest that having an email address makes someone more prone to having an accident or filing insurance claims, implying a causal relationship.
However, the authors use this example to illustrate the importance of considering confounding variables, understanding the difference between correlation and causation, and examining the data collection process. In this case, the insurance company only collected email addresses when a person filed a claim. As a result, there was an inherent bias in the data, as email addresses were only present for those who had already filed claims.
Additionally, age can also be a confounding variable, as older individuals are generally more likely to file insurance claims but less likely to have email addresses, while younger individuals are less likely to file claims but more likely to have email addresses.
Taking these factors into account, the apparent correlation between having an email address and filing insurance claims disappears. It’s a good example to highlight the importance of scrutinizing the data collection process and examining underlying factors before drawing conclusions.
If only the CEO of Clearlink would have considered some other factors, he may not have made such an extreme claim.
Skepticism
A healthy dose of skepticism is usually a good idea. From our work to our personal lives. We shouldn’t allow ourselves to become complacent in what we think we understand or believe.
“Skepticism is the first step towards truth” - Denis Diderot
If we don’t apply a little skepticism to most claims, then we can fall victim to anything.
Keep reading with a 7-day free trial
Subscribe to Prodity: Product Thinking to keep reading this post and get 7 days of free access to the full post archives.