If you had to choose between believing the sun sets twice or trusting the weather app, the choice is obvious. But what happens when the fallacy isn’t so clear-cut, and automation bias quietly takes the wheel?
“Isn’t the sun supposed to be, you know, visible?” I brought up the obvious question that was clearly on everybody’s mind. After all, we had driven over an hour and a half to get to the beach with the prettiest sunset in Bali. The moment I stepped out of the car, I looked up. All I saw was a cloudy sky. There was no sun in sight. I leaned over to check the time on my sister’s phone: 7:01 PM. Traffic had made us arrive much later than we had planned. I did the only logical thing, for an eight-year-old, at least: I checked the weather app. Unfortunately, it only confirmed my worst fears.
We missed the sunset. We shouldn’t have stopped to get extra food. Now we were paying the price. I couldn’t bear to lift my face as I uttered the dreaded words, “We missed it. The sunset is over. We should just go back and order room service or something.”
“Eesha, are you dumb?” What? I looked up at my 15-year-old sister, astonished. Had she not heard me? The app clearly said that the sunset happened at 6:52, and it’s currently 7:01. Clearly, she needs to check her phone again.
“What do you mean?” I replied, thrusting the phone in her face. “Look, it says the sun set at 6:52 PM right here. That was nine minutes ago.”
“Eesha, you are actually stupid.”
“No, I’m not!”
“Yes, you are!”
“No, I’m not, didi! Why are you always so mean to me?” I asked, feeling my face flush in anger and embarrassment.
“Eesha, look up at the sky.” I rolled my eyes, but obliged.
I raised my head, and there it was: beams of light peeked through the clouds. How was that possible? I looked back down at the phone:
Then back up at the sky. Then back at my phone again. Did the sun come back up to set again? I spent the rest of the day in shock, unable to process the thought of whether the sun could set twice in a single day or the weather app could be wrong.
Looking back, I realised that the most embarrassing part of this story wasn’t that I thought the sun could set twice. Rather, it was the fact that I had to be told that the weather app could be wrong. When I was faced with two conflicting pieces of information–the literal sun in the sky, and a phone–I somehow decided that the iPhone 7 Plus in my hand was the more reliable source.
I would like to blame this on the fact that I was eight. Unfortunately, it turns out that adults aren’t much better.
This phenomenon is known as automation bias—our tendency to over-rely on automated systems when making decisions. It extends far beyond weather apps. A systematic review of over 74 studies found evidence of this tendency, ranging from fields like aviation to healthcare. In an alarming example, it was found that healthcare workers who initially made the right call would change their minds after receiving incorrect information from a computerized decision-support system.
This doesn’t mean that we should never trust technology. In lower-stakes situations, it makes perfect sense to rely on automation. That very same review found that automated decision-support systems generally improved people’s overall performance, even if they introduced a new source of error. If my calculator and I were to get a different answer for 203495 divided by 37.5, I would probably trust my calculator. If Google Maps tells me that turning down an unknown side road will save me 15 minutes, I’ll turn. These tools know things I don’t, calculate things faster than I can, and–most importantly–are usually right. The problem arises when that reasonable trust becomes automatic.
Technology's high rate of accuracy introduces a psychological paradox. The better technology becomes, the more rational it seems to trust it, and the easier it becomes to stop questioning it. Researchers found that trust is the biggest factor that influences people’s reliance on automated systems. When having to decide between our easily fooled senses and the historical accuracy of a machine, assuming that the machine is right feels like the rational choice.
The real issue isn’t that technology can be wrong. We already know that. The issue is that we have redefined what counts as “convincing” evidence. You can wake up feeling rested, but if your watch says that your sleep score was “31/100,” you start doubting your own body.
And given the fact that our senses can be tricked, it’s understandable. Our eyes can be deceived. Memory can be wrong. My spatial awareness is…lacking. Technology gives us the objective information that we otherwise may not have known.
Maybe that’s why it is so hard to tell when our trust extends a little too far. I was eight when I stood on that beach, staring at a screen that told me the sun had disappeared when it was still hanging in the sky. It is so easy to laugh and joke about it now. What is harder to laugh off is when I end up in a random place because of Google Maps, or when I dismiss something I remember because a Google search told me otherwise. Somewhere, the idea that “technology is usually correct” becomes “I am wrong.”
And that scares me more than a simple weather app getting the sunset wrong. Because as technology gets better at telling us where to go, how well we slept, what to watch, what to believe, and increasingly even what decisions to make, we should not lose the instinct to trust ourselves. Technology is right most of the time, but not all the time!
