Will robots make doctors obsolete? Nothing could be further from the truth

“Doctor, will they replace you with a robot?”

The question takes me by surprise but I suppress my instinct to laugh.

His life has been marked by mental illness and cancer. The first diagnosis presented after a series of missed opportunities as he cycled through woeful visits that ignored his symptoms. Finally, one doctor listened and his life changed.

In retrospect, his cancer was just waiting to be found by the first person to seriously entertain the notion that mental illness often coexists with physical illness. Fortunately, the cancer was amenable to a cure.

Related: Robots and AI to give doctors more time with patients, says report

Many years later, he faced a difficult decision when the cancer recurred and could not be removed. It was causing no symptoms and in fact, he felt great. Chemotherapy was feasible but the timing must be right. Give it too soon and he’d confront toxicities. Leave it too late and he may miss the window of opportunity. Given the same equation, different people arrived at different conclusions.

“What is important to you?” I asked when we first met.

His grandchildren, he replied. Having missed out on their early years, now he played with them and helped them with homework, but the best part of the day was driving them to school. This simple wish became the defining theme of his journey and he chose to defer treatment.

A year passed before things changed, although he remained well. I presented his choices – the standard recommendation of an aggressive regimen with substantial toxicities or a less intensive combination with less toxicity but potentially less efficacy.

“What would you do?” His eyes looked at me searchingly.

I told him honestly that different oncologists would advise differently – some would go all out while others might adopt a stepwise approach. Also, statistics applied to groups rather than an individual. For example, no one could have said he’d stay well for so long without treatment and it was hard to say how he would now respond to treatment.

I gently led him back to the same question: “What matters to you?”

“My quality of life.”

“Would a potential few months of extra survival sway your decision?”

“Not at the cost of suffering.”

This time, he accepted “a sniff” of chemotherapy. And now, all the headlines proclaiming the virtues of artificial intelligence in overcoming the limitations of human medicine have got him thinking.

A watch to tally our steps, count our calories, nudge us to exercise and even detect heart abnormalities.

Algorithms that beat radiologists at reading mammograms. Robots that detect the earliest signs of Alzheimer’s disease through eye patterns years before the sufferer notices something amiss.

Robots that delve into places where surgeons can’t. Computers that beat pathologists in tirelessly and accurately examining slides. Machines that outclass oncologists in predicting who will respond favourably to chemotherapy.

No wonder he is worried.

But as usual, the reality is more nuanced. Better technology may detect more cancers but what then? When South Korea began to offer a $30 thyroid ultrasound as an inexpensive add-on to existing cancer screening, thyroid cancer diagnoses shot up. In a country where 400 people died of thyroid cancer each year, 400,000 were now diagnosed with the condition. People didn’t suddenly begin dying in greater numbers but almost everyone was treated, with complications ranging from fear and anxiety to unnecessary surgery, vocal cord paralysis and lifelong thyroid replacement, not to mention the stigma of having cancer.

The problem of overdiagnosis is often mentioned in relation to two common cancers: breast and prostate. In both cases, enhanced technology is already detecting small abnormalities that may never result in harm during a lifetime. Machine-learning may trump human interpretation but merely making a diagnosis does not bring us closer to the truth about the impact of the finding. In other words, will the cancer ever cause symptoms, and crucially, will the patient die from it? How will the knowledge of cancer alter the rest of a person’s days?

Machines do not place significance on the intangible values that inform our lives. This may sound academic, but as an oncologist, I have a front seat to the untold psychological harm associated with the mere mention of cancer and the panic to treat it. Rarely anyone wants to wait and see but terrified patients are willing to sustain perennial injury or even risk their life to remove the cancer. Cancer patients are fearful, and oncologists hate doubt, making it easy to err on the side of overtreatment and exchange early relief for delayed harm.

Wearable technology, robot-assisted surgery and artificial intelligence-aided diagnoses are here to stay and likely improve lives but only as a complement to human experience, and we will have to address their ethical and legal implications. Do we really want to know our propensity for dementia decades before its (unpredictable) onset? Will we have to tell our workplace that we are suicide-prone? Exactly who will have access to the health data on our device?

Related: The AI program that can tell whether you may go blind

Much of the advice I gave patients 20 years ago as a resident might now be deemed outdated, such is the progress of cancer medicine. But one thing is unchanged: an abiding respect for the human element that makes all the difference to the experience of illness.

Therefore, while it might appear as if technology will make doctors an optional extra, from where I sit nothing could be further from the truth. Understandably, everyone wants a quick diagnosis and timely treatment but by far the most earnest gratitude is reserved for professionals who help patients navigate the many dimensions of illness with genuine care and compassion. We don’t remember machines, but no one forgets the experience of someone holding a hand, sitting through tears, explaining things and listening. For all those who scoff that the job of medicine is to diagnose and treat and not to hold a hand or speak nicely, I say wait until you get sick. In the face of astonishing new drugs and discoveries, the art of medicine will calmly hold on to its relevance.

But all this seems too weighty to explain to the patient in the next few seconds, so I merely say: “The robot wouldn’t enjoy hearing about your grandkids as much as I do, so I think I’ll stay on a bit longer.”

His relief is clear as he beams at me.

“Doc, do you have time to see just one photo?”