Mehul Ruben Das12 Jan. 2023 19:45:54 IST
Mental health is a rather tricky subject to deal with, even with the best of intentions. Trust, both in the counselor and in the process, is very important. So how do artificial intelligence and machine learning fit into this? A US mental health platform recently conducted an experiment to find out how AI, specifically ChatGPT, can be used in counseling. Unfortunately for them, the experiment caused more problems than it solved.
Koko, a mental health platform, used ChatGPT in counseling sessions with more than 4,000 users, raising ethical concerns about using AI bots to treat mental health.
Koko is a nonprofit mental health platform that connects teens and adults in need of mental health help with volunteers through messaging apps like Telegram and Discord. On Friday, Koko co-founder Rob Morris announced on Twitter that his company conducted an experiment to give 4,000 people AI-written mental health care without informing them first, to see if they could tell any difference.
Critics have called the experiment highly unethical because Koko was not given informed consent by people seeking help.
Koko works through a Discord server, users log into the Koko Cares server and send direct messages to a Koko bot that asks several multiple choice questions such as “What’s the darkest thought you have about this?”. It then shares one person’s concerns – written as a few sentences of text – anonymously with someone else on the server who can reply anonymously with their own short message.
During the AI experiment, which involved about 30,000 messages, volunteers providing assistance to others had the option of using a response automatically generated by OpenAI’s GPT-3 major language model, the model on which ChatGPT is based, in instead of writing one yourself.
After the experiment, Morris posted a thread on Twitter, explaining the experiment they conducted. This is where things got ugly for Koko. Morris says people rated the AI-crafted responses highly until they learned they were AI-written, suggesting a significant lack of informed consent during at least one stage of the experiment.
Morris received many responses criticizing the experiment as unethical, citing concerns about the lack of informed consent and whether an Institutional Review Board (IRB) approved the experiment.
The idea of using AI as a therapist is far from new, but the difference between Koko’s experiment and typical AI therapy approaches is that patients typically know they are not talking to a real human being.
In Koko’s case, the platform offered a hybrid approach where a human intermediary could preview the message before sending it, rather than a direct chat format. But without informed consent, critics claim Koko violated prevailing ethical standards designed to protect vulnerable people from harmful or abusive investigative practices.
{n.callMethod? n.callMethod.apply(n,arguments):n.queue.push(arguments)}
;
if(!f._fbq)f._fbq=n;n.push=n;n.loaded=!0;n.version='2.0';
n.queue=[];t=b.createElement(e);t.async=!0;
t.src=v;s=b.getElementsByTagName(e)[0];
s.parentNode.insertBefore(t,s)}(window,document,'script',
'https://connect.facebook.net/en_US/fbevents.js');
fbq('init', '259288058299626');
fbq('track', 'PageView');
Source link
