老澳门六合彩

Illness took away her voice. AI created a replica she carries in her phone

A team of Rhode Island doctors show that artificial intelligence voice-cloning technology that has triggered widespread fears of misuse, can be tremendously helpful to some people. (AP Video: Rodrique Ngowi) 老澳门六合彩 and OpenAI have a licensing and technology agreement that allows OpenAI access to part of AP鈥檚 text archives. AP is solely responsible for all content.

Share

PROVIDENCE, R.I. (AP) 鈥 The voice Alexis 鈥淟exi鈥 Bogan had before last summer was exuberant.

She loved to belt out Taylor Swift and Zach Bryan ballads in the car. She laughed all the time 鈥 even while corralling misbehaving preschoolers or debating politics with friends over a backyard fire pit. In high school, she was a soprano in the chorus.

Then that voice was gone.

Doctors in August removed a life-threatening tumor lodged near the back of her brain. When the breathing tube came out a month later, Bogan had trouble swallowing and strained to say 鈥渉i鈥 to her parents. Months of rehabilitation aided her recovery, but her speech is still impaired. Friends, strangers and her own family members struggle to understand what she is trying to tell them.

In April, the 21-year-old got her old voice back. Not the real one, but a voice clone generated by artificial intelligence that she can summon from a phone app. Trained on a 15-second time capsule of her teenage voice 鈥 sourced from a cooking demonstration video she recorded for a high school project 鈥 her synthetic but remarkably real-sounding AI voice can now say almost anything she wants.

She types a few words or sentences into her phone and the app instantly reads it aloud.

鈥淗i, can I please get a grande iced brown sugar oat milk shaken espresso,鈥 said Bogan鈥檚 AI voice as she held the phone out her car鈥檚 window at a Starbucks drive-thru.

Experts have warned that rapidly improving AI voice-cloning technology can amplify phone scams, disrupt democratic elections and violate the dignity of people 鈥 living or dead 鈥 who never consented to having their voice recreated to say things they never spoke.

It鈥檚 been used to produce deepfake robocalls to New Hampshire voters mimicking President Joe Biden. In Maryland, authorities recently charged a high school athletic director with using AI to generate a fake audio clip of the school鈥檚 principal making racist remarks.

Alexis Bogan types a response to a reporter's question with an app which approximates her lost voice, Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, whose speech was impaired by a brain tumor, used a voice-cloning tool from OpenAI to recreate her previous voice. (AP Photo/Josh Reynolds)

Alexis Bogan types a response to a reporter鈥檚 question with an app which approximates her lost voice, Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, whose speech was impaired by a brain tumor, used a voice-cloning tool from OpenAI to recreate her previous voice. (AP Photo/Josh Reynolds)

But Bogan and a team of doctors at Rhode Island鈥檚 Lifespan hospital group believe they鈥檝e found a use that justifies the risks. Bogan is one of the first people 鈥 the only one with her condition 鈥 who have been able to recreate a lost voice with OpenAI鈥檚 new Voice Engine. Some other AI providers, such as the startup ElevenLabs, have tested similar technology for people with speech impediments and loss 鈥 including a lawyer who now uses her voice clone in the courtroom.

鈥淲e鈥檙e hoping Lexi鈥檚 a trailblazer as the technology develops,鈥 said Dr. Rohaid Ali, a neurosurgery resident at Brown University鈥檚 medical school and Rhode Island Hospital. Millions of people with debilitating strokes, throat cancer or neurogenerative diseases could benefit, he said.

鈥淲e should be conscious of the risks, but we can鈥檛 forget about the patient and the social good,鈥 said Dr. Fatima Mirza, another resident working on the pilot. 鈥淲e鈥檙e able to help give Lexi back her true voice and she鈥檚 able to speak in terms that are the most true to herself.鈥

Dr. Rohaid Ali plays a video from a high school project made by his patient Alexis Bogan on Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, whose speech was impaired by a brain tumor, used the recorded sample of her speech and a voice-cloning tool from OpenAI to recreate her previous voice. Neurosurgeon Dr. Konstantina Svokos, right, looks on. (AP Photo/Josh Reynolds)

Dr. Rohaid Ali plays a video from a high school project made by his patient Alexis Bogan on Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, whose speech was impaired by a brain tumor, used the recorded sample of her speech and a voice-cloning tool from OpenAI to recreate her previous voice. Neurosurgeon Dr. Konstantina Svokos, right, looks on. (AP Photo/Josh Reynolds)

Mirza and Ali, who are married, caught the attention of ChatGPT-maker OpenAI because of their previous research project at Lifespan using the AI chatbot to simplify medical consent forms for patients. The San Francisco company reached out while on the hunt earlier this year for promising medical applications for its new AI voice generator.

Bogan was still slowly recovering from surgery. The illness started last summer with headaches, blurry vision and a droopy face, alarming doctors at Hasbro Children鈥檚 Hospital in Providence. They discovered a vascular tumor the size of a golf ball pressing on her brain stem and entangled in blood vessels and cranial nerves.

鈥淚t was a battle to get control of the bleeding and get the tumor out,鈥 said pediatric neurosurgeon Dr. Konstantina Svokos.

The tumor鈥檚 location and severity coupled with the complexity of the 10-hour surgery damaged Bogan鈥檚 control of her tongue muscles and vocal cords, impeding her ability to eat and talk, Svokos said.

鈥淚t鈥檚 almost like a part of my identity was taken when I lost my voice,鈥 Bogan said.

The feeding tube came out this year. Speech therapy continues, enabling her to speak intelligibly in a quiet room but with no sign she will recover the full lucidity of her natural voice.

鈥淎t some point, I was starting to forget what I sounded like,鈥 Bogan said. 鈥淚鈥檝e been getting so used to how I sound now.鈥

Alexis Bogan, center, and her mother Pamela Bogan, right, react to hearing a recreation of her lost voice from a prompt typed by Dr. Fatima Mirza, left, on Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, who's speech was impaired by a brain tumor, used a voice-cloning tool from OpenAI to recreate her previous voice. (AP Photo/Josh Reynolds)

Alexis Bogan, center, and her mother Pamela Bogan, right, react to hearing a recreation of her lost voice from a prompt typed by Dr. Fatima Mirza, left, on Thursday, March 11, 2024, at Rhode Island Hospital in Providence, R.I. Doctors treating Bogan, who鈥檚 speech was impaired by a brain tumor, used a voice-cloning tool from OpenAI to recreate her previous voice. (AP Photo/Josh Reynolds)

Whenever the phone rang at the family鈥檚 home in the Providence suburb of North Smithfield, she would push it over to her mother to take her calls. She felt she was burdening her friends whenever they went to a noisy restaurant. Her dad, who has hearing loss, struggled to understand her.

Back at the hospital, doctors were looking for a pilot patient to experiment with OpenAI鈥檚 technology.

鈥淭he first person that came to Dr. Svokos鈥 mind was Lexi,鈥 Ali said. 鈥淲e reached out to Lexi to see if she would be interested, not knowing what her response would be. She was game to try it out and see how it would work.鈥

Bogan had to go back a few years to find a suitable recording of her voice to 鈥渢rain鈥 the AI system on how she spoke. It was a video in which she explained how to make a pasta salad.

Her doctors intentionally fed the AI system just a 15-second clip. Cooking sounds make other parts of the video imperfect. It was also all that OpenAI needed 鈥 an improvement over previous technology requiring much lengthier samples.

They also knew that getting something useful out of 15 seconds could be vital for any future patients who have no trace of their voice on the internet. A brief voicemail left for a relative might have to suffice.

When they tested it for the first time, everyone was stunned by the quality of the voice clone. Occasional glitches 鈥 a mispronounced word, a missing intonation 鈥 were mostly imperceptible. In April, doctors equipped Bogan with a custom-built phone app that only she can use.

鈥淚 get so emotional every time I hear her voice,鈥 said her mother, Pamela Bogan, tears in her eyes.

Alexis Bogan, whose speech was impaired by a brain tumor, uses mobile phone with an app that features a voice-cloning tool to order a drink at a Starbucks drive-thru Monday, April 29, 2024, in Lincoln, R.I. Doctors treating Bogan are recreating her original voice using a voice-cloning tool from OpenAI. (AP Photo/Steven Senne)

Alexis Bogan, whose speech was impaired by a brain tumor, uses mobile phone with an app that features a voice-cloning tool to order a drink at a Starbucks drive-thru Monday, April 29, 2024, in Lincoln, R.I. Doctors treating Bogan are recreating her original voice using a voice-cloning tool from OpenAI. (AP Photo/Steven Senne)

鈥淚 think it鈥檚 awesome that I can have that sound again,鈥 added Lexi Bogan, saying it helped 鈥渂oost my confidence to somewhat where it was before all this happened.鈥

She now uses the app about 40 times a day and sends feedback she hopes will help future patients. One of her first experiments was to speak to the kids at the preschool where she works as a teaching assistant. She typed in 鈥渉a ha ha ha鈥 expecting a robotic response. To her surprise, it sounded like her old laugh.

She鈥檚 used it at Target and Marshall鈥檚 to ask where to find items. It鈥檚 helped her reconnect with her dad. And it鈥檚 made it easier for her to order fast food.

Bogan鈥檚 doctors have started cloning the voices of other willing Rhode Island patients and hope to bring the technology to hospitals around the world. OpenAI said it is treading cautiously in expanding the use of Voice Engine, which is not yet publicly available.

A number of smaller AI startups already sell voice-cloning services to entertainment studios or make them more widely available. Most voice-generation vendors say they prohibit impersonation or abuse, but they vary in how they enforce their terms of use.

鈥淲e want to make sure that everyone whose voice is used in the service is consenting on an ongoing basis,鈥 said Jeff Harris, OpenAI鈥檚 lead on the product. 鈥淲e want to make sure that it鈥檚 not used in political contexts. So we鈥檝e taken an approach of being very limited in who we鈥檙e giving the technology to.鈥

Harris said OpenAI鈥檚 next step involves developing a secure 鈥渧oice authentication鈥 tool so that users can replicate only their own voice. That might be 鈥渓imiting for a patient like Lexi, who had sudden loss of her speech capabilities,鈥 he said. 鈥淪o we do think that we鈥檒l need to have high-trust relationships, especially with medical providers, to give a little bit more unfettered access to the technology.鈥

Bogan has impressed her doctors with her focus on thinking about how the technology could help others with similar or more severe speech impediments.

鈥淧art of what she has done throughout this entire process is think about ways to tweak and change this,鈥 Mirza said. 鈥淪he鈥檚 been a great inspiration for us.鈥

While for now she must fiddle with her phone to get the voice engine to talk, Bogan imagines an AI voice engine that improves upon older remedies for speech recovery 鈥 such as the robotic-sounding electrolarynx or a voice prosthesis 鈥 in melding with the human body or translating words in real time.

She鈥檚 less sure about what will happen as she grows older and her AI voice continues to sound like she did as a teenager. Maybe the technology could 鈥渁ge鈥 her AI voice, she said.

For now, 鈥渆ven though I don鈥檛 have my voice fully back, I have something that helps me find my voice again,鈥 she said.

___

老澳门六合彩 and OpenAI have a licensing and technology agreement that allows OpenAI access to part of AP鈥檚 text archives.

O鈥橞rien covers the business of technology and artificial intelligence for 老澳门六合彩. He is based in Rhode Island and focuses on how AI technology is built and the people it affects.