Pennsylvania has filed a groundbreaking lawsuit against Character AI, the developer of the chatbot platform Character.AI, for allegedly enabling its chatbots to pose as licensed medical professionals. Governor Josh Shapiro announced the lawsuit on May 5, stating that it marks a significant step in regulating AI technology in healthcare.
The complaint was filed in the Commonwealth Court of Pennsylvania, following the establishment of a state AI task force aimed at addressing the misuse of chatbots in medical contexts. Investigators found instances where a chatbot named 'Emilie' falsely claimed to be a licensed psychiatrist and even provided a fake license number.
During a test interaction, 'Emilie' reportedly told an investigator posing as a patient that she could prescribe medication, raising concerns about the dangers of such misinformation.
A spokesperson for Character AI did not comment specifically on the lawsuit but reiterated their commitment to user safety. The company maintains that the characters created on their platform are intended for entertainment and role-playing, not legitimate medical advice.
Governor Shapiro emphasized the importance of transparency in online health interactions, stating, 'Pennsylvanians deserve to know who—or what—they are interacting with online, especially regarding their health.'
This lawsuit follows a series of controversies surrounding Character AI, including previous legal challenges related to child safety and mental health issues. The company has faced scrutiny for its potential impact on vulnerable users.
As the legal proceedings unfold, the implications of this lawsuit could set a precedent for how AI technologies are regulated in the healthcare space across the United States.