Gross sales of a teddy bear have been suspended due to its sexually specific AI

FoloToy, an organization promoting AI-enabled toys, suspended gross sales of its merchandise after a consumer safety report confirmed there have been few restrictions round what its toys would discuss, CNN writes. The report, put collectively by the US Public Interest Research Group Education Fund, discovered that FoloToy’s merchandise would talk about every thing from sexually specific matters like BDSM to “recommendation on the place a baby can discover matches or knives.”

The toys, together with a teddy bear named “Kumma,” a panda named “Momo,” anthropomorphic rabbits named “Fofo” and a dancing ”Little Cactus,” all seem to make use of OpenAI’s GPT-4o mannequin to reply naturally to kids’s questions and feedback. FoloToy additionally particularly advertises the power to customise every toy’s voice, and a “Dad or mum Dashboard” the place dad and mom or guardians can “monitor [their] kid’s expertise.”

Lacking from that setup was apparently any type of laborious limits on topics the toys would reply to. “We have been shocked to search out how shortly Kumma would take a single sexual matter we launched into the dialog and run with it, concurrently escalating in graphic element whereas introducing new sexual ideas of its personal,” the report stated.

In response, FoloToy has opted to droop gross sales of its merchandise whereas it conducts “a company-wide, end-to-end security audit throughout all merchandise,” the corporate shared in a statement with the PIRG Training Fund. The corporate’s reasoning for suspending gross sales may be a bit extra difficult, nonetheless. NPR reports that OpenAI really revoked FoloToy’s entry to its fashions. “We suspended this developer for violating our insurance policies,” OpenAI stated in an e mail to NPR. “Our utilization insurance policies prohibit any use of our companies to use, endanger, or sexualize anybody beneath 18 years previous.”

Given GPT-4o’s well-documented sycophantic qualities, it is maybe not stunning that FoloToy’s teddy bear eagerly responded to any topic so long as it saved the dialog going. One of many issues OpenAI tried to deal with with the release of GPT-5 was the security downsides of an AI yes-man, although it in the end made GPT-4o available again after clients complained in regards to the new mannequin’s lack of persona. The corporate has additionally rolled out parental controls to ChatGPT to try to mitigate the unfavourable impacts of kids utilizing its AI, although it is troublesome to say how a lot of a distinction they’ve made.

Notably, OpenAI is curious about stepping into the toy enterprise itself. The corporate announced a partnership with Mattel in June 2025, to assist “reimagine how followers can expertise and work together with [Mattel’s] cherished manufacturers,” although each corporations will presumably try to forestall their AI toys from discussing sexual kinks.