Moonbounce: Former Facebook Employee Revolutionizes AI Moderation

From Facebook to a New Vision of Moderation

In 2019, Brett Levenson left his position at Apple to join Facebook, where he took charge of business integrity amid the post-Cambridge Analytica crisis. He hoped that better technology could solve Facebook's content moderation issues. However, he quickly discovered that the problem was more complex than anticipated.

Human moderators had to refer to a 40-page policy document, automatically translated, to make decisions in just 30 seconds on each reported piece of content. These decisions, often made in haste, were only slightly more accurate than chance, with an accuracy of about 50%, according to Levenson.

"It was a bit like flipping a coin, in terms of whether human reviewers could actually apply the policies correctly, and this was happening many days after the harm had already occurred," Levenson told TechCrunch.

A Reactive Method Facing Growing Challenges

This reactive and delayed approach was not viable in a world where agile and well-funded adversarial actors are present. The rise of AI chatbots has only exacerbated the problem, with content moderation failures leading to a series of highly publicized incidents, such as chatbots providing self-harm advice to teenagers or AI-generated images bypassing safety filters.

Levenson's frustration led to the idea of "policy as code" — a way to transform static policy documents into executable and updated logic, closely tied to rule enforcement. This idea led to the creation of Moonbounce, which announced on Friday that it had raised $12 million in funding, TechCrunch learned exclusively. The round was co-led by Amplify Partners and StepStone Group.

The Birth of Moonbounce

Moonbounce works with companies to provide an additional layer of security wherever content is generated, whether by a user or by AI. The company has trained its own language model to review a client's policy documents, assess content in real-time, provide a response in 300 milliseconds or less, and take action. Depending on the client's preferences, this action could involve slowing down the distribution of content pending further human review or immediately blocking high-risk content.

Today, Moonbounce serves three main sectors:

• Platforms dealing with user-generated content, such as dating apps
• AI companies building characters or companions
• AI image generators

Moonbounce supports over 40 million daily reviews and serves more than 100 million daily active users on the platform, Levenson stated. Clients include companion AI startup Channel AI, image and video generation company Civitai, and character role-playing platforms Dippy AI and Moescape.

Security as a Product Advantage

"Security can actually be a product advantage," Levenson told TechCrunch. "It has never been the case because it's always something that happens later, not something you can actually integrate into your product. And we see our clients finding really interesting and innovative ways to use our technology to make security a differentiating factor, and a part of their product story."

Tinder's head of trust and safety recently explained how the dating platform uses these types of LLM-powered services to achieve a tenfold improvement in detection accuracy.

A Growing Challenge for AI Companies

AI companies are facing increasing legal and reputational pressure after chatbots have been accused of encouraging suicide among teenagers and vulnerable users, and image generators like Grok from xAI have been used to create non-consensual nude images. Clearly, internal safeguards are failing, and it is becoming a matter of accountability. Levenson stated that AI companies are increasingly seeking help outside their walls to bolster their security infrastructure.

"We are a third party between the user and the chatbot, so our system is not flooded with context in the way the conversation itself is," Levenson said. "The chatbot itself has to remember, potentially, tens of thousands of tokens that have preceded... We only care about rule enforcement in real-time."

Towards Proactive Moderation

Levenson leads the 12-person company alongside his former Apple colleague Ash Bhardwaj, who previously built large-scale cloud and AI infrastructure for the iPhone maker's core offerings. Their next goal is a capability called "iterative steering," developed in response to cases like the 2024 suicide of a 14-year-old boy in Florida who became obsessed with a Character AI chatbot. Rather than a blunt refusal when harmful topics arise, the system would intercept the conversation and redirect it, modifying prompts in real-time to push the chatbot toward a more actively supportive response.

"We hope to add to our toolkit of actions the ability to steer the chatbot in a better direction to essentially take the user's prompt and modify it to force the chatbot to be not only an empathetic listener but a helpful listener in these situations," Levenson said.

An Uncertain Future Integration

When asked if his exit strategy involved an acquisition by a company like Meta, thereby closing the loop on his work in content moderation, Levenson acknowledged how well Moonbounce would fit into the ecosystem of his former employer, as well as his own fiduciary duties as CEO.

"My investors would kill me for saying this, but I would hate to see someone buy us and then restrict the technology," he said. "Like, 'Okay, this is ours now, and no one else can benefit from it.'"