The Internet Is Going Live. Trust and Safety Has to Catch Up

Going live is no longer a novelty. On Tango, roughly 176,000 livestreams go out every day, while livestreaming, social commerce, gaming and other interactive environments are moving more of our digital lives from posting to participating. When the internet operates in real time, trust and safety has to operate in real time too.

The real challenge is knowing where to look

The challenge is no longer simply detecting harmful content. Software can identify nudity, weapons, violence, probable age and other signals across video, audio and text. The constraint is applying that scrutiny intelligently at scale.

Live video makes that harder because risk changes while the content is being made. A stream that looks harmless at 8:01 p.m. can look entirely different at 8:03. The people in the frame change, the conversation changes and behavior changes. At the same time, treating every broadcast as high-risk would waste enormous computing power and human attention while making the experience worse for legitimate users.

So the real question is not how often to inspect everything. It is how to identify where the risk is right now.

At Tango, an internal scoring service draws on behavioral and content signals to produce a risk score for each livestream. Streams with a higher risk score are sampled more frequently by automated deep review, which can itself restrict or terminate a stream, and are escalated to human moderators where necessary. What goes into that score is a policy decision as much as an engineering one: the possible presence of a young person, for example, carries more weight than many other signals and can move a stream to the front of the human-review queue.

Better safety does not come from spreading the same scrutiny thinly across everything. It comes from knowing where that scrutiny needs to go.

Harassment is where context decides everything

The gap becomes particularly clear once you move past content that is easy to label. Software can be trained to spot a weapon, detect certain kinds of nudity and recognize particular words. Human behavior is messier. Take harassment.

Going live means opening yourself to strangers in a way that posting a photograph does not. People respond immediately. They return repeatedly. They address you personally. That is what makes livestreaming powerful, and it is also what leaves creators exposed.

Harassment might be an obviously abusive message. But it might also emerge through persistence, repetition or an interaction that becomes steadily more uncomfortable. Something that looks harmless on its own can feel very different once you see it as part of a pattern, and a pattern is not always visible in a single feed or message.

This is why it is worth being skeptical whenever the future of online safety is presented simply as a race toward better AI. AI is essential to doing any of this at scale. It is not enough on its own.

What software finds, people still have to judge

AI is extraordinarily good at volume. It can work through more images, audio, text and behavior than any human team could handle, and identify the small fraction that deserves a closer look. What it does not reliably understand is context: intent, culture, language and the history between two people.

So the systems worth building are the ones that let software and people each do what they are better at. Age is a good example. At Tango, where a stream raises a concern that someone on camera may be underage, two independent automated systems assess it rather than one. Where the two disagree, the case goes to a person rather than whichever system is more confident.

Disagreement between machines is treated as a reason to involve a human, not a tie to be broken automatically. That work sits alongside human moderation running 24 hours a day, with dedicated compliance staff taking escalated cases and investigating further when something may be illegal.

That division of labor is the point. Software does not have to make every decision. Often its most valuable job is making sure the right person is looking at the right thing at the right moment.

The technology has to keep moving

The people breaking the rules adapt too, so a system that worked a few years ago cannot simply be left running. Filters change. Language changes. People learn what triggers moderation systems and look for ways around them.

Trust and safety therefore cannot be treated as a product a company finishes building. It has to keep learning.

This is bigger than moderation technology itself. As digital products evolve, safety teams have to evolve alongside them - and that sometimes means questioning features that were never designed with safety risks in mind.

Child safety shows why that matters

Nowhere is this more important than child safety. Tango requires broadcasters to be 18 or over, a stricter limit than many major livestreaming platforms apply. But an age limit is a starting point rather than a guarantee, which is why age assessment continues long after someone signs up rather than stopping at registration. The account holder may clearly be an adult, for example, but a child might appear in the background of a stream or use a parent's account to go live. When there is reason to believe an underage person may be broadcasting, the account can be routed to third-party identity verification before broadcasting is allowed to resume, and visuals compared between the stream and verification data.

Some of that work runs into challenges created by ordinary product features. Beautification filters and augmented-reality effects are everywhere in social video and usually harmless, but they also make it harder to judge someone's age. So Tango's moderation tools are trained to ignore those effects in the feed used for review.

There is a broader product lesson here. Every new consumer feature creates new possibilities for users and new blind spots for the people responsible for safety. Safety review has to move with product development.

For suspected child sexual abuse material, the safeguards are more specialized still. Material of this kind is reported to the National Center for Missing & Exploited Children, and the account is permanently banned. Detection does not depend on a single method, because material that may already be present in databases such as Thorn (co-founded by actor Ashton Kutcher and Demi Moore) and material that has never been seen before benefit from different approaches.

These are not pleasant systems to build. Serious safety work means designing for the situations you hope never happen.

Companies should try to break their own safety systems

There is also a cultural point. Technology companies naturally like proving that their products work. Trust and safety asks for close to the opposite instinct: looking hard for the places where they do not.

In April 2025, Tango completed an independent assessment it had commissioned of its moderation controls for explicit and illegal content. Within its scope, the assessment examined policy documentation, personnel, technology, moderation data and live stress-testing of the systems.

It concluded that Tango's policies, procedures and moderation capabilities were aligned with regulations and industry standards in the areas reviewed. Under live stress-testing, automated technology flagged and removed objectionable content within seconds, while human moderators accurately identified content and triggered enforcement.

But the point of an independent assessment is not to produce a flattering sentence for a press release. The useful part is learning what you did not already know.

Trust and safety improves when companies invite that kind of examination before someone else forces it on them. More technology leaders should be willing to try to break their own safety systems before bad actors do it for them.

Trust and safety is not a cost center

For years, the industry treated moderation as defensive plumbing: a way to keep regulators satisfied, meet app-store rules and take down what should not be there. That view is going out of date.

If your business depends on people being willing to show their faces, speak to strangers, build communities, spend money or make themselves vulnerable in public, then trust is not something sitting next to the product. Trust is part of the product. It requires technology, engineers, moderators, compliance expertise, outside scrutiny and continuous investment because the environment itself does not stop changing.

The companies that understand this will not be the ones promising a perfectly safe internet. Any product built around human interaction will encounter behavior its designers wish did not exist.

But, can the platform tell that the risk has changed? Can technology surface the right signals? Can a person get involved when context matters? Can the company learn from what happened and do better next time?

The hardest part of live is not seeing what happened. It is knowing, while it is still happening, where to look.

For two decades, the technology industry has optimized almost everything for speed: publishing, sharing, recommendations, payments and communication. Now the internet itself is becoming real time. Our ability to build trust has to catch up.