It's back-to-school season, which shifts children's attention and screens time to other activities. The return to school season means that there will be a shift in how children interact daily with technology platforms, including social media, gaming, e-commerce and education. We will use technology platforms in different ways and volumes in the weeks ahead. This change in online activities for children brings new (and old!) risks that will require a shift in the focus of Trust & Safety teams.

Trust & Safety teams must quickly and effectively respond to the increasing number of complex daily threats children face. They should be aware of the different types of online threats that children face and have a plan for action to counter them.

Online Safety Risks for Children

Although content moderation teams have the tools and experience to spot potential risks, children are more vulnerable online. Often the team can overlook the more serious risks if we focus only on the content. Trust & Safety teams need to evaluate the entire online environment, from what children are exposed to, their interactions with others and the information they share.

The Four main types of online risks for children are known. Each presents a unique threat, so it is essential to have different detection and response mechanisms. Learn more about the Four C's and how teams can respond.

Content Risks

Children and online audiences face many risks from content. These include exposure to harmful or unsafe content and these include exposure to profanity and sexual content.

Talk to Risks

Contact risks are the possibility of harming children by communicating with threat actors. These actors could include child predators or criminals, terrorists, and adults pretending they are children.

Take Risks

Conduct risks refer to children who engage in physically and emotionally harmful behaviours. These could include bullying, self-harm, encouraging eating disorders, and dangerous viral challenges.

Contract Risks

Contract risks are the possibility that children will agree to terms and contracts they don't agree upon or understand. They could include consenting to inappropriate marketing messages, purchasing an item inadvertently, or giving access to personal information.

How trust and safety can help

Equip Trust & Safety teams with various tools, procedures and processes to overcome children's many online risks.

1. Create strong policies that are focused on children

An extensive policy will help to deter harmful content and other illegal activities. While policies should consider the most shared content risks, their design must also have children in mind. Including policies that govern who can and can't communicate with them, what content they may have access to, and what information and documents they can and can not provide.

It cannot be easy to build a policy. Reactive policies should be able to adapt to changes in the online threat environment and keep up with emerging trends in violations. Child Safety, Second Edition.

2. Establish Subject Matter Expertise

Subject matter expertise is crucial to counter online children's many complex and multilingual risks. Each child predator group, terrorist network, or eating disorder community is unique, and they use specific terminology and exploit different platforms in unique ways to reach a large audience. Experts who are well-versed in the dangers, code words and jargon can identify these threats and help to stop them.

Subject matter experts know, for example, that #Iatepastatonight is suicide content or self-harm. A pizza emoji, however, signals child pornography. These experts are familiar with the hidden codes and chatter within these groups and wherever their location is. 

Unique terminologies and Emojis are one-way predators that communicate with children online to abuse them. Look at this report about the Weaponization of Emojis to see a selection of these emojis.

3. Use on- and offline intelligence

Protecting children and vulnerable users is urgent, and it is essential to act quickly to prevent harm. Trust & Safety teams need to use subject expertise and foresight to prevent children from being harmed.

Teams can gather proactive intelligence from the dark web and the open internet to understand the risks and take action to prevent them.

ActiveFence's Trust & Safety Intelligence solution is available to you.

4. Safety by Design

Safety by design puts safety at the heart of product development to prevent injury before it occurs. Teams protect children from known and unknown dangers by using safety as a guiding principle in product design.

These are just a few examples of product safety measures that increase online child safety:

a. Fundamental age verification tools verify users' age before accessing certain content.

b. Reporting features that allow children and all users to flag dangerous users. Software that allows users to manage the data that platforms access. This software limits the ability of threat actors to exploit loopholes to harm.

c. Our safety-by-design guide will help you better understand safety design fundamentals and their uses.

5. Scale with Contextual Artificial Intelligence

Intelligent AI is required to detect and prevent children from being in danger. Using contextual AI to analyse content can uncover patterns the human eye cannot see. A simple "how are ya" might seem innocent, but AI can add additional information, such as previous violations and the age of users, to give a completely different picture.

Contextual AI also uses many factors to accurately flag content when it comes down to CSAM. Machine learning models and classifiers analyse text, images, logos, and text for nudity, and age, allowing them to flag CSAM and harassment with high precision. Text models can detect harmful text even when it's deliberately misspelt or spaced out as "u gl y"

Please read our AI and human content detection tools blog to learn more about AI content moderation.

Conclusion

Online dangers are increasing, so Trust & Safety teams must improve and scale their efforts to ensure child safety online. Teams can prevent harm by understanding and implementing the best processes and procedures.