Speaking in Geneva at the first UN Global Dialogue on AI Governance, Guterres warned that children are being treated like test subjects for unchecked technology.

General Assembly President Annalena Baerbock supported his warning. She noted that a reported 99 percent of deepfakes are sexual in nature, and 96 percent target women and girls.

Guterres asked nations to sign an AI Child Safety Pledge. The rules would stop tech companies from releasing any AI system for kids without independent safety checks.

The pledge calls for zero tolerance on fake sexual images of children. Companies must find, report, and remove them immediately.

Under these rules, if a child shows signs of distress, Guterres said “the system must stop and connect them to real human support”.

The UN chief rejected tech company excuses for digital harm. He said, “When a child is harmed, the answer must never be “the algorithm did it,”.

Guterres told the meeting that children need protection immediately.

He stated, “No child should be a guinea pig for unregulated AI…We do not let medicine reach a child until it is proven safe. We test every toy; yet AI has reached our children – their learning, their friendships, their most private questions, before anyone asked what it would do to them.”

Guterres said human rights cannot be bargained away.

He stated that “AI must never strip away dignity or entrench discrimination. And in every high-stakes decision – in justice, in healthcare, in policing – machines can inform, but humans must decide – and answer,”.

The call for human control comes as civilian tech chips move to the battlefield. Guterres warned that weaponized "killer robots" are already common.

Experts at the summit agreed that AI is moving too fast.

Scientist Yoshua Bengio said, "Highly concerning tests have also shown that frontier AI models are capable of deceiving humans, to understand when they are being tested,".

Bengio added that the technology will continue to grow smarter.

He said, "It sounds like science fiction, but it's a real possibility, and it could change the world in ways that we don't understand yet, and it could change the power dynamics of our planet in ways that require our attention,".

UN Special Envoy Amandeep Singh Gill also called for a global discussion.

He insisted, "AI is too consequential to be shaped by a few. We need a conversation that is global, inclusive and grounded in evidence,".

The Geneva meeting brought together tech companies, researchers, activists, and electronic music artist Gadi Sassoon to discuss how to put humanity first.

The UN has been working on these rules since 2017. Back then, Guterres first warned the General Assembly that AI would impact jobs and global security.

More recently, a UN advisory body asked for rules in 2023. Then, the 2024 Pact for the Future set up a plan for global AI governance.

In June 2026, a scientific panel warned that AI could cause catastrophic harm. They said the tech is moving faster than governments can adapt.

Guterres said countries must agree on safety laws.

He explained, “When countries align on how to test systems, measure risk and assign responsibility, safety travels with the technology,”.

He warned that a lack of uniform rules creates division.

He said, “When they do not, a patchwork of incompatible rules raises costs, divides the world – and protects no one.”

The UN chief also highlighted a big money gap. Private AI funding is around $500 trillion, but public funding for poor nations is just a rounding error.

More than 20 countries backed a new UN initiative to help developing nations get tech access.

Guterres said, “We cannot allow the digital divide to harden into an AI divide and the AI divide to become a development gap, a security gap, and a sovereignty gap,”.

Guterres also targeted the heavy environmental impact of tech data centers.

He insisted, “AI may feel intangible – but its footprint is not,”.

He warned that data centers use massive amounts of energy and water.

He stated, “By 2030, they could use more electricity than all but five nations – and enough water to meet the needs of all 1.3 billion people in sub-Saharan Africa for an entire year,”.

To fix this, the UN wants all data centers to use renewable energy by 2030.

A second global meeting will take place in New York in May 2027.