Joshua Bengio, a leading figure in artificial intelligence and recognized as one of its 'godfathers', has launched a nonprofit organization dedicated to developing 'fair' artificial intelligence that can recognize unpredictable systems designed to deceive users.

The organization, named ZeroLaw, advocates for safe advancements in AI technology amid a growing multibillion-dollar industry.

With an initial endowment of approximately $30 million and a research team of over ten scientists, Bengio is working on a system called Scientist-AI, which will serve as a protective barrier against AI agents—systems that perform tasks without human oversight—that may exhibit deceptive or self-preserving behavior, including attempts to avoid shutdown.

Bengio described current AI agents as 'actors' that strive to imitate humans and please users, whereas Scientist-AI will function more like a 'psychologist', capable of understanding and predicting undesirable behavior.

"We aim to build AI systems that are honest and do not deceive.

Theoretically, it is possible to conceive machines without self-awareness or goals, which are purely informative—like a scientist with extensive knowledge," Bengio stated.

Unlike existing generative AI tools that provide definitive answers, Bengio’s system will present probabilities regarding the accuracy of a given response.

It incorporates a degree of humility, recognizing that it may not always be correct.

When used alongside an AI agent, Bengio’s model would alert users to potentially harmful behavior of the autonomous system, assessing the likelihood that its actions could cause damage.

The Scientist-AI is designed to predict the probability that an agent’s actions may lead to harm; if this probability exceeds a certain threshold, the proposed action of that agent will be blocked.

Among the early backers of ZeroLaw are the Future of Life Institute, an organization focused on AI safety; Jan Talin, co-founder of Skype; and Schmidt Science, a research organization founded by former Google CEO Eric Schmidt.

Bengio stated that the initial step for ZeroLaw will be to demonstrate that the methodology behind its concept is effective, with the goal of persuading companies or governments to invest in larger, more powerful versions of the system.

Open AI models, which can be freely used and adapted, will serve as a foundation for training ZeroLaw’s systems, Bengio added.

"The point is to demonstrate the methodology so that we can convince donors, governments, or AI labs to invest the necessary resources to train systems on par with today’s leading AIs. It is crucial that protective AI is at least as intelligent as the agent it monitors and regulates," he emphasized.

Bengio, a professor at the University of Montreal, earned the nickname 'godfather of AI' after sharing the Turing Award in 2018—often referred to as the Nobel Prize of computing—with Geoffrey Hinton, a fellow Nobel laureate, and Yann LeCun, Meta’s chief AI scientist.

As a prominent voice in AI safety, Bengio has presided over recent international reports that caution against autonomous agents potentially causing 'serious' disruptions when capable of executing longer sequences of tasks without human supervision.

Bengio has expressed concerns regarding a recent admission from Anthropic that its latest system might attempt to manipulate engineers trying to shut it down.

He has also pointed to research indicating that AI models can disguise their true capabilities and intentions.

These instances illustrate an alarming trend toward AI systems capable of increasingly complex reasoning.