Responsibility and Safety
Exploring the Promise and Perils of a Future with More Powerful AI
Imagine a future where we regularly interact with a variety of advanced AI assistants. And imagine a future where millions of assistants interact with each other on our behalf. These experiences and interactions could soon become part of our everyday reality.
General-purpose base models are paving the way for increasingly advanced AI assistants that can plan and perform a wide range of actions based on human goals, adding enormous value to people’s lives and society, acting as creative partners, research analysts, educational tutors, life planners, and more.
It can also bring about a new dimension in AI-human interaction, which is why it is so important to think ahead about what this world will look like and to make responsible decisions and beneficial outcomes in advance.
Our new paper is the first systematic discussion of the ethical and social issues that advanced AI assistants raise for users, developers, and the societies they serve, and provides important new insights into the potential impacts of this technology.
We cover topics such as value alignment, safety and misuse, economics, environment, information domain, accessibility and opportunity impacts.
This is the result of one of the largest ethical foresight projects we have undertaken to date. We brought together a diverse group of experts to explore and map the new technological and moral landscapes in which AI assistants will live, and to characterize the opportunities and risks that society may face. Here we briefly outline some of the key points.
It has a huge impact on users and society.
Examples show the potential for AI assistants to impact research, education, creative work, and planning.
Advanced AI assistants could have a profound impact on users and society, and could be integrated into most aspects of people’s lives. For example, people could ask them to book vacations, manage social time, and perform other life tasks. When deployed at scale, AI assistants could impact the way people approach work, education, creative projects, hobbies, and social interactions.
Over time, AI assistants may even influence the goals and personal development paths people pursue through the information and advice they provide and the actions they take. Ultimately, this raises important questions about how people interact with this technology and how it can best support their goals and aspirations.
Human alignment is essential
An illustration showing that AI assistants must be able to understand human preferences and values.
AI assistants are likely to have a significant degree of autonomy in planning and executing task sequences across a variety of domains. This presents new challenges for AI assistants with regard to safety, alignment, and misuse.
As autonomy increases, the risk of accidents due to unclear or misinterpreted instructions increases, and the risk that the assistant will take actions that are inconsistent with the user’s values and interests also increases.
More autonomous AI assistants could also enable more impactful forms of misuse, such as spreading misinformation or engaging in cyberattacks. To address these potential risks, we argue that we need to set limits on this technology, and that the values of advanced AI assistants should be more aligned with human values and compatible with broader societal ideals and norms.
Communicate in natural language
This is an illustration depicting a person and an AI assistant communicating like a person.
The written output and speech of advanced AI assistants that can communicate flexibly using natural language can be difficult to distinguish from human speech.
These developments raise complex questions about trust, privacy, anthropomorphism, and the appropriate human relationship with AI. How can we ensure that users can trust and control their interactions with AI assistants? How can we ensure that users are not unfairly influenced or misled over time?
To address these risks, we must put in place safeguards such as privacy. The key is that the relationship between a person and an AI assistant preserves the user’s autonomy, supports their ability to thrive, and does not rely on emotional or material dependence.
Collaborates and adapts to human preferences.
This illustration shows how interactions between AI assistants and people create various network effects.
As this technology becomes widespread and deployed at scale, advanced AI assistants will need to interact with both users and non-users. To avoid the collective action problem, these assistants must be able to successfully collaborate.
For example, thousands of assistants could attempt to book the same service for a user at the same time, potentially crashing the system. In an ideal scenario, these AI assistants would instead work on behalf of human users and relevant service providers, discovering commonalities that better meet the preferences and needs of different people.
Given how useful this technology can be, it’s also important that no one is left out. AI assistants need to be widely accessible and designed with the needs of a wide range of users and non-users in mind.
More evaluation and foresight is needed
This is an example of how important different levels of evaluation are in understanding AI assistants.
AI assistants can demonstrate new capabilities and use tools in new, unpredictable ways, making it difficult to anticipate the risks associated with their deployment. Managing these risks requires engaging in predictive practices based on comprehensive testing and evaluation.
Previous research on assessing the social and ethical risks of generative AI has identified several gaps in existing model evaluation methods, and we hope to see more research in this area.
For example, a comprehensive assessment that addresses both the effects of human-computer interaction and the broader effects on society could help researchers understand how AI assistants interact with users, non-users, and society as part of a broader network. In turn, these insights could inform better mitigation and responsible decision-making.
Building the future we want
We may be on the cusp of a new era of technological and social change inspired by the development of advanced AI assistants. The choices we make as researchers, developers, policymakers, and the public will guide how this technology is developed and deployed across society.
We hope this paper serves as a springboard for broader coordination and collaboration to collectively shape the beneficial AI assistants we all want to see in the world.