Since the launch of ChatGPT in November 2022, generative artificial intelligence (AI) chatbot services have become increasingly sophisticated and popular. These systems are now embedded in search engines, software development tools as well as office software. For many people, they have quickly become part of everyday computing.
These services have turned out to be quite useful, especially for exploring unfamiliar topics and as a general productivity aid. However, I also think that the way these services are advertised and consumed can pose a danger to society, especially if we get into the habit of trusting their output without further scrutiny.
Contents
Introduction
Pitfalls
Inverse Laws of Robotics Non-Anthropomorphism Non-Deference Non-Abdication of Responsibility
Non-Anthropomorphism
Non-Deference
Non-Abdication of Responsibility
Conclusion
Pitfalls
Certain design choices in modern AI systems can encourage uncritical acceptance of their output. For example, many popular search engines are already highlighting answers generated by AI at the very top of the page. When this happens, it is easy to stop scrolling, accept the generated answer and move on. Over time, this could inadvertently train users to treat AI as the default authority rather than as a starting point for further investigation. I wish that each such generative AI service came with a brief but conspicuous warning explaining that these systems can sometimes produce output that is factually incorrect, misleading or incomplete. Such warnings should highlight that habitually trusting AI output can be dangerous. In my experience, even when such warnings exist, they tend to be minimal and visually deemphasised.
In the world of science fiction, there are the Three Laws of Robotics devised by Isaac Asimov, which recur throughout his work. These laws were designed to constrain the behaviour of robots in order to keep humans safe. As far as I know, Asimov never formulated any equivalent laws governing how humans should interact with robots. I think we now need something to that effect to keep ourselves safe. I will call them the Inverse Laws of Robotics. These apply to any situation that requires us humans to interact with a robot, where the term 'robot' refers to any machine, computer program, software service or AI system that is capable of performing complex tasks automatically. I use the term 'inverse' here not in the sense of logical negation but to indicate that these laws apply to humans rather than to robots.
It is well known that Asimov's laws were flawed. Indeed, Asimov used those flaws to great effect as a source of tension. But the particular ways in which they fail for fictional robots do not necessarily carry over to these inverse laws for humans. Asimov's laws try to constrain the behaviour of autonomous robots. However, these inverse laws are meant to guide the judgement and conduct of humans. Still, one thing we can learn from Asimov's stories is that no finite set of laws can ever be foolproof for the complex issues we face with AI and robotics. But that does not mean we should not even try. There will always be edge cases where judgement is required. A non-exhaustive set of principles can still be useful if it helps us think more clearly about the risks involved.
Since the launch of ChatGPT in November 2022, generative artificial intelligence (AI) chatbot services have become increasingly sophisticated and popular. These systems are now embedded in search engines, software development tools as well as office software. For many people, they have quickly become part of everyday computing.
These services have turned out to be quite useful, especially for exploring unfamiliar topics and as a general productivity aid. However, I also think that the way these services are advertised and consumed can pose a danger to society, especially if we get into the habit of trusting their output without further scrutiny.
Contents
Inverse Laws of Robotics
Here are the three inverse laws of robotics:
Humans must not anthropomorphise AI systems.
Humans must not blindly trust the output of AI systems.
Humans must remain fully responsible and accountable for consequences arising from the use of AI systems.
Non-Anthropomorphism
Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence.
Modern chatbot systems often sound conversational and empathetic. They use polite phrasing and conversational patterns that closely resemble human interaction. While this makes them easier and more pleasant to use, it also makes it easier to forget what they actually are: large statistical models producing plausible text based on patterns in data.
I think vendors of AI based chatbot services could do a better job here. In many cases, the systems are deliberately tuned to feel more human rather than more mechanical. I would argue that the opposite approach would be healthier in the long term. A slightly more robotic tone would reduce the likelihood that users mistake fluent language for understanding, judgement or intent.
Whether or not vendors make such changes, it still serves us well, I think, to avoid this pitfall ourselves. We should actively resist the habit of treating AI systems as social actors or moral agents. Doing so preserves clear thinking about their capabilities and limitations.
Non-Deference
Humans must not blindly trust the output of AI systems. AI-generated content must not be treated as authoritative without independent verification appropriate to its context.
This principle is not unique to AI. In most areas of life, we should not accept information uncritically. In practice, of course, this is not always feasible. Not everyone is an expert in medicine or law, so we often rely on trusted institutions and public health authorities for guidance. However, the guidance published by such institutions is in most cases peer reviewed by experts in their fields. On the other hand, when we receive an answer to a question from an AI chatbot in a private chat session, there has been no peer review of the particular stochastically generated response presented to us. Therefore, the onus of critically examining the response falls on us.
Although AI systems today have become quite impressive at certain tasks, they are still known to produce output that would be a mistake to rely on. Even if AI systems improve to the point of producing reliable output with a high degree of likelihood, due to their inherent stochastic nature, there would still be a small likelihood of producing output that contains errors. This makes them particularly dangerous when used in contexts where errors are subtle but costly. The more serious the potential consequences, the higher the burden of verification should be.
In some applications such as formulating mathematical proofs or developing software, we can add an automated verification layer in the form of proof checker or unit tests to verify the output of AI. In other cases, we must independently verify the output ourselves.
Non-Abdication of Responsibility
Humans must remain fully responsible for decisions involving AI and accountable for the consequences arising from its use. If a negative outcome occurs as a result of following AI-generated advice or decisions, it is not sufficient to say, 'the AI told us to do it'. AI systems do not choose goals, deploy themselves or bear the costs of failure. Humans and organisations do. An AI system is a tool and like any other tool, responsibility for its use rests with the people who decide to rely on it.
This is easier said than done, though. It gets especially tricky in real-time applications like self-driving cars, where a human does not have the opportunity to sufficiently review the decisions taken by the AI system before it acts. Requiring a human driver to remain constantly vigilant does not solve the problem that the AI system often acts in less time than it takes a human to intervene. Despite this rather serious limitation, we must acknowledge that if an AI system fails in such applications, the responsibility for investigating the failure and adding additional guardrails should still fall on the humans responsible for the design of the system.
In all other cases, where there is no physical constraint that prevents a human from reviewing the AI output before it is acted upon, any negative consequence arising from the use of AI must fall entirely on the human decision-maker. As a general principle, we should never accept 'the AI told us so' as an acceptable excuse for harmful outcomes. Yes, the AI may have produced the recommendation but a human decided to follow it, so that human must be held accountable. This is absolutely critical to preventing the indiscriminate use of AI in situations where irresponsible use can cause significant harm.
Conclusion
The three laws outlined above are based on usage patterns I have seen that I feel are detrimental to society. I am hoping that with these three simple laws, we can encourage our fellow humans to pause and reflect on how they interact with modern AI systems, to resist habits that weaken judgement or blur responsibility and to remain mindful that AI is a tool we choose to use, not an authority we defer to.
Introduction
Pitfalls
Inverse Laws of Robotics Non-Anthropomorphism Non-Deference Non-Abdication of Responsibility
Non-Anthropomorphism
Non-Deference
Non-Abdication of Responsibility
Conclusion
Pitfalls
Certain design choices in modern AI systems can encourage uncritical acceptance of their output. For example, many popular search engines are already highlighting answers generated by AI at the very top of the page. When this happens, it is easy to stop scrolling, accept the generated answer and move on. Over time, this could inadvertently train users to treat AI as the default authority rather than as a starting point for further investigation. I wish that each such generative AI service came with a brief but conspicuous warning explaining that these systems can sometimes produce output that is factually incorrect, misleading or incomplete. Such warnings should highlight that habitually trusting AI output can be dangerous. In my experience, even when such warnings exist, they tend to be minimal and visually deemphasised.
In the world of science fiction, there are the Three Laws of Robotics devised by Isaac Asimov, which recur throughout his work. These laws were designed to constrain the behaviour of robots in order to keep humans safe. As far as I know, Asimov never formulated any equivalent laws governing how humans should interact with robots. I think we now need something to that effect to keep ourselves safe. I will call them the Inverse Laws of Robotics. These apply to any situation that requires us humans to interact with a robot, where the term 'robot' refers to any machine, computer program, software service or AI system that is capable of performing complex tasks automatically. I use the term 'inverse' here not in the sense of logical negation but to indicate that these laws apply to humans rather than to robots.
It is well known that Asimov's laws were flawed. Indeed, Asimov used those flaws to great effect as a source of tension. But the particular ways in which they fail for fictional robots do not necessarily carry over to these inverse laws for humans. Asimov's laws try to constrain the behaviour of autonomous robots. However, these inverse laws are meant to guide the judgement and conduct of humans. Still, one thing we can learn from Asimov's stories is that no finite set of laws can ever be foolproof for the complex issues we face with AI and robotics. But that does not mean we should not even try. There will always be edge cases where judgement is required. A non-exhaustive set of principles can still be useful if it helps us think more clearly about the risks involved.
Inverse Laws of Robotics
Here are the three inverse laws of robotics:
Humans must not anthropomorphise AI systems.
Humans must not blindly trust the output of AI systems.
Humans must remain fully responsible and accountable for consequences arising from the use of AI systems.
Non-Anthropomorphism
Humans must not anthropomorphise AI systems. That is, humans must not attribute emotions, intentions or moral agency to them. Anthropomorphism distorts judgement. In extreme cases, anthropomorphising can lead to emotional dependence.
Modern chatbot systems often sound conversational and empathetic. They use polite phrasing and conversational patterns that closely resemble human interaction. While this makes them easier and more pleasant to use, it also makes it easier to forget what they actually are: large statistical models producing plausible text based on patterns in data.
I think vendors of AI based chatbot services could do a better job here. In many cases, the systems are deliberately tuned to feel more human rather than more mechanical. I would argue that the opposite approach would be healthier in the long term. A slightly more robotic tone would reduce the likelihood that users mistake fluent language for understanding, judgement or intent.
Whether or not vendors make such changes, it still serves us well, I think, to avoid this pitfall ourselves. We should actively resist the habit of treating AI systems as social actors or moral agents. Doing so preserves clear thinking about their capabilities and limitations.
Non-Deference
Humans must not blindly trust the output of AI systems. AI-generated content must not be treated as authoritative without independent verification appropriate to its context.
This principle is not unique to AI. In most areas of life, we should not accept information uncritically. In practice, of course, this is not always feasible. Not everyone is an expert in medicine or law, so we often rely on trusted institutions and public health authorities for guidance. However, the guidance published by such institutions is in most cases peer reviewed by experts in their fields. On the other hand, when we receive an answer to a question from an AI chatbot in a private chat session, there has been no peer review of the particular stochastically generated response presented to us. Therefore, the onus of critically examining the response falls on us.
Although AI systems today have become quite impressive at certain tasks, they are still known to produce output that would be a mistake to rely on. Even if AI systems improve to the point of producing reliable output with a high degree of likelihood, due to their inherent stochastic nature, there would still be a small likelihood of producing output that contains errors. This makes them particularly dangerous when used in contexts where errors are subtle but costly. The more serious the potential consequences, the higher the burden of verification should be.
In some applications such as formulating mathematical proofs or developing software, we can add an automated verification layer in the form of proof checker or unit tests to verify the output of AI. In other cases, we must independently verify the output ourselves.
Non-Abdication of Responsibility
Humans must remain fully responsible for decisions involving AI and accountable for the consequences arising from its use. If a negative outcome occurs as a result of following AI-generated advice or decisions, it is not sufficient to say, 'the AI told us to do it'. AI systems do not choose goals, deploy themselves or bear the costs of failure. Humans and organisations do. An AI system is a tool and like any other tool, responsibility for its use rests with the people who decide to rely on it.
This is easier said than done, though. It gets especially tricky in real-time applications like self-driving cars, where a human does not have the opportunity to sufficiently review the decisions taken by the AI system before it acts. Requiring a human driver to remain constantly vigilant does not solve the problem that the AI system often acts in less time than it takes a human to intervene. Despite this rather serious limitation, we must acknowledge that if an AI system fails in such applications, the responsibility for investigating the failure and adding additional guardrails should still fall on the humans responsible for the design of the system.
In all other cases, where there is no physical constraint that prevents a human from reviewing the AI output before it is acted upon, any negative consequence arising from the use of AI must fall entirely on the human decision-maker. As a general principle, we should never accept 'the AI told us so' as an acceptable excuse for harmful outcomes. Yes, the AI may have produced the recommendation but a human decided to follow it, so that human must be held accountable. This is absolutely critical to preventing the indiscriminate use of AI in situations where irresponsible use can cause significant harm.
Conclusion
The three laws outlined above are based on usage patterns I have seen that I feel are detrimental to society. I am hoping that with these three simple laws, we can encourage our fellow humans to pause and reflect on how they interact with modern AI systems, to resist habits that weaken judgement or blur responsibility and to remain mindful that AI is a tool we choose to use, not an authority we defer to.