This is the twenty-eighth article in a series dedicated to the various aspects of machine learning (ML). Today’s article will extend the discussion of the types of environments an AI agent will need to deal with.
Our last article in the series covered a few of the types of environments that an AI agent may find itself in. These included stochastic vs. determined environments, dynamic vs. static, and fully vs. partially observable environments. What this article will do is continue this discussion by giving an overview of some more environmental characteristics that impact the work of an AI agent.
To do that, let’s continue our example from the last article, about driving through a rainstorm to the hospital to get a nurse to work on time. Let’s assume the driver is actually a self-driving AI car, and the passenger is actually a robot nurse. The car gets to the hospital safely, and the robot nurse gets to work right on time to begin its rounds.
The chief plastic surgeon (not a robot) assigns the nurse to a brand new real-life task that it has only experienced in training: Assisting with high-stakes rhinoplasty.
There’s no time to think—the nurse, whose name is NurseBot 9000, follows the general surgeon into the room. Though it’s NurseBot’s first time doing real-world surgery, it knows a few things about the surgery environment from its training: It is an accessible, sequential, multi-agent, cooperative environment.
What does this mean? Keep on reading to find out.
Accessible vs. Inaccessible
We say the rhinoplasty room’s environment is accessible because the agent can obtain the relevant information at any time: Which tool it needs to hand the surgeon, the state of the patient, etc.
An inaccessible environment, on the other hand, is the opposite: NurseBot would not be able to learn the information relevant to its task, and would therefore be “in the dark” to some extent during the surgery—a scary thought for the plastic surgery patient!
Sequential vs. Episodic
The surgery room is a sequential environment because the NurseBot 9000 needs to know the entire sequence of the surgery to know what’s next—when to hand over a tool, when it is necessary to put the patient under, etc.
Episodic environments require NurseBot 9000 to only know the current state to accomplish its goal.
In surgery, it would be best to train robots to think of the entire sequence of actions to be taken rather than just the current state. Otherwise, it may make some very expensive mistakes, like putting the patient to sleep just because it sees the patient lying awake on the surgery chair, even if the patient just woke up.
Single-Agent vs. Multi-Agent
The rhinoplasty surgery room is multi-agent, because, well, NurseBot isn’t the only agent in the room that has an impact on the goal. The surgeon is probably the most significant “other” agent in this scenario, as the surgeon’s actions has a huge impact on the accomplishing of the goal (i.e., successful surgery).
Single-agent, on the other hand, refers to environments where NurseBot 9000 would be the only one, operating by itself.
Cooperative vs. Competitive
The surgery is cooperative because NurseBot and the surgeon, and the rest of the staff, are pulling together to achieve the same goal of rhinoplasty. This is the nature of cooperative environments.
Competitive environments, like chess matches, involve agents trying to best each other for a personal goal.
Can you imagine competitive surgery? If you can’t, just know that that’s a table you don’t want to be lying on.
Summary
There are many ways to characterize environments, and an AI agent’s knowledge of the “type” of environment it is in can be hugely influential towards how it approaches its goal-accomplishing. Cooperative vs. Competitive, Single-Agent vs. Multi-Agent, Sequential vs. Episodic, and Accessible vs. Inaccessible were environment types covered in this article.
Recent Comments