This is the twenty-seventh article in a series dedicated to the various aspects of machine learning (ML). Today’s article will cover the types of environments an AI agent will need to deal with.
Driving on the highway during a heavy downpour is one of the most stressful things you can do as a driver. Your windshield wipers are flying back and forth like crazy, the rain doesn’t even look like rain but just violent sheets of grey nothingness slapping against your windshield like a biblical curse, and to top it all off you are on the highway, and as per usual there are just enough psychos out on the road who will go ten over the speed limit no matter what the road conditions are.
Let’s up the stakes a bit: You’ve got someone in the car who desperately needs to go to the hospital. It’s not bad, this particular person is on the verge of being late to work and needs to get there in order to not get fired—this is the third instance of tardiness this year, and strike three could mean no more nursing job. Despite the weather you try to maintain a safe speed-limit beeline down the middle lane until you reach the appropriate exit. Upping the stakes a bit more, the drivers on the road are all unpredictable 10-mph-over+ daredevils who are swerving left and right at unpredictable intervals.
It’s quite obvious that your goal has been made much more difficult than usual because of both the weather conditions and the quality of the drivers on the road. You pray to every deity and storied wish-maker secular and otherwise for a little sunshine in the sky and sanity on the road, but unfortunately Providence has decreed your hospital-trip be littered with difficulties.
In moments like this, we realize that sometimes it is not the goal itself but rather the environment in which the agent is in that makes its accomplishing difficult. This principle is mightily important to the development and deployment of AI agents.
AI Agents and Their Environment
The issues in the example that opened this article are the same that trouble AI agents: Limited perception, unpredictable environments, and the like. Not every environment is the same, which is important for an AI agent. Developers need to consider the issues agents will rub up against while trying to accomplish their task.
The vocabulary to characterize environments in the AI world is worth learning, because it can help you understand why it is necessary that agents need to be capable of machine learning. Not all environments can be fully known before entering.
Keep on reading for a breakdown of the types of environments an AI agent may find itself in.
Observability
Environments that can be completely perceived at all times are called “fully observable” environments.
On the other hand, when an agent cannot get a full picture of everything, the environment is characterized as “partially observable.” The highway example fits this description, as there are blind spots drivers have to deal with.
Deterministic vs. Stochastic
In philosophy, “determinism” refers to the idea that world events are predetermined, meaning that free will either doesn’t exist or is limited to some extent. “Indeterminism” is its opposite, which holds that the agents of the world are more or less free and their actions self-caused, a position that the common human probably assumes.
In the AI world, “determinism” is related to this idea, in that the word describes an AI agent’s ability to have an impact on the world. Determined environments are ones where the agent can be the sole cause of the environment’s next state.
“Stochastic” environments are the counterpoint here, referring to environments that are unpredictable, and have elements that can impact its state outside of the influence of the agent. The highway is one such place, where a driver swerving in front of the agent represents a chancy occurrence that the agent had nothing to do with.
Static vs. Dynamic
Static environments are ones that don’t change on its own. The agent enters it, and doesn’t need to worry about something changing, like the weather.
Dynamic environments are ones that agents need to “keep an eye on,” in that they can be subject to change at any moment, like a highway where cars can enter or leave it along the exits.
Note that the difference between dynamic/static and stochastic/deterministic is that the latter is directly concerned with the impact that the agent can have in changing the environment, while the former is focused on how the environment itself changes, or doesn’t, over time.
Summary
AI agents don’t achieve their goals in a vacuum. Rather, they work towards them in an environment. The nature of the environment varies, from predictable to unpredictable, fully observable or not, factors that have a big impact on how an agent achieves its goal.
Recent Comments