Terminology
Important Terms used in Reinforcement Learning:
We will use the following cliff-problem which was mentioned in the introduction-post to describe the important terms which are used throughout the RL course:
- Agent
An agent is just something that acts (agent comes from the Latin agere, to do). Of course, all computer programs do something, but computer agents are expected to do more: operate autonomously, perceive their environment, persist over a prolonged time period, adapt to change, and create and pursue goals. [1]
In the context of RL, 'Agent is the "decision-maker" while solving a sequential decision making problem. It can also be called "learner" i.e. the one who learns to make sequential decisions.
In the cliff problem above, the little elf-like figure is the agent, as it makes a decision at each step regarding the direction in which to move.
A rational agent is one that acts so as to achieve the best outcome or, when there is uncertainty, the best expected outcome.
- Environment
The world with which the agent interacts.
- State (s)
The current situation of the agent/environment.
- Action (a)
The move the agent chooses.
- Reward (r)
Feedback signal for the agent’s action.
- Policy (\( \pi \) )
The strategy the agent uses to decide actions.
- Trajectory
A sequence of states, actions, and rewards.
Interaction Cycle
Loop: Observe state \(s_t\) Choose action (a_t) based on policy π(s_t) Receive reward (r_t) and next state (s_{t+1}) from the environment Update policy to improve future decisions