Gridworld
An agent, a grid,
a goal.
The animation in this site's hero, made real. A little game of trial and error: an agent that starts out knowing nothing about the grid it lives in. Press Train and watch it go from random walking to the shortest path.
- Episode
- 0
- Steps
- 0
- Best
- ·
- Epsilon
- 1.00
Episode length · last 100
No episodes finished yet ·
How it works
Every cell of the grid is a state, and the agent has four actions: up, down, left, right. Each step costs a reward of -1 and reaching the goal pays +10, so shorter paths simply earn more. The agent explores with an epsilon-greedy policy: with probability epsilon it moves at random, otherwise it takes the best action it currently knows. Epsilon decays over time, so early wandering gradually gives way to confident, learned behaviour.
After every single step, one value in a table is updated:
Q(s,a) += alpha * (r + gamma * max Q(s',a') - Q(s,a))
That is the whole algorithm. No neural networks, no libraries, just a lookup table and one update rule applied thousands of times.
Tabular Q-learning is the simplest case of reinforcement learning: a lookup table and one update rule. That simplicity is exactly what makes the learning easy to watch.