Hands-On AI for High School, grades 9 to 12

Grid World

Dr. Ali Al Bataineh

Higher Colleges of Technology · aalbataineh@hct.ac.ae

Teach a robot to find the goal

The red dot in the top-left corner is a robot that knows nothing. It can move up, down, left or right, but it does not know where the goal is or that the pits are bad. It has to work that out by trying moves and remembering what happened.

Reaching the green goal earns +10 points, falling into a pink pit costs 10 points, and every step costs 0.1 points, so wandering around is not a good plan. Click "Train 100 Episodes" and watch it learn, then change the settings and see what happens.

Ready. Hit "Train 100 Episodes" to start, or click "Edit World" to design your own maze.

Click a cell to cycle it: empty → wall → pit → goal → start → empty. To move the goal or the start, keep clicking a new cell until it becomes the goal or the start.

robot

goal (+10)

pit (−10)

wall

value (bright = good)

Controls

Parameters

Epsilon (exploration) 0.20

Gamma (future-thinking) 0.90

Speed Fast

Rewards

Goal reward +10

Pit penalty −10

Step cost −0.10

Statistics

Episode0

Total steps0

Last rewardn/a

Avg reward (10)n/a

Success raten/a

Missions

Mission 01 Train 100 episodes. Watch the arrows appear.

Mission 02 Get the success rate above 80%.

Mission 03 Set epsilon to 0 and reset. Train again. What happens?

Mission 04 Use Edit World to make your own maze, then train.

Mission 05 Make the step cost 0. See if the robot wanders.

Recent Episodes

waiting for training...

Experiments to try

1. The greedy robot

Reset everything. Set epsilon to 0. Train 100 episodes. The robot never explores on purpose. It always picks the move that currently looks best, choosing at random only when several moves look equally good.

Predict: Will it find the goal? Or get stuck doing the same wrong thing forever?

2. The chaotic robot

Reset, then push epsilon all the way to 1.00. Train. Now the robot ignores what it learned and just moves randomly every time.

Predict: Why does the success rate go down even after a lot of training?

3. Short-sighted vs. far-seeing

Try gamma = 0.10. The robot only cares about what's about to happen, not the goal far away. Then try gamma = 0.99.

Predict: Which one ends up with brighter green cells far from the goal?

4. The lava floor

Drag the step cost slider to its most negative value. Now every step really hurts. The robot wants to escape as fast as possible.

Predict: Will the path it learns be shorter than before?

5. Free wandering

Set step cost to 0. Now there's no penalty for taking forever. Reset and train.

Predict: Will the arrows still point a sensible direction, or will they look messy?

6. Build a trap

Switch to Edit World. Surround the goal with pits, leaving only one safe square next to it. Switch back and train.

Predict: Can it still solve it? How long does it take?

7. Two robots, same world

Open the page in two browser tabs. Use the same maze in both. In tab one, set epsilon high. In tab two, set epsilon low. Train both for 100 episodes.

Predict: Which tab learns the maze faster? Does the answer change if you make a really hard maze?

8. The huge prize

Crank the goal reward up to +50. Reset and train.

Predict: Does it learn faster, slower, or about the same? Why?

Words you'll hear

Agent: The robot. The thing that's learning. In real life, this could be a self-driving car, a video game character, or a chess program.
Environment: The grid. Everything the agent moves around in.
Reward: A number the environment gives back after every move. Positive means good, negative means bad. The agent's only goal is to collect as much total reward as it can.
Episode: One full attempt, from the start square until the agent reaches the goal, falls in a pit, or runs out of time (100 steps).
Policy: The agent's strategy. After training, the arrows on each cell show its policy: "if I'm here, this is the direction I should go."
Q-value: How good a move looks, based on what's happened so far. Each cell has four Q-values, one for each direction. The brightness of the cell is the best of those four.
Epsilon (ε): Chance of moving randomly instead of doing what the agent thinks is best. High epsilon = lots of exploring. Low epsilon = stick with what you know.
Gamma (γ): How much the agent cares about future rewards versus right-now rewards. Low gamma = lives in the moment. High gamma = thinks ahead.
Alpha (α): How fast the agent updates its mind. Locked at 0.1 here. Bigger alpha = changes its strategy faster (sometimes too fast).

Q-learning · γ = 0.90 · α = 0.1 Dr. Ali Al Bataineh · Higher Colleges of Technology