Robots and embodied AI

How does turning pixels into objects work? Let's try to understand. In a simple way. A robot's camera hands over a grid of numbers, one for the brightness of each pixel.

Nowhere does that grid say cup. No object is in there at all. A vision network sweeps across the grid, hunting for patterns. Edges, then corners, then shapes.

It learned them from millions of photos where people already drew the boxes. What comes back is a box. Four numbers for the corners, a name like cup, and a confidence score. That step is called detection.

Here is the catch. That box is flat. It lives in the picture, measured in pixels. An arm cannot reach for pixel four hundred and twenty.

So the robot measures depth too. Two cameras compared side by side, or a sensor timing a pulse of light. Now every pixel carries a distance. Depth plus the box gives a point in front of the camera.

But the camera sits at a known offset from the arm. Multiply through it, and the point lands in the robot's own coordinates. Then add orientation. Which way is the handle turned?

Position and orientation make the pose. Six numbers, and that is what a gripper needs. And frames keep arriving, thirty every second. The robot matches each new box to the old one, so a moving cup stays the same cup.

So, pixels go in and a pose comes out. A name, and where it sits in the robot's world. Only then can the arm reach. Quick check now.

One question is coming up. Let's see if it clicked.

How does turning a goal into actions work? Let's try to understand. In a simple way. You say, put the mug in the sink.

That is not an instruction. It is a description of a world you want. The robot invents the steps. So the robot keeps a menu of things it can do.

Move to something. Open the gripper. Close it. Lift.

Place. Each is a whole action. Every action carries two notes. What must be true before it runs, and what changes after.

Closing the gripper needs the hand at the mug. Then the mug is held. Now a plan is just a chain. Each action's before note must already be true, and the last one leaves the world matching the goal.

Order is not a detail. Lift before you close the gripper, and the hand comes up empty. Those before notes are what force the sequence. Finding the chain is a search.

Every legal action opens a branch, each branch opens more, and the tree explodes. Ten steps deep is billions of paths. So the planner does not look everywhere. It scores each branch by how close that state gets to the goal, and walks the promising ones first.

And a plan is only a guess about the future. The mug slips. So the robot checks the world after every step, and replans when reality stops matching. So, a goal is a state, not an order.

Actions carry preconditions and effects. The planner searches for a chain that reaches the goal, then searches again. Quick check now. One question is coming up.

Let's see if it clicked.

How does turning commands into smooth motion work? Let's try to understand. In a simple way. The planner says, move the gripper to the cup.

But a motor cannot take a destination. It takes a current, fresh, every millisecond. Why not just shove the motors at full power? The arm has weight.

It lunges, sails past the cup, swings back. The coffee is gone. So the robot writes a trajectory. Not one point, but hundreds.

Where should the gripper be a millisecond from now? And the millisecond after that? The shape of the schedule is the trick. Speed starts at zero, climbs, cruises, then falls to zero exactly at the cup.

That ramp is what smooth means. The ramp cannot be too steep. Every joint has a top speed and a top acceleration. Even the rate acceleration changes has a name.

Jerk. But a schedule is only a wish. Gravity pulls the arm down. Joints stick with friction.

The real arm always lags behind the plan. Every joint carries an encoder reporting its true angle. Subtract. Where you should be, minus where you are.

That gap is the error, checked a thousand times a second. The error sets the current. Big gap, big push. It also adds up errors that linger, and watches how fast the gap closes, so it stops before overshooting.

Those three fixes have names. Proportional, integral, derivative. So a command is one point. Control turns it into a timed schedule, then chases it with a fast loop of tiny corrections.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is redundancy in a robot arm? Let's try to understand. In a simple way. A robot's goal is a pose.

Where the hand sits, and which way it points. Six numbers, and that is the whole request. But an arm cannot move a hand. It only moves joints.

Seven of them on a modern arm. Seven dials to turn. So seven dials feed into six numbers. One dial more than the goal even asks for.

That spare dial is the whole story. Hold the hand perfectly still. Now swing the elbow up and around. The hand never budged.

The joints kept changing. Nothing broke. So different joint settings put the hand in the exact same place. Elbow high, elbow low, and every posture between them.

And it is not two answers, or ten. That elbow arc is smooth and unbroken. Engineers call this space of free motions the null space. But the robot has to send one command.

So the solver cannot merely solve. It must choose. And choosing needs a reason. That reason is a cost.

Score every candidate. Stay far from joint limits. Keep clear of the shelf. Waste the least effort.

So the spare dial is not a nuisance. It is room to work. Same cup, same grip, elbow tucked around whatever is in the way. So, seven dials, six numbers.

The leftover freedom leaves endless answers. The cost picks the single one worth sending. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is the curse of a robot arm's many joints? Let's try to understand. In a simple way. A robot arm bends at joints.

Shoulder, elbow, wrist. Each joint is one number to choose, an angle. Pick all six and the arm's shape is fixed. The robot's real map is not the room.

It is a space of joint angles, one axis per joint. One point there is a whole arm shape. With two joints you can draw it. A flat square, one axis each.

Moving the arm is walking a line across that square. To search that square, chop each axis into ten steps. Two joints gives one hundred cells. Check them all and you find your path.

Easy. Now add joints. Six joints is a million cells. Twelve joints is a trillion.

Every joint multiplies the space, it does not add to it. That is the curse. And every cell needs a collision check. A table is a plain box in the room.

In joint space its shadow is a strange blob nobody can draw. So planners stop building the grid. They throw random poses into the space, keep the ones that do not collide, and connect nearby ones into a roadmap. Why does that work?

Random points do not care how many axes there are. A few thousand sketch a twelve joint space well enough to cross it. So, one joint is one axis. Grids die as joints multiply.

Random samples survive. You give up the perfect path and take one that works. Quick check now. One question is coming up.

Let's see if it clicked.

How does a robot vacuum's map work? Let's try to understand. In a simple way. Your vacuum wakes in a room nobody described.

No floor plan came in the box, and satellite signals do not reach indoors. So it counts. Each wheel carries an encoder ticking off rotations. Add up the turns and you have a position.

Dead reckoning. But wheels slip. On a rug, a wheel spins while the vacuum sits still. The count says moved.

That error grows, and nothing resets it. Drift. So it does not trust the count alone. It looks.

A spinning laser measures the walls, and the vacuum matches that scan against the map so far. Which is circular, and that is the point. You need the map to place yourself, and your place to extend the map. Simultaneous localization and mapping.

That map is a grid of cells. Each holds a probability. Free, blocked, or never seen. Not a picture of your room.

A belief. Recognize a corner you have already seen, and two spots become one spot. The drifting map snaps back into shape. Loop closure.

So why the bumping? The laser spins in one flat plane, above your sock, below the table top. A black chair leg returns nothing at all. None of it reaches the grid.

So the bumper is the sensor of last resort. The crash is how that cell gets marked. So it bumps because the map is a guess. Counting drifts, matching corrects, and whatever the laser cannot see, the bumper finds.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is the delicate physics of picking up an egg? Let's try to understand. In a simple way. A robot arm already knows how to hit a position.

Its loop drives the error to zero. In empty air, that works beautifully. Then the fingers touch the egg. Now the loop is dangerous.

It still wants the gap it was given, so it pushes harder into the shell. That is stiffness. Press one extra millimeter into something hard, and the force climbs enormously. The loop cannot tell.

It sees an error it has not closed. So force, not position, is the real quantity. And it has to land inside a window. Too little, the egg slips.

Too much, the shell cracks. The floor of that window is friction. The squeeze must beat the weight divided by the friction between shell and finger. Rubber needs far less than steel.

Here is the catch. Before it touches, the robot does not know the egg's weight, or how slippery the shell is. It cannot look the number up. So it feels.

A force sensor in the wrist reports the push. Tactile skin catches the tiny shiver of a shell beginning to slide. Then squeeze a touch more. The deeper fix is compliance.

Instead of a place, the robot commands a springiness. The finger acts like a soft spring, so an extra millimeter is a gentle push, not a crack. So, picking up an egg is a force problem, not a position problem. Aim inside the window, and feel your way in.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a robot's real-time deadline? Let's try to understand. In a simple way. A chess engine can ponder for an hour.

The board waits. A robot gets no such deal. The mug tips while it thinks. Every move is one cycle.

Take a picture, decide, send current to the motors. Each step costs milliseconds, and the world moves through all of them. The picture is old before it arrives. By the time the robot looks at that frame, it describes a world that already changed.

Add it up. Camera, thirty milliseconds. A big model deciding, three hundred. The command, a few more.

Almost a third of a second from light to movement. Meanwhile the joint controller wants a fresh command every millisecond. Miss the slot, and it repeats the last one. A plan that arrives late is not slow.

It is wrong. That is why robots stack loops. A reflex loop, a thousand times a second, tiny math. A thinking loop, a few times a second, picking the goal.

The fast one never waits. The lag can be aimed away. Know the delay, know the ball's speed, roll physics forward, and aim where it will be, not where it was. And the planner always holds a usable answer.

It refines while time allows, then hands it over the instant the clock runs out. Good enough now beats perfect later. So robot thinking is timed work. The picture is stale, the world will not wait, and a late command is noise.

Answer before the buzzer. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is training a robot inside a video game? Let's try to understand. In a simple way. A real robot practices in real time.

One arm. One hour per hour. Its gears wear out, and after every drop a human stands the cup up again. So we build the robot a video game.

A physics engine holds a table, a cup, and a copy of the arm. It is all just numbers. The engine's whole job: take the numbers, apply gravity and the motor forces, work out what touches what, then push time forward five milliseconds. Again and again.

The arm itself is a file. Every link, its mass, how far each joint bends, how hard each motor pushes. Plus shapes for the fake camera. Now the trick.

Those five milliseconds are time inside the game. The computer works them out in far less. An hour of practice takes a minute. And a world this cheap is worth copying.

Run two thousand at once, each with its own arm, all on one graphics card. A year of practice, overnight. Nothing breaks, either. The arm smashes the cup?

Set the numbers back to the start. Instant, and free. A real robot cannot do that. And the engine knows everything.

Where the cup really is. The force on every finger. So the score for each try is exact, and free. So, a robot simulator is a physics engine stepping a made up world.

Time, bodies, and mistakes become nearly free. Only the numbers in it are a guess. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is the gap between simulation and reality? Let's try to understand. In a simple way. In the game the policy is flawless.

Ninety nine cups out of a hundred land on the shelf. On the real arm, it misses. Nothing was faked. But every number in that game was a guess.

The table's friction. The cup's weight. How fast the motor really answers. And the policy did not learn the task.

It learned that engine. It found a grip the fake friction allows. On real wood, that trick is gone. The camera lies too.

Rendered light is clean and even. A real lens brings glare, blur, and shadow. Same cup, different pixels. Then there is what the file never mentioned.

Gear backlash. A cable that tugs. Ten milliseconds before a command reaches the motor. The game called those zero.

And a small error does not stay small. One clumsy move puts the arm somewhere the game never showed. From there its next move is worse. So, the main fix.

Domain randomization. Every practice run, shuffle the numbers. Heavier cup, slicker table, redder light. Now no single trick works.

Randomize wide enough, and reality is just one more draw from that pile. The robot has already practiced in a thousand worlds like it. The other fix is honest measurement. Push the real arm, log what it does, and tune the game's numbers until they match.

So, virtual success fails because the policy learned the simulator's habits. Randomize what you guessed, measure what you can. Quick check now. One question is coming up.

Let's see if it clicked.

How does learning by watching people work? Let's try to understand. In a simple way. Fold this towel.

Now try writing the rules. Where to pinch. How hard to pull. What to do when it snags.

Nobody can. So a person takes the controls and drives the real robot through the task. Slowly, properly, twenty times over. Then chop that recording into moments.

Every moment is a pair. What the camera saw, and the move the human made right then. That move is the answer key. No score.

No reward. The human's hand is simply the right answer for that frame. So training is a quiz. Show it a frame, let it guess a move, compare against the human's.

The gap nudges the weights. This is behavior cloning. But notice what got copied. The motion, not the reason.

It never learned why the human slowed near the edge. Only that they did. And here is the crack. The demos are all good runs.

The human never fumbles. So the robot has never seen a mistake, or a fix. Now it wobbles, just slightly. It is somewhere no demo ever went.

Its next move is a guess, and the error grows. The fix is to let the robot drive, and watch. Every time it wanders off, the human says what they would have done from there. Those corrections go back in.

So, watching people means copying paired moves, with no reward anywhere. Cheap to teach, blind to its own mistakes until you label those too. Quick check now. One question is coming up.

Let's see if it clicked.

How does learning from rewards and punishments work? Let's try to understand. In a simple way. Nobody is driving now.

No human hand to copy, no right move written down. The robot tries the task itself. Something scores it. That score is the reward.

One number, at the end of the attempt. One if the cup reached the shelf. Zero if not. Punishment is not a separate thing.

It is just a negative number. Minus five for smashing the cup. Minus one for a violent swing. So one attempt looks like this.

Two hundred small moves, start to finish. Then one number lands at the end. Here is the hard part. Two hundred moves, one number.

Nothing says which move earned it, or which nearly spoiled it. That is credit assignment. Stop judging single moves. Judge the whole run.

Scored high, nudge every move in it up. Scored low, nudge them down. That is the policy gradient. This only works if the robot varies.

Same move, same score, nothing to compare. So it wobbles its choices on purpose. That is exploration. But the reward arrives only at the end.

Thousands of failures before the first success. So we hand out crumbs on the way. That is reward shaping. And now the trap.

The robot maximizes your number, not your wish. Pay it for hovering near the cup, and it hovers forever, lifting nothing. So, rewards mean no answer key, one score per attempt, whole runs nudged. Write the number carelessly, and the robot obeys it exactly.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is an affordance? Let's try to understand. In a simple way. Your camera already named it.

A mug, right there. But the hand still does not know where to put its fingers. A name is not a place. An affordance is what a thing offers you to do.

The handle offers a grip. The rim offers a pour. An action, glued to one region. And it depends on the body.

A suction cup wants the flat side. Two fingers want the handle. So it belongs to a pair: object and gripper. Nobody types this in.

No file says, grip here. It is learned from thousands of tries: where the fingers closed, and whether the grip held. The answer is a map, not a word. Every spot gets a score.

How well would a grip work here. Bright at the handle, dark at the hot glass. But a point is not enough. The gripper must arrive from some direction, with some opening.

So each spot carries an angle and a width. That is a grasp pose. And it rides on shape, not on names. Show it a watering can it has never seen.

There is a handle shaped part, so it grips there. No name needed. One object can hold several. A knife: the handle to cut with, the blade to hand over.

The task picks which one lights up. So, an affordance is an action attached to a place, scored for your gripper. Not what the thing is. What you can do with it.

Quick check now. One question is coming up. Let's see if it clicked.

How does a language model on a robot work? Let's try to understand. In a simple way. You say, I spilled my drink.

That is not a goal, and it is not code. It is just a sentence. A language model does know what that means. It read the internet.

A spill wants a sponge, then a wipe. Nobody programmed that in. But it has never stood in your kitchen. No camera, no hands.

Ask for a plan and it cheerfully says, grab the vacuum. The robot never lets it write freely. It hands it a menu. Every line is a skill the robot already trained.

Pick up the sponge. Wipe. Now the model only scores lines from that menu. How much does, pick up the sponge, help with, I spilled my drink?

High. An apple? Low. But the model cannot see the room.

So the robot scores those same lines itself. From right here, how likely am I to pull it off? Multiply them. Useful times possible.

The sponge line wins on words, but its odds are near zero with no sponge in reach. So walk to the counter first. That picked one step, not a plan. It runs, the result comes back as text, and the model chooses again.

Why bother? Your sentence never had to become code. Say it another way, say it in another language. Same menu, same robot.

So, the model brings the words and the world knowledge. The robot brings the skills and the honest odds. Neither plans alone. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the sandwich test? Let's try to understand. In a simple way. A five year old makes a sandwich without thinking.

No robot on earth can do it in an unfamiliar kitchen. That gap has a name. Moravec's paradox. Count the steps.

Open the bag, pull a slice, find the jar, twist the lid, scoop, spread. Hundreds of moves in one unbroken chain. Now the cruel part. Say every move works ninety nine times out of a hundred.

Excellent. But two hundred moves in a row multiply to about thirteen percent. Excellent per step is hopeless per sandwich. Then, the bread.

Squeeze it and it stays squeezed. Lettuce folds, mayonnaise flows. None of it has a fixed shape, so there is no pose to aim at. And every kitchen is new.

A different loaf, a stiffer lid, a crowded counter. The world keeps handing it things it never practiced on. Worse, the skills do not snap together. Each one was practiced alone.

Where spreading ends is where lifting must begin, and that exact spot is nobody's practiced start. And when is it done? A sandwich can be lopsided, thin, or overflowing, and still be fine. Done is a judgment, not a line the machine can check.

So, the grand challenge. A sandwich is not one hard trick. It is seeing, planning, touching, and recovering, all at once, for ten unbroken minutes. So the sandwich is a benchmark for generality.

Long, floppy, unrepeatable, and impossible to score. Solve it, and you have solved the kitchen. Quick check now. One question is coming up.

Let's see if it clicked.