August 18, 2025

How to Dodge a Ball

(with a robot)

A couple of years ago I published a paper about transporting objects on a tray with a robot, which is known as the waiter's problem. My favourite experiment from that paper was having the robot dodge a volleyball (thrown by me) while keeping a bottle balanced on its tray (a still frame from this experiment is shown below; you can see a video here). This experiment demonstrates that our method is fast enough to react to obstacles moving fairly quickly.

The robot must dodge a thrown ball while balancing the pink bottle.

Getting this experiment to work required more engineering effort than I had room to fully describe in the paper (though the code is open-source and available here), so this blog post is going to explain some of the implementation behind the experiment in more detail.

Model Predictive Control

We use a model predictive controller (MPC) to generate commands for the robot. At each control timestep, the MPC predicts what will happen over a time horizon using a model of the system, and computes the sequence of control commands that will lead to the best outcome over that horizon (in our paper, the time horizon is two seconds long). "Best" is defined by the control designer (me, in this case), and includes objectives that we would like to minimize (like the distance between the robot and a goal location), as well as constraints that must be satisfied (like maintaining a minimum distance from obstacles). The first control command in the optimized sequence is sent to the robot and the process is repeated at the next timestep. The topic of this post is the particular constraint we use to avoid collisions between the robot and a thrown ball.

Projectile Motion

We assume that the ball behaves like a simple projectile after it is thrown, which means that the only force acting on it is gravity. Given an initial position in 3D space $\bm{r}(0)$ and an initial velocity $\bm{v}(0)$ , the position of the ball at time $t$ is

\begin{equation}\label{1} \bm{r}(t) = \bm{r}(0) + t\bm{v}(0) + (1/2)t^2\bm{g}, \end{equation}

where $\bm{g}$ is the acceleration due to gravity.

A Simple Approach for Avoiding the Ball

Suppose the controller is predicting what will happen at some time $\tau$ along its time horizon. The ball has predicted position $\bm{r}(\tau)$ and the tray held by the robot has predicted position $\bm{p}(\tau)$ . The simplest approach is to just enforce the constraint that the distance between these two points is above a certain safety threshold $d$ that ensures the ball and robot do not collide, which gives us something like:

\begin{equation}\label{2} \|\bm{r}(\tau) - \bm{p}(\tau)\| \geq d, \end{equation}

where $\|\cdot\|$ is the usual Euclidean norm.

However, I found that this approach did not work reliably. The robot would often not move until it was too late to avoid a collision, as if it initially thought that it wouldn't be hit by the ball (so it doesn't move), and only realizes its mistake at the last moment. I suspect that this occurred for two reasons. The first is that the MPC discretizes its time horizon, so it only checks for collisions at fixed time intervals, and could miss possible collisions that would occur between time intervals. The second is that the robot model is imperfect, so the controller thinks it can move out of the way of the ball more rapidly and precisely than it can in reality.

Essentially, the system was not aggressive enough in getting out of the way. As a person, if someone throws something at me that I want to avoid, I typically err on the side of caution and get well out of the way (if I can), rather than move just enough to barely avoid the projectile.

A More Reliable Approach

To make the robot avoid the ball more aggressively, I decided to constrain it to avoid the entire future path of the ball, rather than just the ball's particular position at each instant in time. That is, if the ball was just released and is currently far away, but the controller predicts that the ball will hit the robot eventually, then the robot will still start getting out of the way immediately (even when the collision is farther away than the time horizon of the MPC). This approach can be thought of as a sort of continuous collision avoidance with (the future path of) the ball.

This approach is a bit more complicated to implement than $\eqref{2}$ . At each time $\tau$ along the MPC's time horizon, we need to ensure that the tray's position $\bm{p}(\tau)$ is at least a distance $d$ away from the entire future path of the ball. This constraint is obviously satisfied if we can ensure that the closest distance between the tray and the ball's path is at least $d$ . To find the time at which the two are closest, we have to find the minimum value of the function

\begin{equation*} q(t) = \| \bm{r}(\tau+t) - \bm{p}(\tau) \|^2, \end{equation*}

which is a quartic polynomial representing the squared distance between the tray's position at time $\tau$ and the ball's future position after additional time $t$ . To solve it, we take the derivative and set it equal to zero (the first-order condition for optimality). After substituting in $\eqref{1}$ , this gives us the cubic equation

\begin{equation}\label{3} \frac{d}{dt}q(t) = at^3 + bt^2 + ct + d = 0, \end{equation}

where

\begin{align*} a &= \|\bm{g}\|^2, & c &= 2(\|\bm{v}(\tau)\|^2 + \Delta\bm{r}^T\bm{g}), \\ b &= 3\bm{v}(\tau)^T\bm{g}, & d &= 2\Delta\bm{r}^T\bm{v}(\tau), \end{align*}

with $\Delta\bm{r}=\bm{r}(\tau)-\bm{p}(\tau)$ .

I used the Newton-Raphson method to find a time $t^\star$ satisfying $\eqref{3}$ . Note that if $t^\star<0$ , then we set $t^\star=0$ , because we don't care about collisions with the past path of the ball. This gives us the closest future position of the ball $\bm{r}(\tau+t^\star)$ (technically, this could actually be any local optimum; see below), so we replace the constraint $\eqref{2}$ with the new constraint

\begin{equation*} \|\bm{r}(\tau+t^\star) - \bm{p}(\tau)\| \geq d. \end{equation*}

Essentially, compared to the original constraint $\eqref{2}$ , we've just had to add some iterations of the Newton-Raphson method to compute the closest future ball position, rather than simply use its predicted position at time $\tau$ .

Local Optima

In general, $q(t)$ is non-convex, with two local minimum and one local maximum (the function opens upward because $a>0$ ). Any of these optima satisfy $\eqref{3}$ , so Newton-Raphson could converge to any of them, not just the desired global minimum. However, I found that the approach worked fairly well despite the non-convexity. One reason may be that since MPC optimizes over a horizon, even if bad local optima are found at some timesteps along the horizon, the correct minimum will be found at others, and this will be sufficient to pull the whole trajectory away from collision.

In hindsight, one possible method to avoid local optima would be to solve $\eqref{3}$ analytically using the cubic formula, which would provide all of the solutions rather than only a single one. It would then be easy to pick the one at the true minimum of $q(t)$ .