Quadratically-Constrained Quadratic Programs

A general QCQP is an optimization problem with the form

where

is a quadratic function with $\bm{Q}_i=\bm{Q}_i^T\in\mathbb{R}^{n\times n}$, $\bm{p}_i\in\mathbb{R}^n$, and $r_i\in\mathbb{R}$ for all $i=0,\dots,m$. The form shown in the second line of $\eqref{2}$ is called the homogeneous form of $f_i$. If $\bm{Q}_i\succcurlyeq\bm{0}$ (that is, $\bm{Q}_i$ is positive semidefinite), then $f_i$ is convex. If $f_i$ is convex for all $i=0,\dots,m$, then $\eqref{1}$ is a convex problem; however, we are interested in the general case when we need not make such convexity assumptions. Note that quadratic equality constraints (which are always non-convex if $\bm{Q}_i\neq\bm{0}$) can be represented in $\eqref{1}$ as two-sided inequalities.

Many interesting problems in robotics can be expressed as QCQPs. Notably, the set of valid 3D rotations can be defined using non-convex quadratic constraints, allowing various state estimation problems to be cast as QCQPs. See e.g. (Holmes & Barfoot, 2023) for more information and examples.

Dual Problem

Let’s derive the (Lagrangian) dual problem of $\eqref{1}$, roughly following Section 3.2 of (Park & Boyd, 2017). (We will refer to $\eqref{1}$ as the primal problem going forward.) The Lagrangian of $\eqref{1}$ is

where

with dual variable $\bm{\lambda}\geq\bm{0}$ (the inequality is taken elementwise). The dual function is

that is, it is the greatest lower bound on $\mathcal{L}$. There are two possibilities: either $\mathcal{L}$ is unbounded below or it has a finite minimum, depending on the value of $\bm{\lambda}$. A finite minimum is clearly the greater lower bound, which is obtained when $\bm{\lambda}$ satisfies $\tilde{\bm{Q}}(\bm{\lambda})\succcurlyeq\bm{0}$ and $\tilde{\bm{p}}(\bm{\lambda})\in\mathcal{C}(\tilde{\bm{Q}}(\bm{\lambda}))$ (i.e., $\tilde{\bm{p}}$ lives in the column space of $\tilde{\bm{Q}}$). In this case, a point $\tilde{\bm{x}}\in\mathbb{R}^n$ is a minimizer of $\mathcal{L}$ if and only if

which we can rearrange to obtain $\tilde{\bm{x}}=-\tilde{\bm{Q}}(\bm{\lambda})^+\tilde{\bm{p}}(\bm{\lambda})$, where $(\cdot)^+$ denotes the Moore-Penrose pseudoinverse. Note that if $\tilde{\bm{Q}}(\bm{\lambda})$ is strictly positive definite, then $\tilde{\bm{Q}}(\bm{\lambda})^+=\tilde{\bm{Q}}(\bm{\lambda})^{-1}$ and $\tilde{\bm{p}}(\bm{\lambda})\in\mathcal{C}(\tilde{\bm{Q}}(\bm{\lambda}))$ is always true.

Substituting the expression for $\tilde{\bm{x}}$ back into $\mathcal{L}$, we obtain the dual function

The dual problem is

The dual problem is always convex, but the convexity of $\eqref{4}$ is not obvious in this form. To rewrite the problem, first notice that maximizing $d(\bm{\lambda})$ is equivalent to maximizing a scalar $\gamma$ with the constraint $\gamma\leq d(\bm{\lambda})$, so we can express the dual problem as

Next, recall that the Schur complement theorem tells us that

which finally allows us to rearrange the dual problem $\eqref{4}$ into the (convex) semidefinite program (SDP)

which can be solved efficiently with software like Clarabel or MOSEK. Note that if any of the constraints in $\eqref{1}$ is an equality rather than an inequality, then the dual remains the same except that the corresponding dual variable $\lambda_i$ is no longer constrained to be non-negative.

QC1QP

A QCQP with only a single constraint is known as a QC1QP. We can write an inequality-constrained QC1QP in the form

Making use of $\eqref{5}$, its dual is

Strong Duality

The duality gap is the difference $p^\star-d^\star$ between the primal and dual optimal values. When the duality gap is exactly zero (i.e., $d^\star=p^\star$), we say that strong duality holds. In this section we review some background on strong duality.

Feasibility and Slater’s Condition

Recall that we say that an optimization problem is feasible when there exists some point $\tilde{\bm{x}}$ satisfying all the constraints; such a point $\tilde{\bm{x}}$ is called a feasible point. A problem is strictly feasible (also known as Slater’s condition) when a solution exists in the interior of the feasible set. For example, the problem $\eqref{1}$ is feasible if and only if there exists some point $\tilde{\bm{x}}\in\mathbb{R}^n$ such that

and is strictly feasible if and only if there exists such a point satisfying

(that is, all of the constraints are satisfied with strict inequality).

Weak Duality

Notice that any feasible pair of primal variable $\bm{x}$ and dual variable $\bm{\lambda}$ (for $\eqref{1}$, that is any pair $(\bm{x},\bm{\lambda})$ satisfying $f_i(\bm{x})\leq0$, $i=1,\dots,m$, and $\bm{\lambda}\geq\bm{0}$) is such that $\sum_{i=1}^m\lambda_if_i(\bm{x})\leq0$, which means the Lagrangian $\eqref{3}$ satisfies $\mathcal{L}(\bm{x},\bm{\lambda}) \leq f_0(\bm{x})$. This implies that the dual function satisfies

for all primal-dual feasible pairs $(\bm{x},\bm{\lambda})$. Since the dual optimum $d^\star$ is just one particular value of $d(\bm{\lambda})$ and the primal optimum $p^\star$ is just one particular value of $f_0(\bm{x})$, this relationship tells us that

a result known as weak duality. Weak duality always holds regardless of whether the primal problem is convex. It tells us that the duality gap is always non-negative and that $d^\star$ acts as a lower bound on the solution to the original problem.

Strong Duality

As mentioned above, when $d^\star=p^\star$, we say that strong duality holds. Strong duality always holds when the primal problem is convex and Slater’s condition is satisfied, but interestingly it occasionally also holds for non-convex problems, allowing us to obtain the exact solution by solving the (convex) dual instead. A notable example is the inequality-constrained QC1QP, which is the main topic of this post.

S-Lemma

To prove that strong duality always holds for an inequality-constrained QC1QP (even when it is not convex), we will make use of the S-lemma (also known as the S-procedure; see Appendix B.2 of Convex Optimization for more information), which states the following:

Suppose we have two quadratic functions $f_0(\bm{x})$ and $f_1(\bm{x})$, defined as in $\eqref{2}$, and assume there exists some $\tilde{\bm{x}}\in\mathbb{R}^n$ such that $f_1(\tilde{\bm{x}})<0$. Then the implication

holds if and only if there exists some $\lambda\geq0$ such that

Discussion and Interpretation

Before using the S-lemma to prove strong duality, let’s take a moment to consider what it means. The S-lemma is a type of result known as a (strong) theorem of alternatives, because it states that exactly one of $\eqref{8}$ and $\eqref{9}$ is true. The most famous theorem of alternatives is Farkas’ lemma, which applies to systems of linear equations rather than quadratic ones.

The existence of a non-negative real number $\lambda$ satisfying $\eqref{9}$ serves as a certificate of the implication $\eqref{8}$; that is, the existence of such a number certifies that the inequality $f_0(\bm{x})\leq0$ holds over the entire set $\{\bm{x}\in\mathbb{R}^n\mid f_1(\bm{x})\leq0\}$ (conversely, if we can find a point $\tilde{\bm{x}}\in\mathbb{R}^n$ such that $f_1(\tilde{\bm{x}})\leq0$ but $f_0(\tilde{\bm{x}})>0$, then we know there is no $\lambda\geq0$ that satisfies $\eqref{9}$). As we will see below, requiring only a single value to prove a set-wise inequality is a powerful tool.

Note that the inequality $\eqref{9}$ is equivalent to

If you are familiar with sum-of-squares polynomials, you may immediately recognize the equivalence. However, we can also show this from first principles. Writing the two functions in their homogeneous forms and rearranging $\eqref{10}$, we get

Clearly $\eqref{9}$ implies $\eqref{11}$. We must prove the other direction as well. To do so, we will show that $\eqref{11}$ implies

which is equivalent to $\eqref{9}$. There are two cases. If $\zeta\neq0$, we can substitute $\bm{x}=(1/\zeta)\bm{z}$ into $\eqref{11}$ and multiply both sides by $\zeta^2$ to obtain $\eqref{12}$. Alternatively, if $\zeta=0$, then $\eqref{12}$ simplifies to

Since $\eqref{11}$ is a quadratic function of $\bm{x}$ that is bounded below (by zero), it must be convex, implying $\lambda\bm{Q}_1-\bm{Q}_0\succcurlyeq\bm{0}$ and thus $\eqref{13}$ holds for all $\bm{z}\in\R^n$. We conclude that $\eqref{11}$ implies $\eqref{12}$ and therefore $\eqref{9}$, as desired.

Example

To gain some intuition about the S-lemma, let’s consider a simple one-dimensional example. Let

as depicted in the figure below.

Two (non-convex) quadratic functions $f_0$ and $f_1$ for which $f_1(x)\leq 0$ implies $f_0(x)\leq 0$ for all $x\in\mathbb{R}$. As stated by the S-lemma, we can thus find a value $\lambda\geq0$ such that $\lambda f_1(x)\geq f_0(x)$ for all $x\in\mathbb{R}$.

As we can see (both from the figure and the fact that the signs of the $x^2$ terms are negative), both of the functions are concave. The roots of $f_1$ are $x=-1$ and $x=3$, so $f_1(x)\leq0$ for any $x\leq-1$ or $x\geq3$ (notice that this is a non-convex, disjoint set). The roots of $f_0$ are $x=-0.5$ and $x=2$; as these are both located between the roots of $f_1$, we have

The S-lemma tells us that there is therefore some $\lambda\geq0$ such that

Multiplying $f_1$ by a non-negative scalar $\lambda$ does not change its roots or convexity, just scales its value, as can be seen in the figure above. We just need to find a $\lambda$ large enough so that $\eqref{14}$ is satisfied. From the figure, we see that $\lambda=1$ is too small while $\lambda=1.5$ is larger than needed. Solving the small SDP

with

we find that the smallest $\lambda$ satisfying $\eqref{14}$ is $\lambda^\star\approx1.203$. The code used to generate the figure and solve the SDP can be found here.

Proof of Strong Duality for the QC1QP

Let us now prove that strong duality holds for the inequality-constrained QC1QP; that is, the optimal values of the primal problem $\eqref{6}$ and dual problem $\eqref{7}$ are equal. By weak duality, we always have $d^\star\leq p^\star$; we will make use of the S-lemma to show that this inequality is in fact always tight.

From the primal problem, we know that

since $p^\star\leq f_0(\bm{x})$ everywhere in the feasible set $\{\bm{x}\in\mathbb{R}^n\mid f_1(\bm{x})\leq0\}$, by definition. Assuming the primal problem is strictly feasible (i.e., there exists some $\tilde{\bm{x}}\in\mathbb{R}^n$ such that $f_1(\tilde{\bm{x}})<0$), the S-lemma tells us that there exists some $\tilde{\lambda}\geq0$ satisfying

Pre- and post-multiplying both sides of the above expression by

respectively, and rearranging a little, we obtain

which holds for all $\bm{x}\in\mathbb{R}^n$. Noticing that the left hand side is equal to $\mathcal{L}(\bm{x},\tilde{\lambda})$, we have

Recalling that

we get $d^\star\geq p^\star$. Combined with weak duality, this reveals that

which of course implies $d^\star=p^\star$, and therefore strong duality holds. This means that we can obtain the exact optimal value of the non-convex problem $\eqref{6}$ by instead solving the convex dual problem $\eqref{7}$.

Equality Constraints

Consider instead the equality-constrained QC1QP

which has dual

To prove strong duality holds, the standard S-lemma is no longer useful. Instead, we must turn to the S-lemma with equality, which is characterized in (Xia, Wang, & Sheu, 2014). It states the following:

S-Lemma with Equality

Suppose we have two quadratic functions $f_0(\bm{x})$ and $f_1(\bm{x})$, defined as in $\eqref{2}$, and assume certain technical conditions hold (see below). Then the implication

holds if and only if there exists some $\lambda\in\mathbb{R}$ such that

Technical Conditions

Notice the similarity to the standard S-lemma, except that the left hand side of the implication $\eqref{15}$ is now an equality, $\lambda$ is unconstrained, and we have assumed some as-yet-unstated technical conditions. The full set of these conditions is provided in the paper linked above but is fairly complicated, so we will only state two sufficient ones here:

• there exist points $\tilde{\bm{x}}_1,\tilde{\bm{x}}_2\in\mathbb{R}^n$ such that $f_1(\tilde{\bm{x}}_1)<0$ and $f_1(\tilde{\bm{x}}_2)>0$ (this is sometimes called the two-sided Slater’s condition) and $\bm{Q}_1\neq\bm{0}$; or
• $\bm{Q}_1$ is strictly positive or negative definite.

The second of these conditions holds in the extremal eigenvalue problem we looked at previously, in which $\bm{Q}_1=\bm{1}_n\succ\bm{0}$, with $\bm{1}_n$ denoting the $n\times n$ identity matrix. An example using the first condition is given in (Bishop, Gerding, & Tran-Thanh, 2020), where it is applied to an optimal learning problem.

Strong Duality

Assuming at least one of the sets of conditions required for the S-lemma with equality holds, the proof of strong duality for the equality-constrained QC1QP is essentially the same as that of the inequality-constrained case, so we won’t bother repeating it here (consider it an “exercise for the reader”). However, we will note that we can construct equality-constrained QC1QPs in which the technical conditions of the S-lemma with equality do not hold and as a consequence neither does strong duality. For example, consider the QC1QP

the constraint of which implies $x_2^\star=0$ and therefore the optimal value is $p^\star=0$. The Lagrangian is

However, since there is no value of $\lambda\in\mathbb{R}$ for which $\tilde{\bm{Q}}(\lambda)\succcurlyeq\bm{0}$ (one way to see this is to recall that the determinant of a positive semidefinite matrix must be non-negative, but $\mathrm{det}\,\tilde{\bm{Q}}(\lambda)=-1/4$ for all $\lambda$), the Lagrangian is unbounded below and the dual optimal value is thus $d^\star=-\infty$, so there is an infinite duality gap.

Thanks to Connor Holmes for reading a draft of this post.

A convex combination of these points is a weighted sum $\bm{x}=\sum_iw_i\bm{v}_i$ with non-negative weights $w_i$ satisfying $\sum_iw_i=1$. The convex hull of $\mathcal{V}$ is the set of all convex combinations of points in $\mathcal{V}$. Gathering the points in $\mathcal{V}$ into the columns of the matrix $\bm{V}=[\bm{v}_1,\dots,\bm{v}_n]$, we can write the convex hull as the set

where $\bm{e}$ is a vector of ones, $\bm{w}=[w_1,\dots,w_n]^T$ is the vector of weights, and the inequality is taken elementwise.

In this post, we explore how to put additional constraints on the weights $\bm{w}$ in $\eqref{1}$ and what the resulting set looks like.

Polyhedral Double Description

Let’s begin with some background on polyhedral geometry. A polyhedron is a geometric shape with flat sides (called faces). In this post we will assume we are talking about convex polyhedra; a polyhedron is convex when the line between any two points in the polyhedron is also completely contained in it. A bounded polyhedron is called a polytope and a two-dimensional polyhedron is called a polygon, some examples of which are shown in Figure 1 below.

Figure 1: Examples of two (convex) polygons, each of which can be defined as the convex hull of its vertices.

By the Minkowski-Weyl theorem, any polyhedron $\mathcal{P}$ can be equivalently represented in two different ways, together known as the double description of $\mathcal{P}$. These two representations are:

Vertex Representation

The vertex representation (V-rep) can be written as

where each column of $\bm{V}$ is a vertex and each column of $\bm{U}$ is a ray (together these are referred to as generators). When $\bm{U}=\bm{0}$, $\mathcal{P}$ is just a convex hull like $\eqref{1}$. When $\bm{V}=\bm{0}$, $\mathcal{P}$ instead reduces to a conical hull; however, we will only consider cases with $\bm{U}=\bm{0}$ in this post.

Halfspace Representation

The halfspace representation (H-rep) can be written as

which is a finite intersection of halfspaces. Equality constraints of the form $\bm{A}\bm{x}=\bm{b}$, which serve to restrict to the polyhedron to a lower-dimensional subspace of $\mathbb{R}^d,$ can also be included by equivalently using two inequalities:

Converting from H-rep to V-rep is called vertex enumeration, while converting from V-rep to H-rep is called facet enumeration. No provably polynomial-time algorithms are known for these conversions, but nevertheless they can be computed fairly quickly when the dimension and number of generators and faces is small, using a library like cddlib or its Python bindings pycddlib.

You can read more about polyhedral double description here and see some examples of its application in robotics in this paper as well as this more recent paper of mine.

Computing Constrained Convex Hulls

Now that we’re equipped with some background on polyhedra, you may notice that the set of weights we used to define the convex hull $\eqref{1}$ is itself the H-rep of the polyhedron

This particular polyhedron is known as a simplex and is a special case of the general form $\eqref{2}$ with

where $\bm{1}_n$ is the $n\times n$ identity matrix. To incorporate additional constraints $\bm{\ell}\leq\bm{w}\leq\bm{u}$, where $\bm{0}\leq\bm{\ell}\leq\bm{u}\leq\bm{e}$, $\bm{e}^T\bm{\ell}\leq1$, and $\bm{e}^T\bm{u}\geq1$ (without these latter two conditions, the constraints could be inconsistent because $\bm{e}^T\bm{w}=1$ is impossible to realize if $\bm{e}^T\bm{\ell}>1$ or $\bm{e}^T\bm{u}<1$), we add them to $\bm{A}$ and $\bm{b}$ to obtain

Converting this H-rep to V-rep, we get the set

with some matrix $\bm{Q}$ (note $\mathcal{W}'$ is no longer a simplex). Substituting $\bm{w}=\bm{Q}\bm{y}$ into $\eqref{1}$, we get the constrained convex hull

which is another V-rep polyhedron with weights $\bm{y}$ and vertices given by the columns of $\bm{V}\bm{Q}$.

Visualizing Constrained Convex Hulls

Now that we can compute the vertices of a constrained convex hull, we can plot it. Figure 2 below shows the constrained convex hulls for different values of $\bm{\ell}=\ell\bm{e}$ and $\bm{u}=u\bm{e}$. You can play with this yourself using the Python code here.

Figure 2: Examples of constrained convex hulls. The original convex hull is shown in blue, while each of the red polygons is the convex hull with different bounds applied. From top to bottom we have bounds: $(\ell,u)=(0.05, 1)$, $(\ell,u)=(0, 0.6)$, and $(\ell,u)=(0.05, 0.6)$.

As we can see, the lower bound $\bm{\ell}$ has the effect of “pulling” the hull in from the faces of the original polytope, while the upper bound $\bm{u}$ pulls the hull away from the vertices. We only consider bounds in this post, but in general we can add any desired affine constraints to $\bm{w}$.

Analytical Solutions

It turns out that when have only lower or upper bounds (not both) then we can calculate the vertices of the constrained convex hulls analytically (that is, without converting between V-rep and H-rep).

Lower Bounds

Let us first consider lower bounds $\bm{\ell}$. The $i$th vertex $\bm{v}_i$, which corresponds to weight $w_i\geq\ell_i$, pulls the convex hull toward it from each other vertex by a factor of $\ell_i$. An example with the shape from Figure 2 with a lower bound on only one of the vertex weights and no (additional) upper bounds is shown in Figure 3 below.

Figure 3: The constrained convex hull (red) of the polygon from Figure 2 (blue) with bounds $\bm{\ell}=[\ell,0,0,0,0]^T$, $\ell=0.2$, and $\bm{u}=\bm{e}$. The vertex corresponding to the non-zero lower bound is on the far left side. The lower bound has the effect of “pulling” each of the other vertices toward the bounded vertex, proportionally to the value of the bound. To see this, a line has been drawn between the bounded vertex and each other vertex, with a fraction proportional to $\ell$ coloured yellow and the remainder, proportional to $(1-\ell)$, in green (entire figure shown on gray background for better contrast).

Each vertex $\bm{v}_i'$ of the constrained convex hull depends on the bounds at all vertices of the original shape. To compute it, we have the formula

Implemented in Python code, it looks like

# V is the array of original vertices, one per row, shape (n, d) (yes, this is
# the transpose of the the definition of V in eq. (1) - sorry)
# l is the array of lower bounds, shape (n,)

V_new = []  # vertices of the lower-bounded convex hull
for i in range(n):
    v_new = V[i, :] + sum([l[j] * (V[j, :] - V[i, :]) for j in range(n)])
    V_new.append(v_new)

Upper Bounds

The upper bounds are a bit different. Each lower bound affected all other vertices, while each upper bound only affects the current vertex (i.e., the one corresponding to the bound), and this effect is a function of only the adjacent (i.e., neighbouring) vertices. Furthermore, unlike the lower-bounded case, the upper bound often results in a constrained convex hull with a different number of vertices than the original polytope. An example with the shape from Figure 2 with only a single upper bound on the vertex weights and no (additional) lower bounds is shown in Figure 4 below.

Figure 4: The constrained convex hull (red) of the polygon from Figure 2 (blue) with bounds $\bm{\ell}=\bm{0}$ and $\bm{u}=[u,1,1,1,1]^T$ with $u=0.6$. The vertex corresponding to the non-unity upper bound is on the far left side. The upper bound has the effect of pulling the hull toward the neighbouring vertices, proportionally to $(1-u)$, creating a new face (dashed line). To see this, a line has been drawn between the bounded vertex and its two neighbours, with a fraction proportional to $u$ coloured yellow and the remainder, proportional to $(1-u)$, in green.

Let $\mathcal{V}'$ be the set of vertices of the constrained convex hull, initially empty. Let $\mathcal{A}(\bm{v}_i)$ be the set of indices of the vertices adjacent to vertex $\bm{v}_i$. For each vertex $\bm{v}_i$ and each of its adjacent vertices $\bm{v}_j,j\in\mathcal{A}(\bm{v}_i)$, we compute

and add it to the set of new vertices:

Once we’ve gone through all vertices and their neighbours, we are left with the complete set of vertices $\mathcal{V}'$ of the upper-bounded convex hull. In Python, we have

# V is the array of original vertices, one per row, shape (n, d)
# u is the array of upper bounds, shape (n,)

# note I assume we have a function adj(V, i) that returns the indices of
# vertices adjacent to V[i, :], but this can be computed fairly efficiently (in
# my actual code I use pycddlib, which uses linear programming to compute
# adjacency)

V_new = []  # vertices of the upper-bounded convex hull
for i in range(n):
    for j in adj(V, i):
        v_new = V[j, :] + u[i] * (V[i, :] - V[j, :])
        V_new.append(v_new)

Note that this approach for computing the upper-bounded convex hull only works when $u_i+u_j\geq1$ for all $i=1,\dots,n$ and $j\in\mathcal{A}(\bm{v}_i)$. Otherwise, the new faces added for each upper bound can intersect with each other, which complicates the solution considerably.

Both Lower and Upper Bounds

If we have both lower and upper bounds, we can independently compute the resulting constrained convex hulls using the analytical formulas above. To combine them into a single constrained convex hull, we need to take their intersection—unfortunately, I am not aware of an analytical solution for this in the vertex representation. Instead, we must again turn to polyhedral double description.

Let $\mathcal{V}_\ell$ and $\mathcal{V}_u$ be the sets of vertices of the lower- and upper-bounded convex hulls, respectively. Then:

1. Convert both of these V-rep polytopes to the H-reps $\{\bm{x}\in\mathbb{R}^d\mid\bm{A}_\ell\bm{x}\leq\bm{b}_\ell\}$ and $\{\bm{x}\in\mathbb{R}^d\mid\bm{A}_u\bm{x}\leq\bm{b}_u\}$.
2. Stack the H-reps to combine them into the single polytope $\{\bm{x}\in\mathbb{R}^d\mid\bm{A}\bm{x}\leq\bm{b}\},$ where$$\begin{align*} \bm{A} &= \begin{bmatrix} \bm{A}_\ell \\ \bm{A}_u \end{bmatrix}, & \bm{b} &= \begin{bmatrix} \bm{b}_\ell \\ \bm{b}_u \end{bmatrix}. \end{align*}$$
3. Convert the combined H-rep polytope back into V-rep if desired (e.g., for plotting).

Uniform Padding

Finally, notice that the width of the padding between the boundary of the constrained convex hull and that of the original shape produced by the lower bound in Figure 2 is not uniform. If you do want uniform padding, do the following:

1. Convert the polytope to the H-rep $\{\bm{x}\in\mathbb{R}^d\mid\bm{A}\bm{x}\leq\bm{b}\}$.
2. For each row $\bm{a}_i^T$ of $\bm{A}$ and corresponding element $b_i$ of $\bm{b}$, normalize and subtract the desired padding width $r$:$$\begin{align*} b_i &\leftarrow b_i/\|\bm{a}_i\|_2 - r, \\ \bm{a}_i &\leftarrow\bm{a}_i/\|\bm{a}_i\|_2. \end{align*}$$
3. Convert back to the V-rep for plotting.

The normalization in the second step makes each row $\bm{a}_i$ a unit vector without changing the polytope defined by the inequalities. This ensures each inequality has the same scale, so that subtracting $r$ from each results in the same width of padding for each face. An example of uniform padding is shown in Figure 5 below.

Figure 5: The same polygon from Figure 2 inset with uniform padding of width $r=50$ pixels.

Thanks to Abhishek Goudar for reviewing a draft of this post.

Physics engines are widely used in games and simulators. These engines simulate an environment in a discrete manner: at a given time $t$, they compute the state of the environment (that is, the position and velocities of all objects) at the next time $t+\Delta t$, where $\Delta t>0$ is a small, usually fixed, time step. This process is repeated until the desired duration has elapsed or the simulation is otherwise ended.

One of the physics engine’s main tasks is collision detection; that is, checking if two objects are in contact or overlapping after each time step. For simple shapes like circles and spheres, conditions for collision can be easily derived by hand; for more general shapes, there are many interesting collision detection algorithms like SAT, GJK, and EPA.

However, what if the collision occurs during a time step? For example, consider the shapes in Figure 1 below. The circle (red) is moving with constant velocity toward the stationary wall (blue). As we can see, the circle should collide with the wall during the time step between times $t$ and $t+\Delta t$, but the two shapes are not colliding at times $t$ or $t+\Delta t$. This means that if we only check for collisions at these discrete instants in time, we would miss the collision entirely. This phenomenon is called tunnelling, and is more likely to occur with objects that are thin and/or moving quickly.

Figure 1: A circle (red) tunnelling through a wall (blue) during the time step between times $t$ and $t+\Delta t$.

The solution is continuous collision detection (CCD), in which we check for overlap between the swept areas (or swept volumes, in 3D) of the two objects over the time step. That is, we don’t just look at where the objects end up, but at all of the space they pass through over the time step. Figure 2 shows the swept area of the circle from Figure 1, which does overlap with the wall. Having computed the swept areas, we can use any of the algorithms mentioned above to check for a collision between them.

Figure 2: The swept area of the circle from Figure 1, which overlaps the wall in the hatched region.

When a collision is detected between two swept areas, we typically also want to compute exactly when the collision occurred during the time step, so that behaviour can be adjusted accordingly. For example, at the instant of collision between a ball and a wall, we may reverse the ball’s direction (normal to the wall) to simulate a bounce. We won’t get more into calculating collision time in this post, but you can read more about CCD in games here and here and in robot motion planning here. I also described a form of CCD I used in a research paper in this previous blog post.

As a Visual Effect

Last year I had some fun making a little browser-based game called Saber, in which the player swings around a lightsaber to destroy bouncing balls (if you haven’t played it before, I encourage you to go do so now—it’s fun and will make the rest of this post easier to follow). The saber itself is essentially a thin line segment which can move quite fast, so CCD is needed to reliably detect collisions between it and the balls.

However, the swept area of the saber is not straightforward to compute exactly. Indeed, since the saber experiences both linear and angular motion, the swept area may not even be a convex shape (convexity generally makes collision detection easier). Therefore, to simplify things, I just approximated the swept area over each time step as a quadrilateral (i.e., a four-sided polygon), as shown in the examples in Figure 3 below (the figure is animated—press the small play button in the bottom left corner).

As a collision detection method for the game, the quadrilateral approximation works well. While it may result in spurious detections for non-convex swept areas like the one on the right side of Figure 3, I haven’t really noticed this during gameplay. And besides, tunnelling would be far more jarring than a slightly conservative collision shape.

What I didn’t realize is that this application of CCD also produces a really cool visual effect (in my opinion) if the swept area is drawn at each time step: it looks like the light trail left by a swinging lightsaber, hence the name of the game. You can play with an interactive demo of this effect in Figure 4 (but you should also just go play the game).

Thanks to Galen Leir-Taha for reviewing a draft of this post.