Convex Sets

CPSC 406 – Computational Optimization

Convex sets

\[ \def\argmin{\operatorname*{argmin}} \def\Ball{\mathbf{B}} \def\bmat#1{\begin{bmatrix}#1\end{bmatrix}} \def\Diag{\mathbf{Diag}} \def\half{\tfrac12} \def\int{\mathop{\rm int}} \def\ip#1{\langle #1 \rangle} \def\maxim{\mathop{\hbox{\rm maximize}}} \def\maximize#1{\displaystyle\maxim_{#1}} \def\minim{\mathop{\hbox{\rm minimize}}} \def\minimize#1{\displaystyle\minim_{#1}} \def\norm#1{\|#1\|} \def\Null{{\mathbf{null}}} \def\proj{\mathbf{proj}} \def\R{\mathbb R} \def\Re{\mathbb R} \def\Rn{\R^n} \def\rank{\mathbf{rank}} \def\range{{\mathbf{range}}} \def\sign{{\mathbf{sign}}} \def\span{{\mathbf{span}}} \def\st{\hbox{\rm subject to}} \def\T{^\intercal} \def\textt#1{\quad\text{#1}\quad} \def\trace{\mathbf{trace}} \]

• definition
• subspaces, affine sets, and spans
• halfspaces and hyperplanes
• cones and hulls
• operations that preserve convexity

Lines and line segments

• line through two points \(x_1,x_2\in\Rn\) is the set

\[ \set{z \mid z = \theta x_1 + (1-\theta)x_2,\ \theta\in\R} \]

• \(x\in\Rn\) is a convex combination of vectors \(x_1,\ldots,x_k\) if \[ x = \sum_{i=1}^k \theta_i x_i, \quad \sum_{i=1}^k \theta_i = 1, \quad \theta_i \ge 0 \]

Convex sets and hulls

• the convex hull of a set of points \(\mathcal{S}\) contains all convex combinations of points in \(\mathcal{S}\): \[ \text{conv}(\mathcal{S}) = \left\{ \sum_{i=1}^k \theta_i x_i \mid x_i\in\mathcal{S},\ \sum_{i=1}^k \theta_i = 1, \ \theta_i\ge 0 \right\} \]

 

 

• \(\mathcal{C}\subset\Rn\) is convex if it contains all convex combinations of its elements, ie, \(\mathcal{C}= \text{conv}(\mathcal{C})\)

 

 

Subspaces, affine sets, and spans

• \(\mathcal{S}\subset\Rn\) is a subspace if it contains all linear combinations of points in the set, ie,

\[ \alpha x + \beta y \in \mathcal{S}, \ \forall x,y\in\mathcal{S}, \ \forall \alpha,\beta\in\R \]

• for any \(m\times n\) matrix \(A\), its range and nullspace are subspaces of \(\Rn\): \[ \mathop{\bf range}(A) = \{Ax \mid x\in\Rn\} \quad\text{and}\quad \mathop{\bf null}(A^T) = \{z \mid A^Tz = 0\} \]

• \(\mathcal{L}\) is an affine set if it’s a translated a subspace, ie, for fixed \(x_0\in\Rn\) and subspace \(\mathcal{S}\),

\[ \mathcal{L} = \{x_0 + v \mid v\in\mathcal{S}\} \equiv x_0 + \mathcal{S} \]

• \(\mathcal{S}\) is the subspace parallel to \(\mathcal{L}\)

• the span of a collection of vectors \({x_1,\ldots,x_k}\) is the subspace of all vectors

\[ y = \sum_{i=1}^k \theta_i x_i, \quad \forall\theta_i\in\R \]

Halfspaces and hyperplanes

fix nonzero vector \(a\in\Rn\) and scalar \(\beta\)

• hyperplanes and halfspaces, respectively, have the form

\[ \mathcal{H} = \{x \mid a^Tx = \beta\} \quad\text{and}\quad \mathcal{H}_- = \{x \mid a^Tx \le \beta\} \]

• \(a\) is the normal to the hyperplane
• hyperplanes are affine and convex
• halfspaces are convex but not affine

Example: the nonnegative orthant is an intersection of \(n\) halfspaces \[ \Rn_+ = \{x \mid x_i \ge 0, \ i=1,\ldots,n\} = \bigcap_{i=1}^n\{x \mid e_i^Tx \ge 0\} \]

Convex cones

• a set \(\mathcal{K}\subset\Rn\) is a cone if \(x\in\mathcal{K} \ \Longleftrightarrow \ \alpha x\in\mathcal{K}\) for all \(\alpha\ge 0\)

• a convex cone is a cone that is also convex

\[ x, y \in \mathcal{K} \text{ and } \ \alpha, \beta\ge 0 \ \implies \ \alpha x + \beta y \in \mathcal{K} \]

Examples

\[ \begin{aligned} \Rn_+ &= \{x \mid x_i \ge 0, \ i=1,\ldots,n\} &\quad\text{(nonnegative orthant)}\\[10pt] \mathcal{L}_+^n&=\left\{\begin{bmatrix}x\\t\end{bmatrix}\in\R^{n+1} \, \middle |\, \|x\|_2\le t, \ x\in\Rn, \ t\in\R\right\} &\quad\text{(second-order cone)}\\[10pt] \mathcal{S}_+^n&=\Set{X\in\R^{n\times n} \mid X\succeq 0} &\quad\text{(positive semidefinite cone)} \end{aligned} \]

Operations that preserve convexity

Let \(\mathcal{C}_1,\mathcal{C}_2\) be convex sets in \(\Rn\).

• nonnegative scaling:

\[ \theta\mathcal{C}_1 = \{\theta x \mid x\in\mathcal{C}_1\}, \quad \theta\ge 0 \]

• intersection:

\[ \mathcal{C}_1\cap\mathcal{C}_2 \]

• sum:

\[ \mathcal{C}_1+\mathcal{C}_2 = \{x+y \mid x\in\mathcal{C}_1,\ y\in\mathcal{C}_2\} \]

Convex polytopes

\(\mathcal{S}\) is a convex polytope if it’s the intersection of a finite number of halfspaces

\[ \mathcal{S} = \bigcap_{i=1}^m\{x \mid a_i^Tx \le β_i\} = \set{x \mid Ax \le b} \]

where

\[ A = \begin{bmatrix} a_1^T\\\vdots\\ a_m^T \end{bmatrix} \in\R^{m\times n} \quad\text{and}\quad b\in\R^m \]

Example: \(n\)-dimensional simplex is intersection of \(n\) halfspaces and a hyperplane

\[ \mathcal{C} = \left\{x \ \middle | \ \sum_{i=1}^{n}x_i=1, \ x_i\ge 0\right\} \]