Standard Form Polyhedra
CPSC 406 – Computational Optimization
Standard Form Polyhedra
\[ \def\argmin{\operatorname*{argmin}} \def\argmax{\operatorname*{argmax}} \def\bbar{\bar b} \def\Ball{\mathbf{B}} \def\bmat#1{\begin{bmatrix}#1\end{bmatrix}} \def\Diag{\mathbf{Diag}} \def\half{\tfrac12} \def\int{\mathop{\rm int}} \def\ip#1{\langle #1 \rangle} \def\maxim{\mathop{\hbox{maximize}}} \def\maximize#1{\displaystyle\maxim_{#1}} \def\minim{\mathop{\hbox{minimize}}} \def\minimize#1{\displaystyle\minim_{#1}} \def\norm#1{\|#1\|} \def\Null{{\mathbf{null}}} \def\Bscr{\mathcal{B}} \def\Nscr{\mathcal{N}} \def\Pscr{\mathcal{P}} \def\proj{\mathbf{proj}} \def\R{\mathbb R} \def\Re{\mathbb R} \def\Real{\mathbb R} \def\Rn{\R^n} \def\rank{\mathbf{rank}} \def\range{{\mathbf{range}}} \def\sign{{\mathbf{sign}}} \def\span{{\mathbf{span}}} \def\st{\hbox{subject to}} \def\T{^\intercal} \def\textt#1{\quad\text{#1}\quad} \def\trace{\mathbf{trace}} \def\xbar{\bar x} \def\pstar{p^*} \def\xstar{x^*} \def\xhat{\hat x} \]
- standard form
- converting to standard form
- two-dimensional representations
- degeneracy: inequality form
- degeneracy: standard form
Polyhedra in standard form
Generic polyhedron \[ \Pscr = \Set{ x | \begin{aligned} Ax &= b \\Cx &\le d \end{aligned}} \]
Standard-form polyhedron \[ \Pscr = \Set{ x | \begin{aligned} Ax &= b \\ x &\ge 0 \end{aligned}} \textt{with} b \ge 0 \]
Ensure nonnegative \(b\)
For \(b_i < 0\), replace \[ a_i x = b_i \quad \longrightarrow \quad (-a_i) x = (-b_i) \]
For \(d_i < 0\), replace \[ \begin{aligned} c_i\T x \le d_i \quad &\longrightarrow \quad (-c_i)\T x \ge (-d_i) \\ c_i\T x \ge d_i \quad &\longrightarrow \quad (-c_i)\T x \le (-d_i) \end{aligned} \]
Introduce slack and surplus variables
For every inequality constraint of the form \[ c_i\T x \le d_i \qquad \left(c_i\T x \ge d_i\right) \]
introduce a new slack (or surplus) variable \(s_i\), replacing the inequality with two constraints
\[ \begin{matrix} \begin{aligned} c_i\T x + s_i &= d_i \\ s_i &\ge 0 \end{aligned}&\qquad \left( \begin{aligned} c_i\T x - s_i &= d_i \\ s_i &\ge 0 \end{aligned} \right) \end{matrix} \]
Reformulate free variables
- \(x_i\) is called a free variable if it has no constraints
- there are no free variables in standard form — every variable must be nonnegative
Converting free variables
every free variable \(x_i\) is replaced with two new variables \(x'_i\) and \(x''_i\), ie, \[ x_i := x'_i - x''_i, \quad x'_i \ge 0 \text{ and } x''_i \ge 0 \]
\(x'_i\) encodes the positive part of \(x_i\)
\(x''_i\) encodes the negative part of \(x_i\)
optimal solution necessarily has \(x'_i\cdot x''_i=0\) (why?)
Example
Consider the following LP problem:
\[ \begin{align} \text{maximize:} \quad & 6x_1 + 8x_2 \\ \text{subject to:} \quad & 4x_1 + 2x_2 \leq 60 \\ & x_1 + 2x_2 \leq 32 \\ & x_2 \geq 5 \\ & x_1 \geq 0 \end{align} \]
To convert to standard form:
- Handle the lower bound on \(x_2\): Substitute \(x_2 = y_2 + 5\) where \(y_2 \geq 0\)
- Introduce slack variables for the inequality constraints
- Rewrite the objective for minimization
\[ \begin{align} \text{minimize:} \quad & -6x_1 - 8y_2 - 40 \\ \text{subject to:} \quad & 4x_1 + 2y_2 + s_1 = 50 \\ & x_1 + 2y_2 + s_2 = 22 \\ & x_1, y_2, s_1, s_2 \geq 0 \end{align} \]
Basic solutions in standard form
\(\xstar\) is a basic solution if the vectors in the basis set are linearly independent: \[ a_{i_1}, a_{i_2}, \ldots, a_{i_n}, \quad i_j\in\Bscr \]
in standard form, there are:
- \(n\) variables (\(x_1, \ldots, x_n\))
- \(m+n\) total constraints
- \(m\) equality constraints (\(Ax = b\))
- \(n\) inequality constraints (\(x \ge 0\))
for any basic solution \(x\):
- the basic set \(\Bscr\) must have \(n\) elements
- thus, exactly \(n\) of the constraints need to be active at \(x\)
- \(m\) equality constraints are always satisfied
- thus \(n-m\) of the inequality constraints \(x\ge0\) should be “active”
Basic solutions in standard form
Choosing \(n-m\) of the inequality constraints to be active is the same as choosing \(n-m\) variables \(x_i\) to be zero.
Making \(x_i\) zero effectively eliminates column \(i\) from the matrix \(A\).
This is equivalent to choosing \(m\) columns of \(A\)! To be a basic solution, we also need these \(m\) columns to be linearly independent. So, permute the variables and partition \[ AP = [B\quad N]\quad\text{where $B$ is nonsingular} \]
Now we have \[ \bar A x = \left[ \begin{matrix} B & N \\ & I \end{matrix} \right] \left[ \begin{matrix} x_B \\ x_N \end{matrix} \right] = \left[ \begin{matrix} b \\ 0 \end{matrix} \right] \]
\[ \begin{aligned} x_N &= 0 \\ Bx_B &= b \end{aligned} \]
Two-dimensional representation
Degeneracy: inequality form
Polyhedron in inequality form: \[ Ax \le b \]
A basic feasible solution \(\xstar\) with \[ a_i\T \xstar = b_i, \quad i\in\Bscr \textt{and} a_i\T \xstar < b_i, \quad i\notin\Bscr \] is degenerate if # of indices in \(\Bscr\) is greater than \(n\)
- property of the description of the polyhedron
- affects the performance of some algorithms
- disappears for small perturbations of \(b\)
Degeneracy: standard form
Polyhedron in standard form: \[ Ax = b, \quad x \ge 0 \]
A basic solution partitions the variables into two sets: \[ \bmat{B&N}\bmat{x_B\\x_N} = b \textt{with} x_N = 0 \] ie, \[ Bx_B = b \]
A basic feasible solution in standard form is degenerate if more than \(n-m\) components in \(x\) are zero, ie, \[ x = \bmat{x_B\\x_N} \begin{matrix} m \hfill \\ n-m \end{matrix} = \bmat{B^{-1}b\\0} \begin{matrix} \longleftarrow\hbox{\small has some zero components} \hfill \\ \ \end{matrix} \]