Computational Optimization

Course goals and emphasis

• recognize and formulate the main optimization problem classes
• understand how to apply standard algorithms for each class
• recognize that an algorithm has succeeded or failed
• hands-on experience with mathematical software

Role of optimization

• fitting a statistical model to data (machine learning)
• logistics, economics, finance, risk management
• theory of games and competition
• theory of computer science and algorithms
• geometry and analysis

Learning models

a model is a simplified abstraction of process

• model parameters
• prediction

least-error principle: the optimal parameters minimizes the distance between the model and the observation

Mathematical Optimization

• Objective function:
• feasible set (eg, "constraints"):
• decision variables:

Abstract problem

• find such that is minimal, eg,

• optimal solution set

Gradients wanted

Assume the objective is differentiable on the nonempty interior of the feasible set

• measures the objective's sensitivity to feasible perturbations
• usually sufficient to devise tractable and implementable algorithms

Example: linear models

• least-squares approximation: minimizes

• least absolute-sum approximation: no closed-form solution for minimizing

Example: Scheduling

Minimize number nurses needed to meet weekly staffing demands

Constraints

• each nurse works 5 straight days with 2 days off
• nurses required on nights

First attempt

• nurses work on night
• minimize subject to with
• doesn't respect days off constraints

Scheduling: second attempt

let be number of nurses starting their 5-day shift on day :

• note the constraint structure. This is almost always true of practical LPs
• we may want to restrict to be integer. That's a much harder problem!

Coursework and evaluation

• 8 homework assignments (30%)
  • programming and mathematical deriviations
  • typeset submissions, correctness, and writing quality graded
• midterm exam (30%): ️ and , short mathematical problems
• final exam (40%): multiple choice

Programming in Julia

• common programming languages in optimization: Python, Matlab, R; C++
• we'll use Julia only
  • Matlab-like syntax, but free and fast
  • lots of tutorials available
• Pluto or Jupyter or ️notebooks highly recommended for assignments

Assignments

• work alone or collaborate in pairs
  • hand in your own assignment and list collaborators
• 3 late days allowed (no permission required)
  • but no more than 2 late days for a particular assignment
• no solutions posted online
  • come to office hours to see solutions
• submit solutions to Crowdmark

Homework 1

• no late days, no collaborators
• hints at required background
• due next week

Resources

• see the course home page for schedule
• announcements and discussions on Piazza
• TA and instructor office hours start week 2