What engineering optimization means
Optimization means choosing the best values of design or decision variables while respecting constraints.
In engineering, constraints often represent limited material, time, energy, budget, force capacity, or machine capacity.
\[
\text{choose variables}
\quad\Longrightarrow\quad
\text{best objective value subject to constraints}.
\]
Linear programming
A linear programming problem has a linear objective function and linear constraints.
\[
\max Z=c_1x+c_2y.
\]
A common constraint form is
\[
a_ix+b_iy\le r_i.
\]
The calculator focuses on two-variable linear programs so the feasible region can be graphed clearly.
Objective function
The objective function is the quantity being maximized or minimized.
Examples include profit, efficiency, production output, cost, error, or weight.
\[
Z=c_1x+c_2y.
\]
For maximization, larger \(Z\) is better. For minimization, smaller \(Z\) is better.
Feasible region
The feasible region is the set of all points that satisfy every constraint.
\[
\text{feasible point}
\quad\Longleftrightarrow\quad
\text{all constraints are satisfied}.
\]
In two variables, the feasible region is usually a polygon or an unbounded polygonal region.
Why the optimum occurs at a corner
For a linear objective over a polygonal feasible region, the optimum occurs at a vertex,
unless there are multiple optimal points along an edge.
\[
\text{linear objective}
+
\text{linear constraints}
\quad\Longrightarrow\quad
\text{check feasible vertices}.
\]
This is why the calculator finds candidate corner points and evaluates the objective at each one.
Slack variables
Slack measures how much unused capacity remains in a less-than-or-equal constraint.
\[
\text{slack}
=
r_i-(a_ix+b_iy).
\]
If the slack is positive, the constraint is not fully used. If the slack is zero, the constraint is active.
Binding constraints
A binding constraint is exactly active at the optimum.
\[
a_ix^{*}+b_iy^{*}=r_i.
\]
Binding constraints are important because changing them can change the optimal objective value.
Simplex method idea
The simplex method moves from one feasible corner to another, improving the objective until no better adjacent
corner is available.
\[
\text{corner}
\longrightarrow
\text{better adjacent corner}
\longrightarrow
\text{optimal corner}.
\]
The calculator includes a basic simplex preview for standard maximization problems.
Entering and leaving variables
In simplex language, an entering variable is chosen to improve the objective.
A leaving variable is chosen by a ratio test so the new solution remains feasible.
\[
\text{ratio}
=
\frac{\text{right-hand side}}{\text{positive entering-column coefficient}}.
\]
The smallest nonnegative ratio identifies the limiting constraint.
Sensitivity analysis
Sensitivity analysis asks how the optimal solution changes when a resource limit or objective coefficient changes.
\[
\text{shadow value}
\approx
\frac{\Delta Z^{*}}{\Delta r_i}.
\]
A positive shadow value means that increasing a resource limit improves the optimal objective in a maximization problem.
Gradient descent
Gradient descent is an iterative method for reducing an objective function.
It moves opposite the gradient.
\[
\mathbf{x}_{k+1}
=
\mathbf{x}_k
-
\alpha\nabla f(\mathbf{x}_k).
\]
The parameter \(\alpha\) is the learning rate or step size.
Quadratic objective
A two-variable quadratic model has the form
\[
f(x,y)=ax^2+by^2+cxy+dx+ey+f_0.
\]
This model is useful for introducing optimization because it can form a bowl-shaped surface.
Gradient of the quadratic model
The gradient contains the partial derivatives with respect to \(x\) and \(y\).
\[
\nabla f(x,y)
=
\left(
2ax+cy+d,\;
2by+cx+e
\right).
\]
When the gradient is close to zero, the method is near a stationary point.
Positive definite quadratic
A positive definite quadratic has one bowl-shaped minimum.
\[
a>0,
\qquad
b>0,
\qquad
4ab-c^2>0.
\]
If this condition is not satisfied, the function may not have a single minimum.
Learning rate
The learning rate controls how far each descent step moves.
\[
\text{small }\alpha
\Rightarrow
\text{slow but safer movement}.
\]
\[
\text{large }\alpha
\Rightarrow
\text{faster but may overshoot}.
\]
If the descent path jumps away from the minimum, the learning rate is probably too large.
Engineering examples
- Maximize profit subject to labor and material limits.
- Minimize material cost while meeting strength requirements.
- Allocate machine time between two products.
- Choose transport quantities under capacity limits.
- Reduce error in a fitted engineering model.
- Iteratively tune a design variable using gradient descent.
Common mistakes
- Forgetting nonnegative variable constraints \(x\ge0\) and \(y\ge0\).
- Maximizing when the problem asks for minimization, or the reverse.
- Using the wrong units for the objective function.
- Ignoring binding constraints when interpreting the result.
- Assuming sensitivity estimates are valid for very large resource changes.
- Using a learning rate that is too large in gradient descent.
- Assuming gradient descent always reaches a global minimum for every function.
