Lagrange Multipliers Optimizer — Theory
1. Constrained optimization
In an unconstrained optimization problem, we look for maximum or minimum values of a function
\(f(x,y,z)\) freely. In a constrained optimization problem, the variables must also satisfy one
or more equations.
\[
\text{optimize } f(x,y,z)
\quad\text{subject to}\quad
g(x,y,z)=0.
\]
The constraint restricts the allowed points to a curve or surface.
2. Main idea of Lagrange multipliers
At a constrained maximum or minimum, the objective function cannot increase along the constraint.
This means the gradient of the objective is parallel to the gradient of the constraint.
\[
\nabla f=\lambda \nabla g.
\]
The number \(\lambda\) is called the Lagrange multiplier.
3. One equality constraint
If the problem is
\[
\text{optimize } f(x,y,z)
\quad\text{subject to}\quad
g(x,y,z)=0,
\]
then solve the system
\[
\begin{cases}
f_x=\lambda g_x,\\
f_y=\lambda g_y,\\
f_z=\lambda g_z,\\
g(x,y,z)=0.
\end{cases}
\]
In two variables, the \(z\)-equation is omitted.
4. Two equality constraints
With two constraints,
\[
g_1(x,y,z)=0,
\qquad
g_2(x,y,z)=0,
\]
the gradient of the objective is a linear combination of both constraint gradients:
\[
\nabla f
=
\lambda \nabla g_1
+
\mu \nabla g_2.
\]
This gives the system
\[
\begin{cases}
f_x=\lambda (g_1)_x+\mu (g_2)_x,\\
f_y=\lambda (g_1)_y+\mu (g_2)_y,\\
f_z=\lambda (g_1)_z+\mu (g_2)_z,\\
g_1(x,y,z)=0,\\
g_2(x,y,z)=0.
\end{cases}
\]
5. Why gradients matter
The gradient points in the direction of greatest increase of a function. For example,
\[
\nabla f
=
\left\langle
f_x,\ f_y,\ f_z
\right\rangle.
\]
If \(g(x,y,z)=0\) is a constraint surface, then \(\nabla g\) is normal to that surface.
At a constrained optimum, \(\nabla f\) must also point normal to the constraint surface.
6. Example: maximize \(xyz\) subject to \(x+y+z=9\)
Consider
\[
f(x,y,z)=xyz,
\qquad
g(x,y,z)=x+y+z-9=0.
\]
Compute the gradients:
\[
\nabla f
=
\langle yz,\ xz,\ xy\rangle,
\qquad
\nabla g
=
\langle 1,\ 1,\ 1\rangle.
\]
The Lagrange condition is
\[
\langle yz,\ xz,\ xy\rangle
=
\lambda
\langle 1,\ 1,\ 1\rangle.
\]
7. Solving the example
From the vector equation, we get
\[
yz=\lambda,
\qquad
xz=\lambda,
\qquad
xy=\lambda.
\]
Therefore,
\[
yz=xz=xy.
\]
Assuming \(x,y,z\neq 0\), this gives
\[
x=y=z.
\]
Using the constraint,
\[
x+y+z=9
\quad\Rightarrow\quad
3x=9
\quad\Rightarrow\quad
x=3.
\]
Thus,
\[
(x,y,z)=(3,3,3).
\]
The objective value is
\[
f(3,3,3)=3\cdot 3\cdot 3=27.
\]
8. Important warning about global extrema
Lagrange multipliers find candidate points. A candidate is not automatically a global maximum or global minimum.
To prove a global result, you may also need to check:
- whether the feasible set is closed and bounded,
- whether there are boundary points,
- whether there are inequality constraints,
- whether the objective is unbounded on the constraint set.
The calculator ranks the numerical candidates it finds inside the selected search box.
10. Common mistakes
- Forgetting the constraint equation: the system must include \(g=0\), not only \(\nabla f=\lambda\nabla g\).
- Entering the constraint incorrectly: write \(x+y+z=9\) as \(x+y+z-9=0\).
- Confusing candidates with proof: Lagrange points are candidates; they still need interpretation.
- Ignoring boundaries: inequality constraints and endpoints may require separate checking.
- Using too small a search box: numerical solvers only find candidates in the selected region.
- Using dependent constraints: if two constraints are not independent, the system can become singular.
