We wish to consider a special type of optimization problem:
Find the maximum (or minimum) of a function subject to the condition
If it is possible to solve for so that it is expressed explicitly as , by substituting in (1), it becomes
Find the maximum (or minimum) of a single variable function .
In the case that can not be obtained from solving , we re-state the problem as:
Find the maximum (or minimum) of a single variable function where is a function of , implicitly defined by
Following the traditional procedure of finding the maximum (or minimum) of a single variable function, we differentiate with respect to :
By grouping and (3), we have
The fact that at any stationary point means for all where ,
If then from (4),
Substitute it into (6),
Let , we have
Combining (7) and (8) gives
It follows that to find the stionary points of , we solve
for and .
This is known as the method of Lagrange’s multiplier.
(9) is equivalent to
Let’s look at some examples.
Example-1 Find the minimum of subject to the condition that
for gives .
, we have
The target function with constraint indeed attains its global minimum at .
I first encountered this problem during junior high school and solved it:
I solved it again in high school when quadratic equation is discussed:
In my freshman calculus class, I solved it yet again:
Example-2 Find the shortest distance from the point to the parabola .
We minimize where .
If we eliminate in , then . Solving gives , Clearly, this is not valid for it would suggest that from , an absurdity.
By Lagrange’s method, we solve
The only valid solution is . At . It is the global minimum:
Example-3 Find the shortest distance from the point to the line .
We want find a point on the line so that the distance between and is minimal.
To this end, we minimize where (see Fig. 2)
We found that
and the distance between and is
To show that (11) is the minimal distance, .
Let , we have
By the fact that , we have
Compute (see Fig. 3)
After some rearrangement and factoring, it becomes
By (12), it reduces to
This is clearly a positive quantity. Therefore,