VC.03 The Gradient
TUTORIALS

\(Off[General::spell]; 
Off[General::spell1]; 

    \(Off[ParametricPlot3D::ppcom]; \)\)Off[ParametricPlot::ppcom]; Off[Plot::plnr]; SetOptions[Limit, Analytic -> True];

CMView = {2.7, 1.6, 1.2};

Vector Drawer

BeginPackage["Vector3D`", "Graphics`Colors`"];

\(Axes3D::"usage" = 
      "Axes3D[a, b] creates a Graphics3D object of cartesian axes with x, y, and z running from -a/3 to a, and with axes labels b units beyond the tips of the axes.  Axes3D[a] is Axes3D[a, a/8]."; 

    \(Perpend::"usage" = 
      "Perpend[a] returns a unit vector which is perpendicular to the vector a."; \)\)Arrow::"usage" = 
      "Arrow[a] produces a 2 or 3 dimensional vector from the origin to a. \
Arrow[a, Tail -> tail] gives a vector from tail to a."; Vector::"usage" = 
      "Vector[a, b] gives a 2D or 3D vector from a to b."; ArrowHead::"usage" = 
      "ArrowHead[a, vec] produces an arrowhead with its tip placed at the \
point a, pointing in the direction of the vector vec."; Tail::"usage" = 
      "Tail -> point puts the tail of the vector at point."; Norm::"usage" = 
      "Norm[vec] returns the Euclidean length of vec."; Normalize::"usage" = 
      "Normalize[vec] returns a unit vector in the direction of vec, \
provided vec is nonzero."; Aperture::"usage" = 
      "Aperture is the ratio of the radius of the base to the length of the \
head of a vector."; TipSize::"usage" = 
      "TipSize specifies an absolute size for the inner tip length of the \
head of a vector."; TipRatio::"usage" = 
      "TipRatio is the ratio of the inner length of the arrowhead to the \
length of the shaft of a vector."; HeadRatio::"usage" = 
      "HeadRatio is the ratio of the outer length of the arrowhead to the \
length of the shaft of a vector."; HeadSize::"usage" = 
      "HeadSize specifies an absolute size for the head of a vector."; 
    ScaleFactor::"usage" = 
      "ScaleFactor specifies the amount to scale a vector in length and may \
be either a number or a pure function."; ZeroVectorPointSize::"usage" = 
      "ZeroVectorPointSize is the size of the point used to represent a \
zero vector."; Options[Vector] = 
      Options[Arrow] = 
        Options[ArrowHead] = {HeadRatio -> 0.18, 
            TipRatio -> 0.14, Aperture -> 0.3, 
            ScaleFactor -> 1, HeadType -> Polygon, 
            ShaftQ -> True, EdgesQ -> True, 
            VectorColor -> RGBColor[0, 0, 1], ShaftWidth -> 0.005, 
            ZeroVectorPointSize -> 0.01}; Begin["`Private`"]; SetOptions[ParametricPlot, AspectRatio -> Automatic]; SetOptions[Plot, AspectRatio -> Automatic]; SetOptions[Graphics, AspectRatio -> Automatic]; Axes3D[u_, v_] := 
      Graphics3D[{{Blue, Line[{{-(u/3), 0, 0}, {u, 0, 0}}]}, 
          Text["x", {u + v, 0, 0}], {Blue, 
            Line[{{0, -(u/3), 0}, {0, u, 0}}]}, 
          Text["y", {0, u + v, 0}], {Blue, 
            Line[{{0, 0, -(u/3)}, {0, 0, u}}]}, 
          Text["z", {0, 0, u + v}]}]; Axes3D[u_] := Axes3D[u, (u/8)]; Norm[a_?VectorQ] := N[(Sqrt[a . a])]; Normalize[a_?VectorQ] := If[a . a == 0, a, (a/Norm)[a]]; Perpend[a_?VectorQ] := 
      Normalize[{a[[2]], 
            -a[[1]]}] /; 
        Length[a] == 2Perpend[a_?VectorQ] := 
      Normalize[
          If[a[[1]] == 0, {1, 0, 0}, {0, 
              a[[3]], 
              -a[[2]]}]] /; 
        Length[a] == 3Base3D = 
      Table[{Cos[((2 k π)/8.) ], Sin[((2 k π)/8.) ]}, {k, 0, 
          8}]; Vector[from_?VectorQ, to_?VectorQ, opts___] := 
      Module[{sq, sw, ht, hs, hr, tr, ap, sf, vc, zvps, diff = to - from, 
            tip}, {sq, sw, ht, hs, hr, ts, tr, ap, sf, vc, zvps} = 
            {ShaftQ, ShaftWidth, HeadType, HeadSize, HeadRatio, TipSize, 
                  TipRatio, Aperture, ScaleFactor, VectorColor, 
                  ZeroVectorPointSize} /. {opts} /. Options[Vector]; 
          If[from == to, 
            Return[Graphics[{PointSize[zvps], vc, Point[from]}]]]; 
          diff = If[NumberQ[N[sf]], sf diff, sf[diff]]; tip = from + diff; 
          If[NumberQ[hs], hr = (hs/Norm)[diff]; tr = 0.75 hr]; 
          If[NumberQ[ts], tr = (ts/Norm)[diff]]; 
          Graphics[{
              If[sq, {vc, Thickness[sw], Line[{from, tip - tr diff}]}, {}], 
              With[{trans = ap hr Norm[diff] Perpend[diff]}, {vc, 
                  Thickness[sw], 
                  ht[{tip, tip - hr diff + trans, tip - tr diff, 
                      tip - hr diff - trans, tip}]}]}]] /; 
        Length[from] == Length[to] == 2Vector[from_?VectorQ, to_?VectorQ, opts___] := 
      Module[{sq, eq, sw, hs, hr, tr, ap, sf, vc, zvps, diff = to - from, 
            tip}, {sq, eq, sw, hs, hr, ap, sf, vc, zvps} = 
            {ShaftQ, EdgesQ, ShaftWidth, HeadSize, HeadRatio, Aperture, 
                  ScaleFactor, VectorColor, ZeroVectorPointSize} /. {opts}
                   /. Options[Vector]; 
          If[from == to, 
            Return[Graphics3D[{PointSize[zvps], vc, Point[from]}]]]; 
          diff = If[NumberQ[N[sf]], sf diff, sf[diff]]; tip = from + diff; 
          If[NumberQ[hs], hr = (hs/Norm)[diff]]; tr = (hr/2); 
          Graphics3D[{
              If[sq, {vc, Thickness[sw], Line[{from, tip - tr diff}]}, {}], 
              If[eq, {}, EdgeForm[]], {SurfaceColor[vc], 
                With[{temp = Perpend[diff]}, 
                  (Polygon[Append[#1, tip]]&)/@
                    Partition[
                      (tip - hr diff + #1&)/@
                        ((ap hr Norm[diff] Base3D) . {temp, 
                              Normalize[temp\[Cross]diff]}), 2, 1]]}}]] /; 
        Length[from] == Length[to] == 3Arrow[a_, Tail -> b_, opts___] := Vector[b, a + b, opts]Arrow[a_, opts___] := Vector[Table[0, {Length[a]}], a, opts]\(ArrowHead[a_, b_, opts___] := 
      Vector[a - b, a, ShaftQ -> False, opts]; 

End[]; 

    \(EndPackage[]; \)\)

Graphics`Colors`GosiaGreen = RGBColor[0,  0.392187,  0];

T.1) The total differential

If you are given a function f[x] of one variable, then you can write the differential
       df=f'[x] dx.
This is really just a suggestive notation, and it is handy for hand calculations:
-> You divide both sides of
       df=f'[x] dx
by dt to get
     d(f/d)t=f'[x]d(x/d)t.
Interpret this as
       df[x[t]]/dt=f'[x[t]]dx[t]/dt=f'[x[t]] x'[t].
This is the chain rule for functions of one variable.

T.1.a.i)

How do you carry off this stunt for a function f[x, y]?

Very easily.
Just write
       df = 
        D[f[x, y], x] dx +  
          D[f[x, y], y] dy.
Divide both sides by dt to get
       d(f/d)t=D[f[x, y], x]d(x/d)t+ D[f[x, y], y]d(y/d)t.
This gives you the chain rule:

\(Clear[f, x, y, t]; 
D[f[x[t], y[t]], \ t]\)

Remember 
       D[f[x, y], x] = (f^((1, 0)))[x, y] 
and  
       D[f[x, y], y] = (f^((0, 1)))[x, y].
It pays to notice once again that 
       d(f/d)t=D[f[x, y], x]d(x/d)t+ D[f[x, y], y]d(y/d)t
is nothing more than 
     gradf[x, y].{x'[t],y'[t]}, 
which is the chain rule.

T.1.a.ii)

Use the total differential
       df = 
        D[f[x, y], x] dx +  
          D[f[x, y], y] dy 
to help do a hand calculation of 
       D[Sin[x[t]^2 y[t]], t].

Answer by hand calculation:

Take f[x, y] = Sin[(x^2) y].
So 
     df = 
        (Cos[(x^2) y] 2  x y) dx +  
          (Cos[(x^2) y] (x^2)) dy.
Divide both sides by dt to get:
     d(f/d)t=(Cos[(x^2) y] 2  x y)d(x/d)t+ (Cos[(x^2) y] (x^2))d(y/d)t.
Interpret the result as
       df[x[t], y[t]]/dt=(Cos[x[t]^2 y[t]] 2  x[t] y[t])dx[t]/dt+
		                                    (Cos[x[t]^2 y[t]] x[t]^2)dy[t]/dt
This is in agreement with:

\(Clear[x, y, t]; 
Expand[D[Sin[x[t]\^2\ y[t]], \ t]]\)

This is a nice hand technique that's especially handy when you are working with a pencil on the back of an old envelope.

T.1.b)

Use the total differential to give a hand derivation of the product rule
           D[x[t] y[t] z[t], t]=x'[t] y[t] z[t]+ x[t] y'[t] z[t]+ 
                                                        x[t] y[t] z'[t]

Put 
       f[x, y, z] = x y z.
This gives 
       df = 
        y z dx +  x z dy +  
          x y dz.
Divide both sides by dt to get
       d(f/d)t=y z d(x/d)t+ x zd(y/d)t+ x yd(z/d)t.
Interpret this as
       d (((x[t] y[t] z[t]))/d)t 
       =y[t] z[t] x'[t]+ x[t] z[t] y'[t]+ x[t] y[t] z'[t]
       =x'[t] y[t] z[t]+ x[t] y'[t] z[t]+ x[t] y[t] z'[t].
Check:

\(Clear[x, y, z, t]; 
D[x[t]\ y[t]\ z[t], \ t]\)

Yep.

T.1.c)

Use the total differential to help give a formula for the derivative with respect to t of 
       g[t] = (∫_(Cos[t])^(Sin[t]))h[s] ds
where h[s] is an unspecified function.

Put 
       f[x, y] = (∫_y^x) h[s] ds.
The total differential is
       df = 
        D[f[x, y], x] dx +  
          D[f[x, y], y] dy 
        = h[x] dx -  h[y] dy.
Divide through by dt to get
       d(f/d)t=h[x]d(x/d)t- h[y]d(y/d)t.
Now put 
       x = Sin[t] and y = Cos[t] 
to get
       D[ (∫_(Cos[t])^(Sin[t]))h[s] ds, t] = 
        h[Sin[t]] Cos[t] -  h[Cos[t]] (-Sin[t])
        = h[Sin[t]] Cos[t] +  h[Cos[t]] Sin[t].
Would you have gotten it right without using the total differential?
Probably not.

T.2) What's the chain rule good for?

T.2.a)

Is the chain rule formula for functions of more than one variable useful for calculating derivatives of specific functions?

Answer:

Not really.
In specific situations, the chain rule for more than one variable is not really needed because if you make the substitutions first and then differentiate, then all you need is the chain rule for functions of one variable.
Case in point:
If you want to differentiate 
       Cos[x[t] y[t]^3] 
with respect to t, you can use the chain rule for a function of one variable and the product rule (from the early part of the course) to get 
       d Cos[x[t] y[t]^3]/dt
       =-Sin[x[t] y[t]^3]d (((x[t] y[t]))/d)t
       =-Sin[x[t] y[t]^3](x[t] 3 y[t]^2 y'[t]+ x'[t] y[t]^3).
You could also use the two variable chain rule from this lesson.  But the point is that you don't really need the two variable chain rule to do this problem.

T.2.b)

If the chain rule formula for functions of more than one variable is not particularly useful for calculating derivatives of specific functions, then what the heck is it good for?

Answer:

Instead of being a great calculational tool, the chain rule for functions of more than one variable is a great theoretical tool.
Reason:
The chain rule for functions of more than one variable helps you unlock the magic of the gradient. Cases in point:
-> The chain rule gave you the basis for explaining why the gradient is perpendicular to level curves and surfaces.
-> Once you knew this, it was not a big jump to see why the gradient points in the direction of greatest initial increase. 
Should you forget about the chain rule?  
Certainly not.

T.3) The gradient and maximization and minimization:
The FindMinimum instruction

When you plot a surface, your eyes look for crest and dips. Tops of crests are the tops of the hills.  Bottoms of dips are the deepest points in depressions that could collect rain water. 
Fancy folks call the tops of crests by the name "local maxima."  The same folks call the bottoms of the dips by the name "local minima."
Most folks, fancy and down-to-earth, are interested in the maximum which is located at the top of the very highest crest. The same folks are also interested in the minimum which is located at the bottom of the deepest dip.

T.3.a.i) Tops of the crests

How do you know that if {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} sits at the top of a crest on the surface z = f[x, y], then 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}?

If you are at {(Subscript[x, 0]), (Subscript[y, 0])} and you leave {(Subscript[x, 0]), (Subscript[y, 0])} in the direction of gradf[(Subscript[x, 0]), (Subscript[y, 0]) ], then f[x, y] initially goes UP unless 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}.
The upshot:
If gradf[(Subscript[x, 0]), (Subscript[y, 0])] is not {0, 0}, then {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} cannot sit at the top of a crest on the surface z = f[x, y].  In other words, if {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} sits at the top of a crest on the surface z = f[x, y], then 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}.

T.3.a.ii) Bottoms of the dips

How do you know that if {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} sits at the bottom of a dip on the surface z = f[x, y], then 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}?

If you are at {(Subscript[x, 0]), (Subscript[y, 0])} and you leave {(Subscript[x, 0]), (Subscript[y, 0])} in the direction of-gradf[(Subscript[x, 0]), (Subscript[y, 0])], then f[x, y] initially goes DOWN unless 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}.
The upshot:
If gradf[(Subscript[x, 0]), (Subscript[y, 0])] is not {0, 0}, then {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} cannot sit at the bottom of a dip the surface z = f[x, y].  In other words, if {(Subscript[x, 0]), (Subscript[y, 0]), f[(Subscript[x, 0]), (Subscript[y, 0])]} sits at the bottom of a dip on the surface z = f[x, y], then 
       gradf[(Subscript[x, 0]), (Subscript[y, 0])] = {0, 0}.

T.3.b.i) Setting the gradient equal to 0

Find the maximizers and minimizers (if any) of
       f[x, y] = 2.1  (x^2) +  5.9 (y^2) -  7.2  x +  3.3  y +  8.7.

Look at the formula
       f[x, y] = 2.1  (x^2) +  5.9 (y^2) -  7.2  x +  3.3  y +  8.7. 
For large |x| and |y|, the dominant terms 
       2.1  (x^2) +  5.9 (y^2)
make f[x, y] really huge.  This means f[x, y] has no maximum value.  It also means that there is no way for f[x, y] to ever get near -∞.  So f[x, y] has a minimum (at the bottom of the deepest dip) and the minimizer must be a point at which gradf[x, y] = 0:

\(Clear[x, y, f, gradf]; 

    f[x_, y_] = 2.1\ x\^2 + 5.9\ y\^2 - 7.2\ x + 3.3\ y + 8.7; 

    gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}; 


    Solve[gradf[x, y] == {0, 0}, {x, y}]\)

The minimizer is {1.71429, -0.279661} and the minimum value is f[1.71429, -0.279661]:

f@@{1.71429, -0.279661}

Take a look:

deepestdip = {1.71429, -0.279661, 2.06713}; flagpole = 
      Graphics3D[{Red, Thickness[0.01], 
          Line[{deepestdip, {1.7, -0.3, 12}}]}]; \(surface = 
      Plot3D[f[x, y], {x, 1.7 - 1, 1.7 + 1}, {y, \(-0.3\) - 1, \(-0.3\) + 1}, 
        DisplayFunction -> Identity]; 


    Show[surface, flagpole, AxesLabel -> {"x", "y", "z"}, 
      ViewPoint -> CMView, DisplayFunction -> $DisplayFunction]; 
    \)

The flagpole is planted at the bottom of the deepest dip.

T.3.b.ii) More than one candidate

Find the maximizers and minimizers (if any) of
       f[x, y] = 0.2  (x^4) +  0.1  (y^4) +  8.2  x y -  18.4  x

Look at the formula
       f[x, y] = 0.2  (x^4) +  0.1  (y^4) +  8.2  x y -  18.4  x 
For large |x| and |y|, the dominant terms 0.2  (x^4) +  0.1  (y^4) make f[x, y] really huge. This means f[x, y] has no maximum. It also means that there is no way for f[x, y] to ever get near -∞; so f[x, y] has a minimum and the minimizer must be a point at which   
       gradf[x, y] = 0:

\(Clear[x, y, f, gradf]; 

    f[x_, y_] = 0.2\ x\^4 + 0.1\ y\^4 + 8.2\ x\ y - 18.4\ x; 

    gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}; 


    Solve[gradf[x, y] == {0, 0}, {x, y}]\)

Tossing out the complex candidates gives three candidates for the minimizer:

candidate1 = {4.07802, -4.37255}; candidate2 = {-0.564143, 2.26142}; candidate3 = {-2.45864, 3.69388};

Compare:

{f@@candidate1, f@@candidate2, f@@candidate3}

It's no contest. The minimizer is:

candidate1

The minimum value is:

f@@candidate1

Not much to it.

T.3.c.i) Terrible gradients and the FindMinimum instruction

Find the maximizers and minimizers (if any) of
       f[x, y] = 1.2  (e^((x^2))) +  2.9  (e^((y^2))) -  8.2  x (y^2).
Show off your results with a plot.

Look at the formula
       f[x, y] = 1.2  (e^((x^2))) +  2.9  (e^((y^2))) -  8.2  x (y^2).
For large |x| and |y|, the dominant terms 
       1.2  (e^((x^2))) +  2.9  (e^((y^2)))
make f[x, y] hugely positive. This means f[x, y] has no maximum. It also means that there is no way for f[x, y] to ever get near -∞; so f[x, y] has a minimum value and the minimizer must be a point at which gradf[x, y] = 0:

\(Clear[f, gradf, x, y]; 

    f[x_, y_] = 1.2\ E\^\(x\^2\) + 2.9\ E\^\(y\^2\) - 8.2\ x\ y\^2; 

    gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}\)

Solve[gradf[x, y] == {0, 0}, {x, y}]

Dung. 
Mathematica had to bail out.
But this is no reason for you to give up. Take another look at the formula:

f[x, y]

For the smaller values of |x| and |y|, the term 8.2  x (y^2) has a lot of influence. Also, to drive f[x, y] down, you'll want x > 0.
Look at plot of some level curves of f[x, y] for the smaller values of |x| and |y| with x > 0:

ContourPlot[f[x, y], {x, 0, 2}, {y, -2, 2}];

The minimizer(s) lurk inside dark zone.  Look again:

ContourPlot[f[x, y], {x, 0, 1.5}, {y, -1, 1}];

This is evidence pointing to two minimizers.  Look at the formula:

f[x, y]

And look at:

f[x, -y]

Aha!
       f[x, y] = f[x, -y]. 
This tells you that if you locate one minimizer at  \({\(x\^*\), \(y\^*\)}\),
then \({\(x\^*\), \(-\(y\^*\)\)}\) is also a minimizer.
Look high:

ContourPlot[f[x, y], {x, 0.8, 1.3}, {y, 0.7, 1.2}];

You've got one of the minimizers trapped!
It's somewhere in the dark zone near {1.05, 1.1}. Now move in for the kill with the Mathematica instruction FindMinimum starting with an initial guess of {1.05, 1.1}:

minimizer = FindMinimum[f[x, y], {x, 1.05}, {y, 1.1}]

Mathematica is telling you that it has located a bottom of a dip at the point
       {1.12131, 1.07421, 2.80399}.
Test the gradient at {1.12131, 1.07421}:

gradf@@{1.12131, 1.07421}

Great; it's darn close to {0, 0}.
Now you can say with considerable authority that your estimate is that the minimizers are 
       {1.12131, 1.07421} and  {1.12131, -1.07421}
and the minimum value of f[x, y] is:

f@@{1.12131, 1.07421}

f@@{1.12131, -1.07421}

Take a look at the fruits of your labor:

h = 0.3; deepestdip1 = {1.12131, 1.07421, 2.80399}; deepestdip2 = {1.12131, -1.07421, 2.80399}; flagpoles = {
        Graphics3D[{Red, Thickness[0.01], 
            Line[{deepestdip1, {1.1, 1.07, 10}}]}], 
        Graphics3D[{Red, Thickness[0.01], 
            Line[{deepestdip2, {1.1, -1.07, 10}}]}]}; \(surface = 
      Plot3D[f[x, y], {x, 1.1 - h, 1.1 + 0.8\ h}, {y, \(-1.07\) - h, 
          1.07 + h}, PlotPoints -> {20, 15}, 
        DisplayFunction -> Identity]; 


    Show[surface, flagpoles, AxesLabel -> {"x", "y", "z"}, 
      ViewPoint -> CMView, DisplayFunction -> $DisplayFunction]; 
    \)

The two flagpoles are planted at the bottoms of the two dips which are equally deep. Here's a look from a different viewpoint:

Show[surface, flagpoles, 
      AxesLabel -> {"x", "y", "z"}, 
      ViewPoint -> {10, 0, 1}, 
      DisplayFunction -> $DisplayFunction];

T.3.c.ii) How FindMinimum works

Roughly speaking, how did the FindMinimum instruction find the minimizer in part i)?

Here're the function and its gradient.

\(Clear[f, gradf, x, y]; 

    f[x_, y_] = 1.2\ E\^\(x\^2\) + 2.9\ E\^\(y\^2\) - 8.2\ x\ y; 

    gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}\)

Starting with an initial guess at {1.05, 1.1}, the FindMinimum instruction found a minimizer at:

minimizer = {1.12131, 1.07421}

To get to this point, Mathematica tried to move from {1.05, 1.1} on a path whose tangent vectors always point with the negative gradient of f[x, y].
This is a good strategy because when you go in the direction of the negative gradient, you drive f[x, y] down.

T.3.c.iii)

Can you use FindMinimum to help to find maximizers?

Answer:

Sure.
The maximizers of f[x, y] are the minimizers of -f[x, y]. 
You can use FindMinimum[-f[x, y], {x, a}, {y, b}] to start your search at {a, b} for maximizers of f[x, y]. This time Mathematica will try to start at {a, b} and go on a path whose tangent vectors always point with the positive gradient of f[x, y].

T.3.c.iv) Pretenders versus the real thing

Is there a trick to using FindMinimum?

Sometimes using FindMinimum is as tricky as one of our former presidents who had the nickname "Tricky Dick."
To see what can happen, go with
       f[x, y] = 
        4  (e^(-((x^2) +  (y^2)))) +  
          8  (e^(-((((x - 2.5))^2) +  2 (((y - 2))^2)))) 
This function is never negative or zero, but as |x| and |y| get big, f[x, y] gets really close to 0. Consequently f[x, y] has no minimium value but has at least one maximizer. Look at a plot of some level curves:

\(Clear[f, gradf, x, y]; 

    f[x_, y_] = 
      4\ E\^\(-\((x\^2 + y\^2)\)\) + 
        8\ E\^\(-\((\((x - 2.5)\)\^2 + 2\ \((y - 2)\)\^2)\)\); 

    \(ContourPlot[f[x, y], {x, \(-3\), 3}, {y, \(-3\), 3}]; \)\)

Crests have been trapped near {0, 0} and {2, 2}.
Search for the maximizer at starting at {2, 2}:

FindMinimum[-f[x, y], {x, 2}, {y, 2}]

candidate1 = {2.49996, 1.99998}

Test it:

\(Clear[gradf]; 
gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}; 

    gradf@@candidate1\)

Fairly good.
Search for the maximizer at starting at {0, 0}:

FindMinimum[-f[x, y], {x, 0}, {y, 0}]

candidate2 = {(3.23856/(10^6)), (5.18174/(10^6))}

Test it:

gradf@@candidate2

Good.
Two maximizers?
Take a look at a plot:

surface = 
      ParametricPlot3D[{x, y, f[x, y]}, {x, -2, 4}, {y, -2, 4}, 
        ViewPoint -> CMView, PlotRange -> All, 
        AxesLabel -> {"x", "y", "z"}];

Take a look from a different viewpoint:

Show[surface, ViewPoint -> {8, -8, 0}];

There are two crests. One exhibits the true maximum value; the other is just a silly pretender sitting at the top of its own short crest. The true maximizer sits on the top of the highest crest, master of all.

Compare:

{f@@candidate1, f@@candidate2}

The maximizer is:

candidate1

The maximum value is 8.0014.
The trick to using FindMinimum[g[x, y], {x, a}, {y, b}] is to pick a starting point {a, b} close enough to the true minimizer of g[x, y].
Sometimes this isn't as easy you might wish.

T.3.c.v)

Can you use FindMinimum for functions of more than two variables?

Yep.

T.4) Eye-balling a function for max-min

T.4.a)

Does 
       f[x, y]=((x^4)/10)- 3 x y+ ((y^4)/7)- 8 x+ 9 y 
have a maximizer or a minimizer?

Look at the formula
       f[x, y]=((x^4)/10)- 3 x y+ ((y^4)/7)- 8 x+ 9 y.  
When |x| and |y| are large then the dominant terms 
       ((x^4)/10)+ ((y^4)/7)
make f[x, y] really huge, so the global scale plot of f[x, y] looks like a cup. This means f[x, y] has no tallest crest but does have a deepest dip.  As a result, f[x, y] has no maximum value but does have a minimum value.

T.4.b)

Does 
       f[x, y] = 
        (e^((-(x^2) - (y^2)))) ((x^6) +  7 (y^2) +  4) 
have a maximizer or a minimizer?

Look at the formula
       f[x, y] = 
        (e^((-(x^2) - (y^2)))) ((x^6) +  7 (y^2) +  4).  
When |x| and |y| are large, then f[x, y] is incredibly close to 0.
Also f[x, y] > 0 for all x's and y's.  As a result this the surface z = f[x, y] has a biggest crest, but no dip below 0.
Consequently f[x, y] has a maximizer but no minimizer.

T.4.c)

Does 
       f[x, y] = (((x^3)  +  7 (y^2))/(1  +  (x^4)  +  (y^6))) 
have a maximizer or a minimizer?

Look at the formula
       f[x, y] = (((x^3)  +  7 (y^2))/(1  +  (x^4)  +  (y^6))).  
When |x| and |y| are large, then the dominant terms in the domoninator make f[x, y] incredibly close to 0.
Also f[x, 0] > 0 for x > 0 and f[x, 0] < 0 for x < 0. 
Now you know that f[x, y] has positive and negative values and f[x, y] is arbitrarily close to 0 as |x| and |y| become large.  Consequently the surface z = f[x, y] has a biggest crest and a deepest dip. This means f[x, y] has both a maximizer and a minimizer.

T.5) Data fit

One of the central uses of minimization is to try to fit data by curves.  Usually this involves the idea of least squares, an idea you have seen earlier in the course.  In fact, a lot of this problem might be old hat to some of you.

T.5.a.i)

Given a list of data points 
       {{(Subscript[a, 1]), (Subscript[b, 1])}, {(Subscript[a, 2]), (Subscript[b, 2])}, {(Subscript[a, 3]), (Subscript[b, 3])}, …, {(Subscript[a, n]), (Subscript[b, n])}}
        , 
you reach into your box of functions, pick a few, say f[x], g[x], and h[x] and then you form the square error function
       sqerror[r, s, t] = 
        (((r f[(Subscript[a, 1])] +  s g[(Subscript[a, 1])] +  t h[(Subscript[a, 1])] -  (Subscript[b, 1])))^2)
                                      + 
        (((r f[(Subscript[a, 2])] +  s g[(Subscript[a, 2])] +  t h[(Subscript[a, 2])] -  (Subscript[b, 2])))^2)
                                      + 
        (((r f[(Subscript[a, 3])] +  s g[(Subscript[a, 3])] +  t h[(Subscript[a, 3])] -  (Subscript[b, 3])))^2)
                                      + …
                                      + 
        (((r f[(Subscript[a, n])] +  s g[(Subscript[a, n])] +  t h[(Subscript[a, n])] -  (Subscript[b, n])))^2).
Then you minimize sqerror[r, s, t] and assess the resulting fit.
(As |r|, |s|, and |t| get really large, sqerror[r, s, t] also gets really large; so you are guaranteed that sqerror[r, s, t] has a minimizer.)
Here are some data to work with:

data = {{-3. , -0.1119}, {-2.5, -0.3986}, {-2. , 
          -1.7359}, {-1.5, -2.6926}, {-1. , -1.6163}, {
          -0.5, -0.1308}, {0. , 0.0570}, {0.5, 0.2171}, {1. , 
          1.8582}, {1.5, 2.6742}, {2. , 1.8313}, {2.5, 0.7550}, {3. , 0.2831}}

And a plot:

dataplot = ListPlot[data, PlotStyle -> {PointSize[0.02], Red}];

Try to find a function whose plot runs close to all these points.

Looks like points picked off the plot of an odd function plotted from -π to π.
Fourier (1768-1830) revolutionized mathematics with the idea that all odd functions plotted on [-π, π] are made up of combinations of Sin[x], Sin[2  x], Sin[3  x],…
See whether Fourier was right. 
Reach into the box of functions and pull out 
       Sin[x], Sin[2  x], and Sin[3  x].
Try to fit these data with a function of the form
       r Sin[x] +  s Sin[2  x] +  t Sin[3  x],
where r, s, and t are to selected to give the best possible fit.

\(Clear[fitter, sqerror, gradsqerror, s, t, r, x, k]; 

    fitter[x_] = r\ Sin[x] + s\ Sin[2\ x] + t\ Sin[3\ x]\)

Now define the square error function and minimize it be setting its gradient equal to 0.

sqerror[r_, s_, t_] = 
      (∑_(k = 1)^(Length[data]))(((
            fitter[data[[k, 1]]] - 
              data[[k, 2]]))^2); \(gradsqerror[r_, s_, t_] = {D[sqerror[r, s, t], \ r], 
        D[sqerror[r, s, t], \ s], D[sqerror[r, s, t], \ t]}; 

    Solve[gradsqerror[r, s, t] == {0, 0, 0}]\)

Here is the best you can do with 
       Sin[x], Sin[2  x], and Sin[3  x]:

\(Clear[bestfit]; 

    bestfit[x_] = 
      fitter[x] /. {r -> 1.97845, s -> 0.0475817, 
          t -> \(-0.663794\)}\)

Assess with a plot:

\(fitplot = 
      Plot[bestfit[x], {x, \(-3\), 3}, 
        PlotStyle -> {{Thickness[0.01], RGBColor[0, 0, 1]}}, 
        DisplayFunction -> Identity]; 

Show[dataplot, fitplot]; \)

This is the best you can do with any function of the form:

fitter[x]

And this is not bad!
Maybe you could have done better by choosing different functions in place of Sin[x], Sin[2  x], Sin[3  x].
Maybe you could have done better by choosing more functions in addition to Sin[x], Sin[2  x], Sin[3  x]. 
Try it and see.

T.5.a.ii) The Fit instruction

Do you have to endure the pain of all this typing every time you try to fit data?

No.
The Fit instruction in Mathematica will do this for you.
Here is what you did above done with the Fit instruction:

data = {{-3. , -0.1119}, {-2.5, -0.3986}, {-2. , 
          -1.7359}, {-1.5, -2.6926}, {-1. , -1.6163}, {
          -0.5, -0.1308}, {0. , 0.0570}, {0.5, 0.2171}, {1. , 
          1.8582}, {1.5, 2.6742}, {2. , 1.8313}, {2.5, 0.7550}, {3. , 
          0.2831}}; quickbestfit[x_] = Fit[data, {Sin[x], Sin[2 x], Sin[3 x]}, x]

Compare with what you got in the last part:

bestfit[x]

Exactly the same!
Thank you Stephen Wolfram and Mathematica.

T.5.b.i) C&M students work with engineeering students

The Calculus&Mathematica gang is doing a joint experiment with some engineers, not all of whom turn out to be the nerds you had expected. 
In this joint experiment, the group starts with a heated wire π units long with the temperature allowed to vary from position to position on the wire. As math people, the C&M gang thinks of the wire as the interval 0 ≤ x ≤ π.
At the start of the experiment, the engineers instantly cool the ends at x = 0 and x = π  and maintain these ends at temperature 0. 
At the same time, the engineers take pains to guarantee that the rest of the wire is perfectly insulated.
The engineers check their handbook and tell you that they have adjusted the time unit so that you should be able to get a good fit of the actual data by a function of the form
       temp[x, t] = 
        (Subscript[a, 1]) (e^(-t)) Sin[x] +  
          (Subscript[a, 2]) (e^(-4 t)) Sin[2  x] +  
          (Subscript[a, 3]) (e^(-9 t)) Sin[3  x] + …
where you can use as many terms as you need to get a good fit.    
The engineers carefully make some measurements on the wire reporting 
       {x, t, temp[x, t]} 
for x = 1, 1.5, 2, 2.5, 3 and t = 1, 2, 3.
Here they are:

measurements = {{1. , 0. , 8.87}, {1. , 1. , 3.54}, {1. , 2. , 
          1.34}, {1. , 3. , 0.50}, {1.5, 0. , 11.43}, {1.5, 1. , 4.35}, {1.5, 
          2. , 1.61}, {1.5, 3. , 0.60}, {2. , 0. , 11.74}, {2. , 1. , 4.16}, {
          2. , 2. , 1.5}, {2. , 3. , 0.55}, {2.5, 0. , 9.04}, {2.5, 1. , 
          2.86}, {2.5, 2. , 1. }, {2.5, 3. , 0.36}, {3. , 0. , 2.57}, {3. , 
          1. , 0.69}, {3. , 2. , 0.24}, {3. , 3. , 0.09}};

Your part of the experiment is to try to fit these measurements by a function temp[x, t] of the form given above. 
Do it and assess the quality of the fit.

Try 
       temp[x, t] = 
        (Subscript[a, 1]) (e^(-t)) Sin[x] +  
          (Subscript[a, 2]) (e^(-4 t)) Sin[2  x] +  
          (Subscript[a, 3]) (e^(-9 t)) Sin[3  x] + …
with 6 terms:

measurements = {{1. , 0. , 8.87}, {1. , 1. , 3.54}, {1. , 2. , 
          1.34}, {1. , 3. , 0.50}, {1.5, 0. , 11.43}, {1.5, 1. , 4.35}, {1.5, 
          2. , 1.61}, {1.5, 3. , 0.60}, {2. , 0. , 11.74}, {2. , 1. , 4.16}, {
          2. , 2. , 1.5}, {2. , 3. , 0.55}, {2.5, 0. , 9.04}, {2.5, 1. , 
          2.86}, {2.5, 2. , 1. }, {2.5, 3. , 0.36}, {3. , 0. , 2.57}, {3. , 
          1. , 0.69}, {3. , 2. , 0.24}, {3. , 3. , 0.09}}; \(Clear[fittemp]; 

    fittemp[x_, t_] = 
      Fit[measurements, {E\^\(-t\)\ Sin[x], E\^\(\(-4\)\ t\)\ Sin[2\ x], 
          E\^\(\(-9\)\ t\)\ Sin[3\ x], E\^\(\(-16\)\ t\)\ Sin[4\ x], 
          E\^\(\(-25\)\ t\)\ Sin[5\ x], E\^\(\(-36\)\ t\)\ Sin[6\ x]}, {x, t}]
        \)

Check the fit:
Here are the entries in the {x, t} slots of the table of measurements:

Table[{measurements[[k, 1]], 
        measurements[[k, 2]]}, {k, 1, 
        Length[measurements]}]

The discrepancies are:

Table[
      fittemp@@{measurements[[k, 1]], 
            measurements[[k, 2]]} - 
        measurements[[k, 3]], {k, 1, 
        Length[measurements]}]

Hot Dog! 
The fit is good all the way!

T.5.b.ii)

The engineers ask for a plot displaying your estimate of the temperature of the wire x units from the left at time t = 0.
And they ask for a plot displaying your estimate of the temperature of the wire x units from the left at time t = 2.
Give them the plots.

That's cake.
Here's the plot for t = 0:

Plot[fittemp[x, 0], {x, 0, π}, 
      PlotStyle -> {{Blue, Thickness[0.01]}}, 
      AxesLabel -> {"x", "temp"}, PlotLabel -> "t = 0", 
      PlotRange -> {0, 13}, AspectRatio -> (1/2)];

When t = 0, the wire was hottest slightly to the right of center.
Here comes the plot for t = 2 with the same plot range as above:

Plot[fittemp[x, 2], {x, 0, π}, 
      PlotStyle -> {{Blue, Thickness[0.007]}}, 
      AxesLabel -> {"x", "temp"}, PlotLabel -> "t = 2", 
      PlotRange -> {0, 13}, AspectRatio -> (1/2)];

Cooled off quite a bit.

T.5.b.iii)

The engineers ask for a movie that shows how the wire cools as t advances from 0.1 to 5.1.

No problem-o:

Table[
      Plot[fittemp[x, t], {x, 0, π}, 
        PlotStyle -> {{Blue, Thickness[0.01]}}, 
        AxesLabel -> {"x", "temp"}, 
        PlotLabel -> t "= t", PlotRange -> {0, 13}, 
        AspectRatio -> (1/2)]; , {t, 0.1, 5.1}];

T.5.b.iv)

One of the Calculus&Mathematica crew gets the idea to look at

Plot3D[fittemp[x, t], {x, 0, π}, {t, 0, 5}, 
      AxesLabel -> {"x", "t", "temp"}, 
      PlotRange -> All, ViewPoint -> CMView];

Interpret the plot.

You can see the initial temperature plot at the face where t = 0 and you can see that the entire wire is fairly cold by the time t gets to 3.

T.6) Lagrange's method for constrained maximization and minimization

Lagrange's method deals with maximizing or minimizing one function while another function is held constant. Once you are armed with the fact that the gradient gives you a normal vector, understanding it is pretty easy.

T.6.a.i)

If {(Subscript[x, 0]), (Subscript[y, 0])} is on the level curve g[x, y] = c and {(Subscript[x, 0]), (Subscript[y, 0])} maximizes or minimizes f[x, y] given g[x, y] = c, then how do you know that the level curve f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] is tangent to the level curve g[x, y] = c at the point {(Subscript[x, 0]), (Subscript[y, 0])}?

Suppose {(Subscript[x, 0]), (Subscript[y, 0])} is a point on the level curve g[x, y] = c and  the level curve f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] crosses g[x, y] = c without being tangent to g[x, y] = c. Like this:

Clear[x]thick = 0.01; fcurve = 
      Plot[Sin[x]/2, {x, (π/8), ((3 π)/8)}, 
        PlotStyle -> {{Thickness[thick], RGBColor[1, 0, 0]}}, 
        DisplayFunction -> Identity]; gcurve = 
      Plot[Cos[x]/2, {x, (π/8), ((3 π)/8)}, 
        PlotStyle -> {{Thickness[thick], RGBColor[0, 0, 1]}}, 
        DisplayFunction -> Identity]; crosspoint = 
      Graphics[PointSize[4 thick], Point[{(π/4), (1/(2 (Sqrt[2])))}]]; labels = {
        Graphics[
          Text[{"x0", "y0"}, {(π/4), (1/(2 (Sqrt[2])))}, {-1.5, 
              0}]], Graphics[
          Text["f[x,y] = f[x0,y0]", {((3 π)/16) + 0.1, 
              (1/2) Sin[((3 π)/16)]}, {0.7, 0}]], 
        Graphics[
          Text["g[x,y] = c", {((5 π)/16) + 0.1, 
              (1/2) Sin[((3 π)/16)]}, {0.7, 0}]]}; Show[fcurve, gcurve, crosspoint, labels, 
      DisplayFunction -> $DisplayFunction, Axes -> None];

The first thing you can say is that f[x, y] does not stay equal to  f[(Subscript[x, 0]), (Subscript[y, 0])] on the level curve g[x, y] = c.  
This means that you can leave {(Subscript[x, 0]), (Subscript[y, 0])} on the level curve g[x, y] = c and go in one direction to lower the value of f[x, y] and you can leave {(Subscript[x, 0]), (Subscript[y, 0])} on the level curve g[x, y] = c and go in the other direction to raise the value of f[x, y].
Consequently: 
If the level curve f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] crosses g[x, y] = c without being tangent to g[x, y] = c, then f[(Subscript[x, 0]), (Subscript[y, 0])] is neither the maximum nor the minimum value of f[x, y] given g[x, y] = c.
As a result, if {(Subscript[x, 0]), (Subscript[y, 0])} maximizes or minimizes f[x, y] given g[x, y] = c,  
then level curve 
       f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] 
is tangent to the level curve 
       g[x, y] = c at the point {(Subscript[x, 0]), (Subscript[y, 0])}.

T.6.a.ii)

If {(Subscript[x, 0]), (Subscript[y, 0])} maximizes or minimizes f[x, y] given g[x, y] = c, you know that the level curve f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] is tangent to the level curve g[x, y] = c at the point {(Subscript[x, 0]), (Subscript[y, 0])}.
Why does this force
     gradf[(Subscript[x, 0]), (Subscript[y, 0])] = s gradg[(Subscript[x, 0]), (Subscript[y, 0])]
for some number s, except in the case that gradg[(Subscript[x, 0]), (Subscript[y, 0])] = 0?

-> The tangent to the level curve f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] must be parallel to the tangent to g[x, y] = c at {(Subscript[x, 0]), (Subscript[y, 0])}. 
So any normal vector to the level curve 
       f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] at {(Subscript[x, 0]), (Subscript[y, 0])} 
must be parallel to any normal vector of 
       g[x, y] = c at {(Subscript[x, 0]), (Subscript[y, 0])}.
-> A normal vector to the level curve 
       f[x, y] = f[(Subscript[x, 0]), (Subscript[y, 0])] at {(Subscript[x, 0]), (Subscript[y, 0])} is gradf[(Subscript[x, 0]), (Subscript[y, 0])].
-> A normal vector to the level curve 
       g[x, y] = c at {(Subscript[x, 0]), (Subscript[y, 0])} is gradg[(Subscript[x, 0]), (Subscript[y, 0])].
Consequently gradf[(Subscript[x, 0]), (Subscript[y, 0])] is parallel to gradg[(Subscript[x, 0]), (Subscript[y, 0])].
A compact way of saying this is to say
     gradf[(Subscript[x, 0]), (Subscript[y, 0])] = s gradg[(Subscript[x, 0]), (Subscript[y, 0])]
for some number s except in the case gradg[(Subscript[x, 0]), (Subscript[y, 0])] = 0 in which case all bets are off.

The numbers s you get this way are called Lagrange multipliers by the heavy rollers.

T.6.b.i)

What is Lagrange's method of maximizing or minimizing f[x, y] given g[x, y] = c?

Lagrange's method of maximizing or minimizing 
f[x, y] given g[x, y] = c is to solve the system of equations
       gradf[x, y] = s gradg[x, y]
and 
       g[x, y] = c 
for {x, y}.
If you get all the {x, y}'s that solve this, then you are guaranteed that the true maximizers and minimizers are in this list.

T.6.b.ii)

Look at the following movie:
The area of the bar graph on the right represents (1/4) of the area of the inscribed rectangle.

\(circle = Graphics[{Red, Thickness[0.01], Circle[{0, 0}, 1]}]; 

    Clear[rectangle, t]; 

    rectangle[t_] = 
      Graphics[{Thickness[0.01], Blue, 
          Line[{{Cos[t], Sin[t]}, {\(-Cos[t]\), Sin[t]}, {\(-Cos[t]\), 
                \(-Sin[t]\)}, {Cos[t], \(-Sin[t]\)}, {Cos[t], Sin[t]}}]}]; 

    Clear[point]; 

    point[t_] = 
      Graphics[{PointSize[0.02], Point[{Cos[t], Sin[t]}]}, 
        Text[{"x", "y"}, {Cos[t] +  .2, Sin[t]}]]; 

    Clear[areameter]; 

    areameter[t_] = 
      Graphics[Polygon[{{1.5, 0}, {1.5, 2\ Sin[t]\ Cos[t]}, {2, 
              2\ Sin[t]\ Cos[t]}, {2, 0}, {1.5, 0}}]]; 

    \(Table[Show[circle, rectangle[t], point[t], areameter[t], 
        AspectRatio -> Automatic, Axes -> True, 
        AxesOrigin -> {0, 0}, Ticks -> None, 
        AxesLabel -> {"x", "y"}], {t, 0, π\/2, π\/16}]; 
    \)\)

For a given prescribed radius r, use Lagrange's method to explain why the rectangle inscribed in the circle (x^2) +  (y^2) = (r^2) with corners at {x, y}, {x, -y}, {-x, -y} and {-x, y} with the biggest possible area is a square.

Saying that {x, y} is on the circle of radius r centered at {0, 0} is to say that
       g[x, y] = (x^2) +  (y^2) = (r^2).
For a given {x, y} on this circle, the area of the inscribed rectangle is
       f[x, y] = 4  x y.
To find the largest such rectangle, you've got to maximize f[x, y] given g[x, y] = (r^2). 
This is a snap for Lagrange's method:

\(Clear[f, gradf, g, gradg, x, y, s, r]; 
f[x_, y_] = 4\ x\ y; 

    \(g[x_, y_] = x\^2 + y\^2; \)\)\(gradf[x_, y_] = {D[f[x, y], \ x], D[f[x, y], \ y]}; 

    gradg[x_, y_] = {D[g[x, y], \ x], D[g[x, y], \ y]}; 

    lagrange = 
      Solve[{gradf[x, y] == s\ gradg[x, y], g[x, y] == r\^2}, {x, y, s}]\)

All four solutions for {x, y} give a square; so the square is the largest inscribed rectangle.
This is not big news; you knew this all along.
What might be good news is the method.  It was quick and easy.

T.6.c.i)

Does Lagrange's method work finding the maximizers of f[x, y, z] given g[x, y, z] = c?
Explain.

In theory, yes.
If {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])} is a point on the level surface g[x, y, z] = c and the level surface f[x, y, z] = f[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] cuts g[x, y, z] = c at {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])} without being tangent to g[x, y, z] = c, then you can leave {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])} on the level surface g[x, y, z] = c on one side of the level surface f[x, y, z] = f[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] and raise f[x, y, z]. 
And you can leave {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])} staying on the level surface g[x, y, z] = c on the other side of f[x, y, z] = f[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] and lower f[x, y, z]. 
As a result if {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])} maximizes or minimizes f[x, y, z] subject to g[x, y, z] = c, then the level surfaces
       f[x, y, z] = f[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] and g[x, y, z] = c 
must be tangent at {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])}. 
So, as in the two variable case, their normals must be parallel at {(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])}.
This means there is a number s such that 
       gradf[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] = s gradg[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])]
except in the case gradg[(Subscript[x, 0]), (Subscript[y, 0]), (Subscript[z, 0])] = 0, in which case all bets are off.

T.6.c.ii)

Use Lagrange's method to find the maximum and minimum values (if any) of f[x, y, z] = xyz given x +  y +  z = 6 with x > 0, y > 0, z > 0.

The set up is to maximize and minimize
       f[x, y, z] = x y z given x +  y +  z = 6 with x > 0, y > 0 and z > 0.
You can push f[x, y, z] close to (but not equal to) 0 while holding x +  y +  z = 6 by making x and y small and positive and setting 
       z = 6 -  x -  y.
For x and y small and positive, this makes f[x, y, z] small and maintains the constraint x +  y +  z = 6:

\(Clear[x, y, z, f, g]; 
f[x_, y_, z_] = x\ y\ z; 

    \(g[x_, y_, z_] = x + y + z; \)\){f[x, y, 6 - x - y], g[x, y, 6 - x - y]} /. {x -> 0.01, 
        y -> 0.01}

{f[x, y, 6 - x - y], g[x, y, 6 - x - y]} /. {x -> 0.00001, 
        y -> 0.00002}

Now you see that if {x, y, z} is constrained by 
       g[x, y, z] = 6 
with x > 0, y > 0, z > 0, 
then f[x, y, z] has no minimum value.

But maximizers are another story because there is no way to drive f[x, y, z] to ∞ while maintaining the constraint 
       g[x, y, z] = x +  y +  z = 6
with x > 0, y > 0 and z > 0.

Here is how to use Lagrange's method to go after the maximizers of f[x, y, z] subject to the constraint g[x, y, z] = 6 with x > 0, y > 0 and z > 0:

\(Clear[gradf, gradg, s]; 

    gradf[x_, y_, z_] = {D[f[x, y, z], \ x], D[f[x, y, z], \ y], 
        D[f[x, y, z], \ z]}; 

    gradg[x_, y_, z_] = {D[g[x, y, z], \ x], D[g[x, y, z], \ y], 
        D[g[x, y, z], \ z]}; 

    Solve[{gradf[x, y, z] == s\ gradg[x, y, z], g[x, y, z] == 6}]\)

The candidates are:

\(Clear[candidate]; 
candidate[1] = {2, 2, 2}\)candidate[2] = {6, 0, 0}candidate[3] = {0, 6, 0}candidate[4] = {0, 0, 6}

Compare:

\(Clear[k]; 
Table[{candidate[k], f@@candidate[k]}, {k, 1, 4}]\)

The maximizer of f[x, y, z] subject to the constraint g[x, y, z] = 6  with x > 0, y > 0 and z > 0 is 
       {2, 2, 2} 
and the constrained maximum value is 8:

f[2, 2, 2]

That fellow Lagrange was no jerk.  You should like him because his work made it possible for you to take it easy.

T.6.d)

Is there a catch?

Yes.
If the Mathematica solve command can't hack the algebra, then Lagrange's method is up the creek without a paddle.