> For the complete documentation index, see [llms.txt](https://windmising.gitbook.io/nielsen-nndl/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://windmising.gitbook.io/nielsen-nndl/introduction-2/qi-ta-ji-shu/1.md).

# Hessian技术

令C =C(w)，假设目标是最小化函数C\
根据[泰勒公式](https://windmising.gitbook.io/mathematics-basic-for-ml/gao-deng-shu-xue/function)得：

$$
\begin{eqnarray}
C(w+\Delta w) & = & C(w) + \sum\_j \frac{\partial C}{\partial w\_j} \Delta w\_j
\nonumber \ & & + \frac{1}{2} \sum\_{jk} \Delta w\_j \frac{\partial^2 C}{\partial w\_j
\partial w\_k} \Delta w\_k + \ldots
\tag{103} \\
& = & C(w) + \nabla C \cdot \Delta w +
\frac{1}{2} \Delta w^T H \Delta w + \ldots,
\tag{104}\end{eqnarray}
$$

只保留前三项，得到$$C(w+\Delta w)$$的近似值：

$$
\begin{eqnarray}
C(w+\Delta w) \approx C(w) + \nabla C \cdot \Delta w +
\frac{1}{2} \Delta w^T H \Delta w.
\tag{105}\end{eqnarray}
$$

当$$\Delta w = -H^{-1} \nabla C$$时，不等式右边达到最小值，也近似地认为此时左边也达到最小值。\
其中，H称为Hessian矩阵，$$H\_{jk} = \partial^2 C / \partial w\_j \partial w\_k$$。

因此，基于Hessian技术，w的更新的策略为：

$$
w \rightarrow w' = w-\eta H^{-1} \nabla C
$$

优点：只需要经过非常少的迭代就使C能达到最小值点。\
缺点：$$H^{-1} \nabla C$$计算困难。