> For the complete documentation index, see [llms.txt](https://windmising.gitbook.io/nielsen-nndl/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://windmising.gitbook.io/nielsen-nndl/introduction-2/guo-ni-he-he-zheng-ze-hua/5.md).

# L1正则化

L1正则化的原理与L2相同，都是通过添加一个正则项来限制weights的大小。

$$
\begin{eqnarray}  C = C\_0 + \frac{\lambda}{n} \sum\_w |w|.
\tag{95}\end{eqnarray}
$$

w的偏导公式为：

$$
\begin{eqnarray}  \frac{\partial C}{\partial
w} = \frac{\partial C\_0}{\partial w} + \frac{\lambda}{n} , {\rm
sgn}(w),
\tag{96}\end{eqnarray}
$$

w的更新公式为：

$$
\begin{eqnarray}
w \rightarrow w' = w\left(1 - \frac{\eta \lambda}{n} \right) - \eta \frac{\partial C\_0}{\partial w}
\tag{98}\end{eqnarray}
$$

## L1 VS L2

1. L1的shrink大小为常数。当|w|较大时，L2 shrink更快。当|w|较小时，L1 shrink更快。  
2. L1倾向于使weights留在重要的connect上，而使其它connect趋向于0。