Monday, February 09, 2026

Activation function: ReLU (≈2010)

 ReLU (≈2010) Violent cut. Unbounded positive growth, hard elimination of the negative.

→ Enables depth, but creates dead zones and instability. → Lack is expelled, not resolved.




No comments: