The Spectral Neuron

A neuron that outputs an eigenvalue of a learned matrix, staying readable as it grows.

How it works

Transparency or power
Linear models show what each feature does but stay simple. Neural nets scale in power but hide how features act.
Eigenvalue as nonlinearity
Each feature scales a learned symmetric matrix. The sum is one matrix, and its k-th smallest eigenvalue is the prediction.
Scales, stays readable
Bigger matrices fit harder functions. Results land near MLPs on HIGGS and factorization machines on Criteo, with explicit influence bounds.

try it

Drag the matrix dimension up and watch test loss on a wiggly target.

Matrix dimension5 d

Small matrices flatten out early. Larger ones keep improving, so size is the scaling knob.Illustrative model, not the paper's data

6 terms

Spectral neuron Model whose output is the k-th eigenvalue of a feature-weighted matrix sum.

Like a linear model where each feature gets a weight table instead of a number. The answer is one sorted 'resonance' of the combined table.

Where it breaks

Many different tables give the same output, so individual entries carry no meaning.

Eigensolves cost roughly d³ operations, far slower than linear models.
Slightly behind MLPs of depth two or more on HIGGS.
Only one monotone feature tested; parametrizing several PSD matrices is left open.

From the paper