Fisher Discriminant Analysis

θ

The 30 round points are data. The 15 red points were generated from a normal distribution with mean

{1.2,0}

, the 15 blue ones with mean

{0,1.2}

, and in both cases the covariance matrix was the identity matrix. The problem is to classify or predict the color using the inputs

x

1

and

x

2

.

Fisher linear discriminant analysis determines a canonical direction for which the data is most separated when projected on a line in this direction. The solid gray line shows the canonical direction.

The squares are projected points on a line inclined at the angle

θ

with respect to the origin. When

θ

is adjusted so the projected points are aligned with the gray line, the points are maximally separated in the sense that the ratio of between-classes variances to within-classes variance is maximized.

A point is predicted as red or blue according to whether its projection on the canonical direction lies closest to the projected mean of the red or blue data points.