Open In App

Problem solving on scatter matrix

Improve
Improve
Like Article
Like
Save
Share
Report

Prerequisite : Scatter Plot Matrix

We calculate Sw  ( within class scatter matrix ) and SB ( between class scatter matrix )  for the available data points.

SW : To minimize variability within a class, inner class scatter.
SB :  To increase between class variability, between class scatter.

scatter plot of the points

X1 = (y1, y2) ={ (2,2), (1,2), (1,2), (1,2), (2,2) }
X2 = (y1,y2) ={ (9, 10), (6,8), (9,5), (8,7), (10,8) }

Within class scatter matrix :

S_W = \sum_{i=1}^{c}S_i \\ S_i = \sum_{x\in D_i}^{c} (x-m_i)(x-m_i)^{T}

Si is the class specific covariance matrix.
mi is the mean of individual class

Mean computation :

We calculate the mean for each of the points present in the class. Here mean is the total sum of observations divided by the number of observations, we require this mean for calculating the covariance of the matrix.

m_1 = [\frac{2+1+1+1+2}{5} , \frac{2+2+2+2+2}{5} ] \\  =  [1.4,2]

m_2 = [\frac{9+6+9+8+10}{5} , \frac{10+8+5+7+8}{5} ] \\  =  [8.4,7.6]

Covariance matrix computation :
We subtract the mean value from each of the observation and then we calculate the average after performing the matrix multiplication with the transpose of the matrix.

Class specific covariance for the first class :

(X1-m_1) = \begin{bmatrix} 0.6 & -0.4 & -0.4 & -0.4 & 0.6\\ 0 & 0 & 0& 0& 0 \end{bmatrix}

1) \begin{bmatrix} 0.6 \\ 0 \end{bmatrix} * \begin{bmatrix} 0.6 &0 \end{bmatrix} = \begin{bmatrix} 0.36 &0 \\ 0 &0 \end{bmatrix} \\\\ 2) \begin{bmatrix} -0.4 \\ 0 \end{bmatrix} * \begin{bmatrix} -0.4 &0 \end{bmatrix} = \begin{bmatrix} 0.16 &0 \\ 0 &0 \end{bmatrix} \\\\ 3) \begin{bmatrix} -0.4 \\ 0 \end{bmatrix} * \begin{bmatrix} -0.4 &0 \end{bmatrix} = \begin{bmatrix} 0.16 &0 \\ 0 &0 \end{bmatrix} \\\\ 4) \begin{bmatrix} -0.4 \\ 0 \end{bmatrix} * \begin{bmatrix} -0.4 &0 \end{bmatrix} = \begin{bmatrix} 0.16 &0 \\ 0 &0 \end{bmatrix} \\\\ 5) \begin{bmatrix} 0.6 \\ 0 \end{bmatrix} * \begin{bmatrix} 0.6 &0 \end{bmatrix} = \begin{bmatrix} 0.36 &0 \\ 0 &0 \end{bmatrix} \\\\

Averaging values from 1,2,3,4 and 5.
We calculate the sum of all the values for each element in the matrix for S1 and divide by the number of the observation which in the present computation is 5.

matrix_{00} = \frac{ 0.36+0.16+0.16+0.16+0.36}{5}\\ =\frac{1.2}{5}\\ = 0.24\\\\ matrix_{01} = \frac{ 0+0+0+0+0}{5}\\ = 0\\\\ matrix_{10} = \frac{ 0+0+0+0+0}{5}\\ = 0\\\\ matrix_{11} = \frac{ 0+0+0+0+0}{5}\\ = 0\\\\

Therefore S1 is :

S_1 = \begin{bmatrix} 0.24 &0 \\ 0 &0 \end{bmatrix}

Class specific covariance for the second class :

(X2-m_2) = \begin{bmatrix} 0.6 & -2.4 & 0.6 & -0.4 & 1.6\\ 2.4 & 0.4 & -2.6& -0.6& 0.4 \end{bmatrix}

1) \begin{bmatrix} 0.6 \\ 2.4 \end{bmatrix} * \begin{bmatrix} 0.6 &2.4 \end{bmatrix} = \begin{bmatrix} 0.36 &1.44 \\ 1.44 &05.76 \end{bmatrix} \\\\ 2) \begin{bmatrix} -2.4 \\ 0.4 \end{bmatrix} * \begin{bmatrix} -2.4 &0.4 \end{bmatrix} = \begin{bmatrix} 5.76 &-0.96 \\ -0.96 &0.16 \end{bmatrix} \\\\ 3) \begin{bmatrix} 0.6 \\ -2.6 \end{bmatrix} * \begin{bmatrix} 0.6 &-2.6 \end{bmatrix} = \begin{bmatrix} 0.36 &-1.56 \\ 1.56 &6.76 \end{bmatrix} \\\\ 4) \begin{bmatrix} -0.4 \\ -0.6 \end{bmatrix} * \begin{bmatrix} -0.4 &-0.6 \end{bmatrix} = \begin{bmatrix} 0.16 &0.24 \\ 0.24 &0.36 \end{bmatrix} \\\\ 5) \begin{bmatrix} 1.6 \\ 0.4 \end{bmatrix} * \begin{bmatrix} 1.6 &0.4 \end{bmatrix} = \begin{bmatrix} 2.56 &0.64 \\ 0.64 &0.16 \end{bmatrix} \\\\

Averaging values from 1,2,3,4 and 5
we calculate the sum of all the values for each element in the matrix for S2 and divide by the number of the observation which in the present computation is 5.

matrix_{00} = \frac{ 0.36+5.76+0.36+0.16+2.56}{5}\\ =\frac{9.2}{5}\\ = 1.84\\\\ matrix_{01} = \frac{ 1.44-0.96-1.56+0.24+0.64}{5}\\ =\frac{-0.2}{5}\\ = -0.04\\\\ matrix_{10} = \frac{ 1.44-0.96-1.56+0.24+0.64}{5}\\ =\frac{-0.2}{5}\\ = -0.04\\\\ matrix_{11} = \frac{ 5.76+0.16+6.76+0.36+0.16}{5}\\ =\frac{13.2}{5}\\ = 2.64\\\\

Therefore S2 is :

S_2 = \begin{bmatrix} 1.84 &-0.04 \\ -0.04 &2.64 \end{bmatrix}

Within class scatter matrix Sw :

SW = S1 + S2

S_W=  \begin{bmatrix} 0.24 &0 \\ 0 &0 \end{bmatrix} + \begin{bmatrix} 1.84 &-0.04 \\ -0.04 &2.64 \end{bmatrix} \\\\ =\begin{bmatrix} 2.08 &-0.04 \\ -0.04 &2.64 \end{bmatrix}

Between class scatter matrix SB  :

S_B = (m_1 - m_2) * (m_1 - m_2)^{T}\\\\ S_B = \begin{bmatrix} 7 & 5.6 \end{bmatrix} * \begin{bmatrix} 7 \\5.6 \end{bmatrix} \\\\ =\begin{bmatrix} 49 &37.2 \\ 37.2 &31.36 \end{bmatrix}

Total scatter matrix :

ST = SB + SW

S_w  =\begin{bmatrix} 49 &37.2 \\ 37.2 &31.36 \end{bmatrix} + \begin{bmatrix} 2.08 &-0.04 \\ -0.04 &2.64 \end{bmatrix} \\ =\begin{bmatrix} 51.08 & 37.16 \\ 37.16 &34 \end{bmatrix}

Therefore we have calculated between class scatter matrix and within class scatter matrix for the available data points.

We make use of these computations in feature extraction , where the main goal is to increase the distance between the class in the projection of points and decrease the distance between  the points within the class in the projection. Here we aim at generating data projection at the required dimension. 



Last Updated : 03 Feb, 2022
Like Article
Save Article
Previous
Next
Share your thoughts in the comments
Similar Reads