! Calculate Kendall tau, Spearman, (KendSpear.lng)
and Pearson correlation coefficients.
The advantage of Kendall and Spearman is that
they are non-parametric. E.g.,
if you compute the Kendall or Spearman correlations for
a set of uniform random variables, and then
transform these uniforms into Normal random
variables using monotonic increasing transformations,
the Kendall and Spearman correlations remain unchanged.
The values are in the range [-1,+1].
Also, if you are contemplating whether a Normal vs
a LogNormal vs a Poisson distribution best fits the
joint demand distribution for a set of 2 or more products,
the same rank order correlation applies for all three
distribution choices.
For the Normal distribution, the relationship
between the expected standard Pearson linear correlation, rho,
and tau is: rho = sin(tau*pi/2). This relationship
holds more generally for any so-called elliptic distribution;
! Keywords: Correlation coefficient, Kendall tau,
Non-parametric statistic, Pearson correlation,
Rank order statistics, Spearman rank;
SETS:
OBS: TVAL, TRANK;
VAR: AVG, STD;
OXV( OBS, VAR): X, XRANK;
VXV( VAR, VAR): TAU, SRHO, PEAR;
ENDSETS
DATA:
!Case0; VAR = 1..3! Is there correlation in demand over 4 weeks among 3 products;
! Soup Nuts Chips
!Case0; X =
27 66 47
44 63 43
33 49 39
35 56 44
;
!Case1 VAR = 1..5;
! Ten observations on five variables;
!Case1 X =
1 8 .43 51 -21
2 2 .09 63 19
3 3 .64 42 29
4 4 6 21 -3
5 9 7 91 3
6 7 8 88 12
7 10 7 42 -9
8 1 2.5 39 -1
9 5 4.6 31 0
10 6 7.5 70 -15;
!Case3 VAR = 1..7;
! All possible orderings of 3 observations.
There are 4 possible correlations for both Spearman & Kendall;
!Case3 X =
1 1 1 2 2 3 3
2 2 3 1 3 1 2
3 3 2 3 1 2 1
;
! The Case 4 data set generates every possible rank
correlation for a set of 4 observations,
7 for Kendall tau,
11 for Spearman, so
the Spearman matrix tends to carry more information;
! For this big matrix you may wish to click on:
Solver | Options | Interface | Precision | 5
Solver | Options | Interface | Page Width | 145
;
!Case4 VAR = 1..25;
! The observations;
!Case4 X =
1 1 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4
2 2 2 3 3 4 4 1 1 3 3 4 4 1 1 2 2 4 4 1 1 2 2 3 3
3 3 4 2 4 2 3 3 4 1 4 1 3 2 4 1 4 1 2 2 3 1 3 1 2
4 4 3 4 2 3 2 4 3 4 1 3 1 4 2 4 1 2 1 3 2 3 1 2 1
;
!Case 5 For 5 observations, there are
11 possible Kendall correlations and
21 possible Spearman correlations;
!Case5 VAR = 1..19;
! The observations;
!Case5 X =
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1
2 2 2 2 2 2 2 3 3 3 3 3 3 4 4 4 4 4 4
3 3 3 4 4 5 5 2 2 4 4 5 5 2 2 3 3 5 5
4 4 5 3 5 3 4 4 5 2 5 2 4 3 5 2 5 2 3
5 5 4 5 3 4 3 5 4 5 2 4 2 5 3 5 2 3 2
;
!Case 6 For 6 observations, there are
15 Kendall correlations possible, and
36 Spearman correlations possible;
!Case5 VAR = 1..55;
! The observations;
!Case5 X =
1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 3 3 3 3 3 3
2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4
3 3 3 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 6 6 6 6 6 6 2 2 2 2 2 2 4 4 4 4 4 4 5 5 5 5 5 5 6 6 6 6 6 6 2 2 2 2 2 2
4 4 4 5 5 6 6 3 3 5 5 6 6 3 3 4 4 6 6 3 3 4 4 5 5 4 4 5 5 6 6 2 2 5 5 6 6 2 2 4 4 6 6 2 2 4 4 5 5 1 1 5 5 6 6
5 5 6 4 6 4 5 5 6 3 6 3 5 4 6 3 6 3 4 4 5 3 5 3 4 5 6 4 6 4 5 5 6 2 6 2 5 4 6 2 6 2 4 4 5 2 5 2 4 5 6 1 6 1 5
6 6 5 6 4 5 4 6 5 6 3 5 3 6 4 6 3 4 3 5 4 5 3 4 3 6 5 6 4 5 4 6 5 6 2 5 2 6 4 6 2 4 2 5 4 5 2 4 2 6 5 6 1 5 1
;
ENDDATA
CALC:
! Get number of rows and cols;
NOBS = @SIZE( OBS);
NVAR = @SIZE( VAR);
! Kendall Tau;
! For all pairs of variables,
look at all pairs of observations and
count the number of times the differences
are in the same direction minus number of
times in the opposite direction and
divide by the number of pairings.
(There are more efficient, but more complex,
ways of computing the Kendall correlation);
@FOR( VAR(j1):
@FOR( VAR(j2)| j2 #GT# j1:
TAU(j1,j2) =
@SUM( OBS(i1)| i1 #LT# NOBS:
@SUM( OBS(i2)| i2 #GT# i1:
@SIGN(X(i2,j1)- X(i1,j1))*@SIGN(X(i2,j2)- X(i1,j2)))
)/(NOBS*(NOBS-1)/2);
TAU(j2,j1) = TAU(j1,j2);
);
TAU(j1,j1) = 1;
);
TAU(NVAR, NVAR) = 1;
! Spearman Rho;
! Prepare the ranks;
@FOR( VAR(j):
@FOR( OBS(i):
tval(i) = x(i,j);
);
trank = @RANK( tval);
@FOR( OBS(i):
xrank(i,j) = trank(i);
);
);
! Compute Spearman rank correlation
(= Pearson correlation on ranks);
! Compute n*xbar^2 and sum x(i)^2;
nxbsq = NOBS*(((NOBS+1)/2)^2);
sumsq = @SUM( OBS(i): i*i);
@FOR( VAR(j1):
@FOR( VAR(j2)| j2 #GT# j1:
SRHO(j1,j2) =
(@SUM( OBS(i): xrank(i,j1)*xrank(i,j2))- nxbsq)/
(sumsq - nxbsq);
SRHO(j2,j1) = SRHO(j1,j2);
);
SRHO(j1,j1) = 1;
);
SRHO(NVAR, NVAR) = 1;
! Compute Pearson correlation;
@FOR( VAR(j1):
! Mean and S.D.;
AVG( j1) = @SUM( OBS(i): X(i,j1))/ NOBS;
STD( j1) = (@SUM( OBS(i): (X(i,j1) - AVG(j1))^2)/NOBS)^0.5
);
@FOR( VAR(j1):
@FOR( VAR(j2)| j2 #GT# j1:
PEAR(j1,j2) =
@SUM( OBS(i): (X(i,j1) - AVG(j2))*(X(i,j2) - AVG(j2)))/(STD(j1)*STD(j2)*NOBS);
PEAR(j2,j1) = PEAR(j1,j2);
);
PEAR(j1,j1) = 1;
);
PEAR(NVAR, NVAR) = 1;
ENDCALC
DATA:
@TEXT() = ' Kendall Tau correlation';
@TEXT() = @TABLE(TAU);
! @TEXT() = " ";
! @TEXT() = " Matrix of ranks";
! @TEXT() = @TABLE(xrank);
@TEXT() = " ";
@TEXT() = " Spearman rank correlation";
@TEXT() = @TABLE(SRHO);
@TEXT() = " ";
@TEXT() = " Pearson correlation";
@TEXT() = @TABLE(PEAR);
ENDDATA
|
