Metrics

Compute metrics for assessing the performance of classification/regression models.

Classification

mlpy.error(t, p)

Error for binary and multiclass classification problems.

Parameters :

t : 1d array_like object integer

target values

p : 1d array_like object integer

predicted values

Returns :

error : float, in range [0.0, 1.0]

mlpy.accuracy(t, p)

Accuracy for binary and multiclass classification problems.

Parameters :

t : 1d array_like object integer

target values

p : 1d array_like object integer

predicted values

Returns :

accuracy : float, in range [0.0, 1.0]

Examples:

>>> import mlpy
>>> t = [3,2,3,3,3,1,1,1]
>>> p = [3,2,1,3,3,2,1,1]
>>> mlpy.error(t, p)
0.25
>>> mlpy.accuracy(t, p)
0.75

Binary Classification Only

The Confusion Matrix:

Total Samples (ts)	Actual Positives (ap)	Actual Negatives (an)
Predicted Positives (pp)	True Positives (tp)	False Positives (fp)
Predicted Negatives (pn)	False Negatives (fn)	True Negatives (tn)

mlpy.error_p(t, p)

Compute the positive error as:

error_p = fn / ap

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

errorp : float, in range [0.0, 1.0]

mlpy.error_n(t, p)

Compute the negative error as:

error_n = fp / an

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

errorp : float, in range [0.0, 1.0]

mlpy.sensitivity(t, p)

Sensitivity, computed as:

sensitivity = tp / ap

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

sensitivity : float, in range [0.0, 1.0]

mlpy.specificity(t, p)

Specificity, computed as:

specificity = tn / an

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

sensitivity : float, in range [0.0, 1.0]

mlpy.ppv(t, p)

Positive Predictive Value (PPV) computed as:

ppv = tp / pp

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

PPV : float, in range [0.0, 1.0]

mlpy.npv(t, p)

Negative Predictive Value (NPV), computed as:

npv = tn / pn

Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

NPV : float, in range [0.0, 1.0]

mlpy.mcc(t, p)

Matthews Correlation Coefficient (MCC), computed as:

MCC = ((tp*tn)-(fp*fn)) / sqrt((tp+fn)*(tp+fp)*(tn+fn)*(tn+fp))

Only binary classification problems with t[i] = -1/+1 are allowed.

Returns a value between -1 and +1. A MCC of +1 represents a perfect prediction, 0 an average random prediction and -1 an inverse prediction. If any of the four sums in the denominator is zero, the denominator is set to one; this results in a Matthews Correlation Coefficient of zero, which can be shown to be the correct limiting value.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object integer (-1/+1)

predicted values

Returns :

MCC : float, in range [-1.0, 1.0]

mlpy.auc_wmw(t, p)

Compute the AUC by using the Wilcoxon-Mann-Whitney statistic. Only binary classification problems with t[i] = -1/+1 are allowed.

Parameters :

t : 1d array_like object integer (-1/+1)

target values

p : 1d array_like object (negative/positive values)

predicted values

Returns :

AUC : float, in range [0.0, 1.0]

Examples:

>>> import mlpy
>>> t = [1, 1, 1,-1, 1,-1,-1,-1]
>>> p = [1,-1, 1, 1, 1,-1, 1,-1]
>>> mlpy.error_p(t, p)
0.25
>>> mlpy.error_n(t, p)
0.5
>>> mlpy.sensitivity(t, p)
0.75
>>> mlpy.specificity(t, p)
0.5
>>> mlpy.ppv(t, p)
0.59999999999999998
>>> mlpy.npv(t, p)
0.66666666666666663
>>> mlpy.mcc(t, p)
0.2581988897471611
>>> p = [2.3,-0.4, 1.6, 0.6, 3.2,-4.9, 1.3,-0.3]
>>> mlpy.auc_wmw(t, p)
0.8125
>>> p = [2.3,0.4, 1.6, -0.6, 3.2,-4.9, -1.3,-0.3]
>>> mlpy.auc_wmw(t, p)
1.0

Regression

mlpy.mse(t, p)

Mean Squared Error (MSE).

Parameters :

t : 1d array_like object

target values

p : 1d array_like object

predicted values

Returns :

MSE : float

mlpy.r2(t, p)

Coefficient of determination (R^2) computed as 1 - (sserr/sstot), where sserr is the sum of squares of residuals and sstot is the total sum of squares.

Parameters :

t : 1d array_like object

target values

p : 1d array_like object

predicted values

Returns :

R^2 : float

mlpy.r2_corr(t, p)

Coefficient of determination (R^2) computed as square of the correlation coefficient.

Parameters :

t : 1d array_like object

target values

p : 1d array_like object

predicted values

Returns :

R^2 : float

Example:

>>> import mlpy
>>> t = [2.4,0.4,1.2,-0.2,3.3,-4.9,-1.1,-0.1]
>>> p = [2.3,0.4,1.6,-0.6,3.2,-4.9,-1.3,-0.3]
>>> mlpy.mse(t, p)
0.052499999999999998