bijnum
Class BIJstats

java.lang.Object
  |
  +--bijnum.BIJstats

public class BIJstats

extends java.lang.Object

BIJ statistical methods inherited from MatLab. Syntax is as close as possible to matlab. Copyright (c) 1999-2003, Michael Abramoff. All rights reserved.

Field Summary

static float	CI95 Conversion from standard error to confidence intervals at different significance levels.
static float	CI99
static float	CI99_9

Constructor Summary

BIJstats()

Method Summary

static float	accuracy(float[] exp, float[] truth, int c) Compute accuracy of a test for which the result is in exp and the ground truth in truth, for class c.
static float	accuracyMultipleClasses(float[] exp, float[] truth, int n) Compute the accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.
static float	avg(float[] v) Compute average of all values in vector v.
static float	avg(float[] v, float[] mask)
static float	avgNoExtremes(float[] v, float nrstddev)
static int	binIndex(int[] histogram, float fraction) Compute the lowest bin into which the highest p percent of occurrences falls.
static float	correl(float[] a, float[] b) Compute the correlation of a vector with another vector b.
static float[][]	covariance(float[][] m, boolean doShowProgress) Compute the covariance matrix for a matrix of size NxM.
static double	erf(double x) Compute the erf of x.
static int[]	histogram(float[] v, float min, float d, int n) Compute a histogram with n bins for the vector v, each bin separated by d.
static float	mean(float[][] m)
static float[]	meanColumn(float[][] m)
static float[]	meanColumnT(float[][] m)
static float	mse(float[] a, float[] b) Compute Mean Square Error (or residual) of vectors a and b.
static int	n(float[] v)
static int	n(float[] v, float[] mask)
static float[]	randomFraction(float[] v, double fraction) Randomly sample a fraction of the elements of vector v.
static float	rmse(float[] a, float[] b) Compute Root Mean Square Error of vectors a and b.
static double	sem(double avg, double avg0, double stderr0, double avg1, double stderr1) Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1.
static double	sem(float[] v) sem(1) computes the standard error of the mean of the elements of v.
static double	sem(float[] v, float[] mask) sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0.
static float[]	sensitivities(float[] exp, float[] truth, int n) Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
static float	sensitivity(float[] exp, float[] truth, int c) Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c.
static float[]	specificities(float[] exp, float[] truth, int n) Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.
static float	specificity(float[] exp, float[] truth, int c) Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c.
static float[]	spectrum(float[] v) Compute the spectrum of v.
static float	stdev(float[] v) Compute stdev (SQRT(var)) of all values in vector v.
static float	stdev(float[] v, float[] mask) stdev(2) computes the stddev of the elements of only where the mask value is not 0.
static double	sum(double[] v) Compute average of all values in vector v.
static double	sum(float[] v) sum(1) computes average of all values in vector v.
static double	sum(float[] v, float[] mask) sum(2) computes the sum of the elements of vector v only where the mask value is not 0.
static float	thresholdFraction(float[] v, double fraction)
static float	thresholdFraction(float[] v, float fraction) Find the value in vector over which fraction of all values in vector lie.
static float	ttest(float[] data1, float[] data2) Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob.
static float[]	unitvar(float[] v) Make a new vector of v that has zero mean and unit variance.
static float	var(float[] v) Compute variance of all elements in vector v.
static double[]	zeromean(double[] v) Zero the mean of all elements of v.
static float[]	zeromean(float[] v) Zero the mean of all elements of v.

Methods inherited from class java.lang.Object

, clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CI95

public static float CI95

Conversion from standard error to confidence intervals at different significance levels. To make a confidence interval (99%, 2.576; 95%, 1.96; 99.9 3.39).

CI99

public static float CI99

CI99_9

public static float CI99_9

Constructor Detail

BIJstats

public BIJstats()

Method Detail

covariance

public static float[][] covariance(float[][] m,
                                   boolean doShowProgress)

Compute the covariance matrix for a matrix of size NxM. (= m (TRANSPOSE(m)/length(m)) The returned matrix will be size NxN.

Parameters:

m - a matrix of float[N][M]

Returns:

a matrix of covariances float[N][N]

n

public static int n(float[] v)

n

public static int n(float[] v,
                    float[] mask)

avg

public static float avg(float[] v,
                        float[] mask)

stdev

public static float stdev(float[] v,
                          float[] mask)

stdev(2) computes the stddev of the elements of only where the mask value is not 0. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:

v - a float[] vector.

mask - a float[] with a value of ! 0 for all elements of v that are valid.

Returns:

the stddev of the masked elements in v.

sem

public static double sem(float[] v)

sem(1) computes the standard error of the mean of the elements of v. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:

v - a float[] vector.

Returns:

the standard error of the mean of all elements in v.

sem

public static double sem(float[] v,
                         float[] mask)

sem(2) computes the standard error of the mean of the elements of v only where the mask value is not 0. Standard error is the standard deviation divided by the square root of the number of elements. See Press, Numerical recipes in C, pp 617 for the computation of variance using the two pass algorithm.

Parameters:

v - a float[] vector.

mask - a float[] with a value of ! 0 for all elements of v that are valid.

Returns:

the stderr of the masked elements in v.

sem

public static double sem(double avg,
                         double avg0,
                         double stderr0,
                         double avg1,
                         double stderr1)

Compute the combined standard error of the mean of the two standard errors stderr0 and stderr1 and corresponding averages avg0 and avg1. The method used to calculate the SE of a calculated result involving multiplication and/or division is similar to, but slightly more complicated than that used for addition and subtraction. In this case, the relative standard deviations (or relative standard errors) of the values involved in the calculation must be used rather than the absolute standard errors. The relative SD (or SE) is obtained from the absolute SD (or SE) by dividing the (SE or SD) by the data value itself. Note that these relative quantities are always unitless.

Parameters:

avg - the average of the combination (avg0/avg1)

avg0, - stderr0 average and standard deviation of first component.

avg1, - stderr1 average and standard deviation of second component.

Returns:

standard error of the mean for avg.

avgNoExtremes

public static float avgNoExtremes(float[] v,
                                  float nrstddev)

meanColumnT

public static float[] meanColumnT(float[][] m)

meanColumn

public static float[] meanColumn(float[][] m)

mean

public static float mean(float[][] m)

avg

public static float avg(float[] v)

Compute average of all values in vector v.

Parameters:

v - a vector of float[]

Returns:

the average of all elements in v.

sum

public static double sum(float[] v)

sum(1) computes average of all values in vector v.

Parameters:

v - a vector of float[]

Returns:

the average of all elements in v.

sum

public static double sum(float[] v,
                         float[] mask)

sum(2) computes the sum of the elements of vector v only where the mask value is not 0.

Parameters:

v - a float[] vector.

mask - a float[] with a value of ! 0 for all elements of v that are valid.

Returns:

the avg of the masked elements in v.

sum

public static double sum(double[] v)

Compute average of all values in vector v.

Parameters:

v - a vector of float[]

Returns:

the average of all elements in v.

var

public static float var(float[] v)

Compute variance of all elements in vector v.

Parameters:

v - a vector of float[]

unitvar

public static float[] unitvar(float[] v)

Make a new vector of v that has zero mean and unit variance.

Parameters:

v - a float[] vector

Returns:

a vector with proprtionally the same elements as v but with zero mean and unit variance.

zeromean

public static double[] zeromean(double[] v)

Zero the mean of all elements of v.

Parameters:

v - a vector.

Returns:

a vector with zero mean and same variance as v.

zeromean

public static float[] zeromean(float[] v)

Zero the mean of all elements of v.

Parameters:

v - a vector.

Returns:

a vector with zero mean and same variance as v.

stdev

public static float stdev(float[] v)

Compute stdev (SQRT(var)) of all values in vector v.

Parameters:

v - a vector of float[]

thresholdFraction

public static float thresholdFraction(float[] v,
                                      double fraction)

thresholdFraction

public static float thresholdFraction(float[] v,
                                      float fraction)

Find the value in vector over which fraction of all values in vector lie.

Parameters:

vector - a vector of values

fraction - a float [0-1].

histogram

public static int[] histogram(float[] v,
                              float min,
                              float d,
                              int n)

Compute a histogram with n bins for the vector v, each bin separated by d.

Parameters:

v - a float[] vector

min - the lowest value for the first bin.

d - the difference in value between each bin

n - the number of bins.

Returns:

a float[] with the number of occurences for each bin value.

binIndex

public static int binIndex(int[] histogram,
                           float fraction)

Compute the lowest bin into which the highest p percent of occurrences falls. Used to compute things as "give me the occurence above which 10% of the histogram falls."

Parameters:

histogram - an int[] with the ouccrence counts for each bin

p - the fraction of histogram values desired.

Returns:

an int, the index into the lowest bin that still falls among p.

correl

public static float correl(float[] a,
                           float[] b)

Compute the correlation of a vector with another vector b. The mean of a and b is subtracted before further processing. a and b therefore do not have to be zero-mean.

Parameters:

a - a float[] vector.

b - a float[] vector of same length.

Returns:

r, the correlation coefficient.

spectrum

public static float[] spectrum(float[] v)

Compute the spectrum of v. The spectrum gives the ratio of each element to v to the sum of v. The variance spectrum gives the ratio of each coordinate to the sum of all coordinates.

Parameters:

v - a float[] vector

Returns:

the spectrum of v as a float[].

erf

public static double erf(double x)

Compute the erf of x.

Parameters:

x - the argument

Returns:

the erf of x.

mse

public static float mse(float[] a,
                        float[] b)
                 throws java.lang.IllegalArgumentException

Compute Mean Square Error (or residual) of vectors a and b. a and b are normalized to zero mean and unit variance.

Parameters:

a - a float[] vector

b - a float[] vector.

Returns:

a float that is the Mean Square Error of a and b.

rmse

public static float rmse(float[] a,
                         float[] b)
                  throws java.lang.IllegalArgumentException

Compute Root Mean Square Error of vectors a and b. a and b are normalized to zero mean and unit variance.

Parameters:

a - a float[] vector

b - a float[] vector.

Returns:

a float that is the Root Mean Square Error of a and b.

ttest

public static float ttest(float[] data1,
                          float[] data2)

Perform Student's t-test on two data sets data1 and data2, with the the t values into t and the probabilities into prob. Variances should be the same.

Parameters:

data1 - array of datapoints

data2 - array of datapoints

t -

prob -

randomFraction

public static float[] randomFraction(float[] v,
                                     double fraction)

Randomly sample a fraction of the elements of vector v.

Parameters:

v - a vector

fraction - the fraction of elements from v to be included.

Returns:

a float[] vector with all the sampled elements.

sensitivities

public static float[] sensitivities(float[] exp,
                                    float[] truth,
                                    int n)
                             throws java.lang.IllegalArgumentException

Compute the sensitivities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.

Parameters:

exp - an int[] vector of test results, where 0 <= exp[n] < n.

truth - an int[] vector of ground truth, where 0 <= truth[n] < n.

n - the number of classes to determine the sensitivity for.

Returns:

a float[] vector with the sensitivities for all classes in truth.

specificities

public static float[] specificities(float[] exp,
                                    float[] truth,
                                    int n)
                             throws java.lang.IllegalArgumentException

Compute the specificities of a test of which the result is in exp and the ground truth in truth, for all classes n that occur in truth.

Parameters:

exp - an int[] vector of test results, where 0 <= exp[n] < n.

truth - an int[] vector of ground truth, where 0 <= truth[n] < n.

n - the number of classes to determine the sensitivity for.

Returns:

a float[] vector with the specificities for all classes in truth.

accuracyMultipleClasses

public static float accuracyMultipleClasses(float[] exp,
                                            float[] truth,
                                            int n)
                                     throws java.lang.IllegalArgumentException

Compute the accuracy of a test of which the result is in exp and the ground truth in truth, where test results in multiple classifications.

Parameters:

exp - an int[] vector of test results, where 0 <= exp[n] < n.

truth - an int[] vector of ground truth, where 0 <= truth[n] < n.

n - the number of classes to determine the sensitivity for.

Returns:

the combined accuracy.

sensitivity

public static float sensitivity(float[] exp,
                                float[] truth,
                                int c)
                         throws java.lang.IllegalArgumentException

Compute sensitivity of a test of which the result is in exp and the ground truth in truth, for class c. Only those elements in truth will be considered sensitivity = true pos / (true pos + false neg)

Parameters:

exp - a float[] vector of test results, where 0 <= exp[n] <= c.

truth - a float[] vector of ground truth, where 0 <= truth[n] <= c.

c - the class to determine sensitivity for.

Returns:

the sensitivity.

specificity

public static float specificity(float[] exp,
                                float[] truth,
                                int c)
                         throws java.lang.IllegalArgumentException

Compute specificity of a test of which the result is in exp and the ground truth in truth, for class c. specificity = true neg / (true neg + false pos)

Parameters:

exp - an int[] vector of test results, where 0 <= exp[n] <= c.

truth - an int[] vector of ground truth, where 0 <= truth[n] <= c.

c - the class to determine specificity for.

Returns:

the specificity.

accuracy

public static float accuracy(float[] exp,
                             float[] truth,
                             int c)
                      throws java.lang.IllegalArgumentException

Compute accuracy of a test for which the result is in exp and the ground truth in truth, for class c. accuracy = true pos + true neg / (true neg + false pos + true pos + false neg)

Parameters:

exp - a float[] vector of test results, where 0 <= exp[n] <= c.

truth - a float[] vector of ground truth, where 0 <= truth[n] <= c.

c - the class to determine accuracy for.

Returns:

the accuracy.