ABXTEST

NAME

abxtest - double-blind ABX comparison testing script

SYNOPSIS

abxtest [-n min] [-m max] [-g goal] A-cmd B-cmd

DESCRIPTION

abxtest is a tool for conducting listening (or other subjective) tests to determine whether a listener can discern a difference between two subjects under test, denoted A and B.

The listener conducts several trials in which an unknown subject, X, is randomly chosen to be either A or B. The probability that X is A or B is the same. The listener must decide whether X sounds the same as A or the same as B. The listener may repeat A, B, and X as many times as necessary before reaching a verdict.

In order to prove the hypothesis that a difference can be discerned between A and B, the listener must disprove the null hypothesis that A sounds the same as B. This is accomplished by correctly identifying X often enough that randomly guessing correctly the same number of times would be statistically improbable.

The A and B subjects for abxtest are actually arbitrary shell commands. The normal mode of use would be to give commands which play two slightly different sounds. The standard output and standard error from each command are suppressed so as not to reveal the identity of X. The listener may interrupt an executing command (usually by pressing ^C) without terminating the test.

OPTIONS

-n min: Set the minimum number of trials. Trials will continue after this number if the statistical probability goal has not yet been met. The default is 10.
-m max: Set the maximum number of trials. Trials will end after this number even if the statistical probability goal has not been met. The default is 20.
-g goal: Set the minimum statistical probability goal. This is the probability that the number of correct identifications is the same as random guesses. In order to disprove the null hypothesis, the result of the test must be a probability less than or equal to this goal. The default is 0.05.

FILES

/dev/random: By default this file is read to generate a random sequence of X.

ENVIRONMENT

RANDOM_FILE: If set, the file named by this variable will be read instead of /dev/random to generate a sequence of X.

NOTES

abxtest uses the binomial distribution to accurately calculate the probability that the result is the same as random guesses.

AUTHOR

Robert Leslie <rob@mars.org>