Unequal Variance: SDT Assumptions Revisited

Relaxing the equal variance assumption

Back when we first discussed Measurement & Sensitivity, we mentioned that the noise distribution and the signal-plus-noise distribution are assumed to have equal variance. It turns out that this assumption is often violated in real world scenarios where we would like to apply SDT. For example, studies of recognition memory consistently find evidence of unequal variance (Glanzer et al., 1999; Ratcliff et al., 1992). This is a problem for equal variance SDT because it can’t account for these observed patterns of performance.

Fortunately, SDT can be generalized to account for distributions with unequal variance. First, we will quantify the relative variance for the signal-plus-noise distribution as compared to the noise distribution as σ. When the variance, σ, is equal to one then the distributions have equal variance. When it is smaller than one, the signal-plus-noise distribution has a smaller variance, and when it is larger than one, the signal-plus-noise distribution has a larger variance.

You can explore this here:

You will notice that this example was set up to show zROC space by default. Observe what happens to the iso-sensitivity curve as you adjust the variance… It changes slope! Granted, you can still see that it changes, somehow, in non-transformed ROC space, but zROC space turns that complex shape-shift into a simple change in slope. Indeed, in the literature, unequal variance is usually determined by testing if the slope of an empirically derived zROC curve is not equal to one (Glanzer et al., 1999; Ratcliff et al., 1992).

Calculating unequal variance d′ and c

As suggested by the diagram of the SDT model, we still have a formal mathematical relationship between performance and model parameters, it’s just a bit more complicated now, since it has to account for the inequality of variance.

Here is the equation for sensitivity, taking variance into account:

And here is the equation for bias, taking variance into account:

Iso-contours with unequal variance

You can also get a sense of the effects of unequal variance on the relationship of sensitivity and bias to hit rate and false alarm rate by looking at how the iso-sensitivity and iso-bias contours change as the variance is manipulated:

Calculating hit rate and false alarm rate with unequal variance

To make model predictions with unequal variance SDT, we will need to calculate hit rate, taking variance into account:

And we will need to calculate false alarm rate, taking variance into account:

Spurious correlation when unequal variance data analyzed with equal variance model

As a final note, let’s observe what can happen if our data is produced by a mechanism that follows unequal variance SDT, but we erroneously analyze it using the equal variance model.

Imagine we have a number of conditions and for each condition we calculate sensitivity and bias using equal variance SDT. According to our calculations, d′ and c are both different for each condition, and curiously, as shown in the example below, they are correlated with each other. We might be tempted to consider what additional process may be acting in our participants to cause this linkage of sensitivity and bias. But, in fact, this seeming relationship is spurious. When unequal variance is taken into account, all of the conditions have the same sensitivity and only the bias is varying between conditions.

If nothing else, this last example demonstrates the importance of understanding and being aware of the assumptions of a model, so that you can keep your eye out for characteristic patterns suggestive of failures of those assumptions.