Measurement and Sensitivity: Towards a Theory of Performance

Measuring stimuli

So far, we have focused on observing and summarizing human performance on a signal detection task. We are now going to shift our focus to understanding a theory of how people perform signal detection tasks. As we go, we will develop a model that applies this theory to our particular random dot kinematogram task. (Much more could be said about the distinction, or lack thereof, between models and theories, but we will leave that for another day, or maybe another explorable explanation?).

According to SDT, the first step that must be performed in signal detection is to take a measurement. To continue our earlier examples, this might be a measurement of how much what we hear sounds like our friend’s voice, or how much what we see looks like a distant boat on the sea, or how much what we smell has the aroma of a truffle. In human signal detection, this measurement is being carried out by the sensory and perceptual mechanisms of the participant’s mind and brain. The result of this measurement is the evidence that a signal is present, represented as a single scalar value between negative infinity and infinity, with the neutral point, where the evidence is equally suggestive of signal present and absent, at the origin.

To help us visualize this measurement process, we can run trials of our RDK experiment, and instead of having you, dear reader, perform the measurement, we can construct a model that will simulate the measurement instead. Each trial will be displayed on the graph below, with the strength of evidence plotted in bins along the horizontal x-axis and the count of trials stacked on the vertical y-axis, resulting in a histogram of evidence across trials.

Presence or absence of signal

In the example above, the evidence is visually represented in the same neutral gray on every trial. This emphasizes the fact that our model does not know anything about the trial other than the value it has measured for the evidence!

However, we, as the experimenters, are privy to more information. We know whether each trial was actually a signal or noise trial. So we can color each trial accordingly.

Run the model for 20 or more trials and observe where the measurements are falling on our evidence scale. Unless you are particularly unlucky, you will probably notice a difference between signal and noise trials.

Distributions!

This leads us to a key fact about the measurements our model is making. In SDT, the evidence on each trial is drawn from a probability distribution — either the noise distribution or the signal-plus-noise distribution. On noise trials it is drawn from a noise distribution, and on signal trials it is drawn from a signal-plus-noise distribution. The two distributions are both normal (Gaussian) with equal variance and with equal and opposite means around the neutral point. (We reconsider the equal variance assumption later on the page about Unequal Variance.)

Why is it signal plus noise and not just signal? Because, there is always noise mixed in with the signal, whether explicitly, as in our stimuli when coherence is less than one, or implicitly due to background noise, noisy neurons, etc…

Try running the example below. The evidence measured on each trial will appear on the graph, with the underlying distributions also shown. If we ran the simulation for enough trials, the histogram formed by the trial-by-trial data would match the underlying probability distributions.

Distributions near or far

We can now get a sense of how the proximity of the two distributions determines how similar or different the observations on signal and noise trials are. When the distributions have similar means, we tend to make similar measurements of evidence whether the signal is absent or present:

On the other hand, when the distributions have very different means, we tend to make distinct measurements of evidence on absent and present trials:

Parameterizing distributions with sensitivity, d′

The amount of overlap between the distributions determines our sensitivity. This is formalized as the distance between the noise distribution and the signal-plus-noise distribution, defined as d′. The sensitivity, d′, lies along a continuum, from negative infinity to infinity. Zero indicates identical means and no sensitivity. Positive numbers indicate stronger evidence measurements for signal trials than for noise trials. The larger the positive number, the more sensitive we are to the difference.

On the other hand, negative numbers would indicate stronger evidence for noise trials than signal trials. Since we are talking about evidence in favor of a signal, it would be unusual to have a measurement device that returns more evidence when the signal is absent, but the theory allows for this possibility. Examples of situations where this could occur would be if the participant is confused about the task or if they are intentionally responding incorrectly.

Explore how the sensitivity relates to the relative evidence on signal versus noise trials and the distance between the distributions:

Perhaps you are wondering why sensitivity is represented with the symbol d′. In the early work leading to SDT, for mathematical convenience, a parameter d was used that was equal to the square of the difference between the means of the distributions (Peterson et al., 1954). This parameter was most likely named d because it represented detectability. As SDT was developed, it became clear that the square root of d was a more useful form, since it expressed detectability directly in the units of the probability distributions. Since “square root of d” was a mouthful, it was replaced with the symbol d′, pronounced ‘dee-prime’ (Creelman, 2015; Tanner & Swets, 1954).