Marginals: More Ways to Summarize Performance

Marginals

We’ve discussed how to summarize performance across trials using hit rate, false alarm rate, and accuracy. Each of these is sometimes referred to as marginal measure (or just marginal for short), because they aggregate performance across a row or column of our data table, and are typically presented, quite literally, on the table’s margins.

The hit rate and false alarm rate aggregate across the rows of our table. In other words, they tell us about how well we performed when the signal was actually present or absent. These measures are the ones most relevant to Signal Detection Theory, but we can also aggregate down the columns.

When we calculate the marginals for the columns in our data table, we are summarizing performance when we responded ‘present’ or ‘absent’.

Positive Predictive Value

The marginal for trials were we responded ‘present’ is the positive predictive value (PPV), also called the precision. The positive predictive value tells us how accurate we were on the trials where we responded ‘present’.

False Omission Rate

The marginal for trials were we responded ‘absent’ is the false omission rate (FOR). The false omission rate tells us how accurate we were on the trials where we responded ‘absent’.

A warning

It is important to note that the PPV and the FOR are highly influenced by the base rate (prevalence) of the signal. In other words, the same signal detector behaving in the same way can have a different PPV if the signal occurs on a different proportion of trials.

Depending on the question you are trying to answer, this may be entirely appropriate, or extremely misleading!

One table to rule them all

Let’s add the new marginals to our table of outcomes and see how they all relate. We can emphasize the rows (i.e. group by stimulus), the columns (i.e. group by responses), accuracy (i.e. group by correct versus error), or the full diversity of our measures: