Model Prediction: Predicting Human Performance with Signal Detection Theory
From model parameters to predicted performance
We have now explored SDT and we have seen how human data can be fit with the model. If we have model parameters, either from fitting human data, or derived from theoretical considerations, we may want to simulate task performance in order to generate predictions. In other words, we can have our model perform the task as if it were a participant.
When you run the task on this page, the model will perform the task based on its parameters. You can adjust those parameters at any time in the model diagram, and performance (both previous and subsequent) will be updated accordingly. The table of outcomes will maintain a running count of aggregate performance. ROC space will show an updating view of the relationship between behavior and theory. By observing and manipulating these relationships, you can gain a deeper appreciation for how the model parameters predict performance.
You can select how many the model will perform, the of the stimulus on each trial, and the proportion of dots that exhibit when the signal is present. You can the task, temporarily it, or totally it.
Each trial will begin with a fixation, +, then a stimulus, and finally a question mark, ?. The model will respond based on it’s measurement of evidence, represented by a box moving across the model diagram. The model diagram shows the selected value for the model’s Sensitivity as the distance, d′, between the distributions. And it shows the selected value for the model’s Bias as the location, c, of the threshold. The threshold divides the Signal + Noise Distribution into regions of Hits and Misses and divides the Noise Distribution into regions of Correct Rejections and False Alarms.
The model will decide whether the signal is Present or Absent based on the accumulation of evidence, and respond by clicking to indicate a ‘Present’ response or to indicate an ‘Absent’ response.
Based on the stimulus and the model’s response, you will then see feedback indicating whether this trial resulted in a Hit, Miss, False Alarm, Correct Rejection, or No Response.
The table of outcomes summarizes the model’s Hits, Misses, False Alarms, and Correct Rejections, along with it’s Hit Rate, False Alarm Rate, and overall Accuracy.
In ROC space, the model’s performance is plotted as Hit Rate versus False Alarm Rate. All of the points with the same Sensitivity (d′) are illustrated with an Iso-Sensitivity Curve. All of the points with the same Bias (c) are illustrated with an Iso-Bias Curve.
In the model diagram, you can move the distributions or the threshold at any time to alter d′ and c, and observe the effect this has on predicted performance in the model diagram, table of outcomes, and ROC space.