How do we define human performance ?

A key factor for evaluation of machine vs humans performance is the standard used to measure the last one. If we are talking for example about a medical classification task, we can use as a human standard the next different measures:

  1. A person (not a doctor)
  2. A general doctor
  3. A specialized doctor
  4. A highly experienced specialized doctor
  5. A team of highly experienced specialized doctors

Which one of all these options defines what human performance is? It is important to define the standard we are comparing against.

In any case, in most of the cases we can consider that the team of highly experienced specialized doctors can be approximated to the Bayes Optimal Error.

Back to blog