morphoclass.metrics module¶
Implementation of various metric functions.
-
morphoclass.metrics.chance_agreement(y_true: ArrayLike[int]) → float¶ Compute accuracy obtained “by chance” by a model giving random predictions.
- Parameters
y_true – 1d array-like. Ground truth (correct) labels.
- Returns
score – The expected accuracy of a model giving random predictions according to the class frequencies observed in y_true.
- Return type
float
- Raises
ValueError – If y_true has no elements.
References
[1] https://en.wikipedia.org/wiki/Cohen%27s_kappa#Definition [2] https://github.com/BlueBrain/morphoclass/issues/49#issue-1224044038
-
morphoclass.metrics.classification_report(targets, pred_proba, labels=None)¶ Create an advanced classification report.
- targets
The target labels.
- pred_proba
Predicted label probabilities.
- labels
Descriptions for the numerical labels.
-
morphoclass.metrics.discordance(targets, probabilities)¶ Compute the discordance score given binary classification probabilities.
- Parameters
targets – Array with true binary labels.
probabilities – Array with prediction probabilities for binary classification.
- Returns
The discordance score.
- Return type
score
-
morphoclass.metrics.discordance_multiclass(targets, probabilities)¶ Compute the discordance score.
- Parameters
targets – Array with true (sparse) multi-class labels of shape (n_samples,).
probabilities – Array with multi-class classification probabilities of shape (n_samples, n_classes).
- Returns
Array of shape (n_classes) containing discordance scores for each class computed as a one-versus-all binary classification.
- Return type
results
-
morphoclass.metrics.fowlkes_mallows_score(targets, pred)¶ Compute the Fowlkes-Mallows score a.k.a. G-means.
It is similar to the F1 score, but instead of a harmonic mean of the precision and recall, a geometric mean is taken.
- Parameters
targets – True labels.
pred – Predicted labels.
- Returns
The Fowlkes-Mallows score
- Return type
score
-
morphoclass.metrics.inter_rater_score(targets, pred, kind='cohen_kappa')¶ Compute different kinds of inter-rater agreement scores.
Also known as the “chance corrected accuracy”.
The different kinds are either “Cohen Kappa” or “Scott’s Pi”.
The score is equal to zero if the agreement p0 is purely due to the expected chance agreement pe. The difference between the Cohen Kappa score and the Scott’s Pi score is that the former computes the chance agreement as squared geometric means of marginal proportions, and the latter as squared arithmetic means.
Cohen’s kappa is considered more informative since Scott’s Pi since it assumes that the annotators draw their responses from different distributions, while Scott’s Pi doesn’t.
- Parameters
targets – True labels.
pred – Predicted labels.
kind – Kind of score to be computed. Either “cohen_kappa” or “scotts_pi”.
- Returns
The inter-rater score.
- Return type
score