Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/api/metrics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ We provide easy to use metrics (the same style and args as `sklearn.metrics`) fo
multiclass classification, multilabel classification.
For applicable tasks, we provide the relevant metrics for model calibration, as well as those for prediction set evaluation.
Among these we also provide metrics related to uncertainty quantification, for model calibration, as well as metrics that measure the quality of prediction sets
We also provide other metrics specically for healthcare
We also provide other metrics specifically for healthcare
tasks, such as drug drug interaction (DDI) rate.
For synthetic (generative) EHR data, we provide privacy, utility, and statistical
fidelity metrics.
Expand Down
2 changes: 2 additions & 0 deletions examples/cardiology_detection_isAR_SparcNet.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
"""Cardiology arrhythmia detection using SparcNet."""

from pyhealth.datasets import split_by_visit, get_dataloader
from pyhealth.trainer import Trainer
from pyhealth.datasets import CardiologyDataset
Expand Down
4 changes: 2 additions & 2 deletions pyhealth/metrics/calibration.py
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ def _ECE_classwise(prob:np.ndarray, label_onehot:np.ndarray, bins=20, threshold=
return summs, class_losses

def ece_confidence_multiclass(prob:np.ndarray, label:np.ndarray, bins=20, adaptive=False):
"""Expected Calibration Error (ECE).
r"""Expected Calibration Error (ECE).

We group samples into 'bins' basing on the top-class prediction.
Then, we compute the absolute difference between the average top-class prediction and
Expand Down Expand Up @@ -133,7 +133,7 @@ def ece_confidence_multiclass(prob:np.ndarray, label:np.ndarray, bins=20, adapti
return _ECE_confidence(df, bins, adaptive)[1]

def ece_confidence_binary(prob:np.ndarray, label:np.ndarray, bins=20, adaptive=False):
"""Expected Calibration Error (ECE) for binary classification.
r"""Expected Calibration Error (ECE) for binary classification.

Similar to :func:`ece_confidence_multiclass`, but on class 1 instead of the top-prediction.

Expand Down