sorbetto.parameterization package

class sorbetto.parameterization.AbstractParameterization

Bases: ABC

This is the base class for all possible ways of mapping ranking scores (or, equivalently, importance values, tha is some application-related preferences) onto Tiles. All ranking scores inducing the same performance ordering should be mapped to the same point. It is recommended that the subclasses implement continuous mappings between the four importance values and the two parameters. Also, it is recommended that (1) the ranking scores giving no importance at all to the true positives are mapped to points on the left border (minimal value for the first parameter), (2) the ranking scores giving no importance at all to the true negatives are mapped to points on the right border (maximal value for the first parameter), (3) the ranking scores giving no importance at all to the false positives are mapped to points on the lower border (minimal value for the second parameter), and (4) the ranking scores giving no importance at all to the false negatives are mapped to points on the upper border (minimal value for the second parameter).

abstract getBoundsParameter1() → tuple[float, float]

abstract getBoundsParameter2() → tuple[float, float]

getCanonicalImportance(param1: float, param2: float) → Importance

Returns the canonical importance corresponding to the given parameters.

Default implementation calls the getCanonicalImportanceVectorized (abstract) method.

Parameters:

• param1 (float) – The first parameter.

• param2 (float) – The second parameter.

Returns:

The canonical importance.

abstract getCanonicalImportanceVectorized(param1: ndarray, param2: ndarray) → ndarray

Computes a array of canonical importances values corresponding to the given parameters.

This needs to be implemented by subclasses.

Parameters:

• param1 (np.ndarray) – The first parameter array.

• param2 (np.ndarray) – The second parameter array.

Returns:

an array of shape (N, 4) where N is the number of elements in param1 and param2.

getCanonicalRankingScore(param1: float, param2: float) → RankingScore

getExtent() → tuple[float, float, float, float]

abstract getName()

abstract getNameParameter1() → str

abstract getNameParameter2() → str

abstract getValueParameter1(rankingScore) → float

abstract getValueParameter2(rankingScore) → float

locateAccuracy() → Point

locateBalancedAccuracy(priorPos) → Point

locateBennettS() → Point

Bennett’s \(S\). This score is related to the accuracy \(A\) by \(S=2A-1\). Reference: Warrens [20].

locateClaytonSkillScore(ratePos) → Point

locateCohenCorrected(score: RankingScore) → Point

See Piérard et al. [16], Section 4.4.

locateCohenKappa(priorPos) → Point

Cohen’s \(\scoreCohenKappa\) statistic. Defined in Cohen [5] References: Kaymak et al. [14] Synonyms: Heidke Skill Score Canbek et al. [4], Wilks [21]. See Piérard et al. [17], Section A.7.4, and Piérard et al. [16], Section A.3.3.

locateCriticalSuccessIndex() → Point

locateCzekanowskiBinaryIndex() → Point

locateDetectionRate(priorPos) → Point

locateDiceSorensenCoefficient() → Point

Dice-Sørensen coefficient. Synonym: F-one \(\scoreFOne\). \(\scoreFOne=\nicefrac{2\scoreJaccardPos}{\scoreJaccardPos+1}\)

locateF(beta=1.0) → Point

See Piérard et al. [16], Section A.3.5.

locateHeidkeSkillScore(priorPos) → Point

locateInformedness(priorPos) → Point

See Piérard et al. [17], Section A.7.4

locateIntersectionOverUnion() → Point

locateInverseF(beta=1.0) → Point

locateInverseIntersectionOverUnion() → Point

locateInverseJaccard() → Point

locateInversePrecision() → Point

locateInverseRecall() → Point

locateJaccard() → Point

See Piérard et al. [16], Section A.3.5.

locateMarkedness(ratePos) → Point

Markedness. Defined in Powers [19] as \(\scoreNPV+\scorePPV-1\). Synonyms: Clayton Skill Score Canbek et al. [4], Wilks [21].

locateMatchingCoefficient() → Point

locateNegativeLikelihoodRatioComplement(priorPos) → Point

Negative Likelihood Ratio. References: Brown and Davis [3], Gardner and Greiner [9], Glas et al. [10], Powers [19] See Piérard et al. [17], Section A.7.4, and Piérard et al. [16], Section A.3.5.

locateNegativePredictiveValue() → Point

locateNormalizedConfusionMatrixDeterminent(priorPos) → Point

The determinant of the normalized confusion matrix is \(\scoreConfusionMatrixDeterminant=\priorneg\priorpos\scoreYoudenJ\). Some works using this score: Wimmer et al. [22].

locateOrderingsInvertedWithOpChangeGroundtruthClass() → Conic

\[\left\{ R_I : I(tp) I(fn) = I(tn) I(fp) \right\} = \left\{ R_I : a(I) + b(I) = 1 \right\}\]

locateOrderingsInvertedWithOpChangePredictedClass() → Conic

\[\left\{ R_I : I(tp) I(fp) = I(tn) I(fn) \right\} = \left\{ R_I : a(I) = b(I) \right\}\]

abstract locateOrderingsPuttingNoSkillPerformancesOnAnEqualFootingForFixedClassPriors(priorPos: float) → AbstractGeometricObject2D

The set of performance orderings induced by ranking scores that put all no-skill performances, for given class priors \((\pi_-, \pi_+)\), on an equal footing is given by

\[\left\{ \pi_+^2 I(tp) I(fn) = \pi_-^2 I(tn) I(fp) \right\}\]

See Piérard et al. [16], Figure 6, left. # See Theorem 3 of future “paper 6”. # See Piérard et al. [16], Figure 8.

Parameters:

priorPos (float) – the prior of the positive class, \(\pi_+\)

Returns:

The locus (a curve).

Return type:

AbstractGeometricObject2D

abstract locateOrderingsPuttingNoSkillPerformancesOnAnEqualFootingForFixedPredictionRates(ratePos: float) → AbstractGeometricObject2D

The set of performance orderings induced by ranking scores that put all no-skill performances, for given prediction rates \((\tau_-, \tau_+)\), on an equal footing is given by

\[\left\{ \tau_+^2 I(tp) I(fp) = \tau_-^2 I(tn) I(fn) \right\}\]

See Piérard et al. [16], Figure 6, right. # See Theorem 4 of future “paper 6”.

Parameters:

ratePos (float) – the prediction rate for the positive class, \(\tau_+\)

Returns:

The locus (a curve).

Return type:

AbstractGeometricObject2D

locatePeirceSkillScore(priorPos) → Point

locatePositiveLikelihoodRatio(priorPos) → Point

Positive Likelihood Ratio. References: Altman and Bland [1], Brown and Davis [3], Glas et al. [10], Powers [19] See Piérard et al. [17], Section A.7.4, and Piérard et al. [16], Section A.3.5.

locatePositivePredictiveValue() → Point

locatePrecision() → Point

locateProbabilityFalseNegativeComplenent(priorPos) → Point

locateProbabilityFalsePositiveComplenent(priorPos) → Point

locateProbabilityTrueNegative(priorPos) → Point

locateProbabilityTruePositive(priorPos) → Point

locateRankingScore(rankingScore) → Point

locateRecall() → Point

locateRejectionRate(priorPos) → Point

locateSelectivity() → Point

locateSensitivity() → Point

locateSimilarity() → Point

locateSimilarityCoefficients() → Conic

Similarity coefficients, as defined in Batyrshin et al. [2]. See Batyrshin et al. [2] and Piérard et al. [16], Section 4.2.

locateSimilarityCoefficientsS() → Point

Similarity coefficients of the family \(S_\theta\), as defined in Gower and Legendre [11]. See Gower and Legendre [11] and Piérard et al. [16], Section 4.2.

locateSimilarityCoefficientsT() → Point

Similarity coefficients of the family \(T_\theta\), as defined in Gower and Legendre [11]. See Gower and Legendre [11] and Piérard et al. [16], Section 4.2.

locateSkewInsensitiveVersionOfF(priorPos) → Point

The skew-insensitive version of \(\scoreFOne\). Defined in cite:t:Flach2003TheGeometry.

locateSpecificity() → Point

locateStandardizedNegativePredictiveValue(priorPos) → Point

Standardized Negative Predictive Value (SNPV). Defined in Heston [13]. \(\scoreSNPV=\frac{\scoreTNR}{\scoreTNR+\scoreFNR}=\frac{\scoreNPV\priorpos}{\scoreNPV(\priorpos-\priorneg)+\priorneg}\) See Piérard et al. [16], Section A.3.5.

locateStandardizedPositivePredictiveValue(priorPos) → Point

Standardized Positive Predictive Value (SPPV). Defined in Heston [13]. \(\scoreSPPV=\frac{\scoreTPR}{\scoreFPR+\scoreTPR}=\frac{\scorePPV\priorneg}{\scorePPV(\priorneg-\priorpos)+\priorpos}\) See Piérard et al. [16], Section A.3.5.

locateTanimotoCoefficient() → Point

locateTrueNegativeRate() → Point

locateTruePositiveRate() → Point

locateWeightedAccuracy(priorPos, weightPos) → Point

locateYoudenJ(priorPos) → Point

Youden’s index or Youden’s \(\scoreYoudenJ\) statistic. Defined in Youden [23] References: Fluss et al. [8]. Related to the balanced accuracy by \(\scoreYoudenJ=\scoreTNR+\scoreTPR-1=2\scoreBalancedAccuracy-1\). Synonyms: informedness and Peirce Skill Score Canbek et al. [4], Wilks [21]. See Piérard et al. [16], Section A.3.5.

locateZijdenbosSimilarityIndex() → Point

unitTest()

class sorbetto.parameterization.ParameterizationDefault

Bases: AbstractParameterization

This is the parameterization described in Piérard et al. [16].

getBoundsParameter1() → tuple[float, float]

getBoundsParameter2() → tuple[float, float]

getCanonicalImportance(param1, param2) → Importance

Returns the canonical importance corresponding to the given parameters.

Default implementation calls the getCanonicalImportanceVectorized (abstract) method.

Parameters:

• param1 (float) – The first parameter.

• param2 (float) – The second parameter.

Returns:

The canonical importance.

getCanonicalImportanceVectorized(param1: ndarray, param2: ndarray) → ndarray

Computes a array of canonical importances values corresponding to the given parameters.

This needs to be implemented by subclasses.

Parameters:

• param1 (np.ndarray) – The first parameter array.

• param2 (np.ndarray) – The second parameter array.

Returns:

an array of shape (N, 4) where N is the number of elements in param1 and param2.

getName()

getNameParameter1()

getNameParameter2()

static getPriorNegForIsoValuedNoSkillPerformances(param1: float, param2: float) → float

Returns the prior of the negative class \(\pi_-\) such that the performance ordering located at (param1, param2) puts all the performances corresponding to the priors \((P(Y=c_-), P(Y=c_+))=(\pi_-, 1-\pi_-)\) on an equal footing.

See Piérard et al. [16], Fig. 6, left.

Parameters:

• param1 (float) – the value of the first parameter, \(a\)

• param2 (float) – the value of the second parameter, \(b\)

Returns:

\(\pi_-\)

Return type:

float

static getPriorPosForIsoValuedNoSkillPerformances(param1: float, param2: float) → float

Returns the prior of the positive class \(\pi_+\) such that the performance ordering located at (param1, param2) puts all the performances corresponding to the priors \((P(Y=c_-), P(Y=c_+))=(1-\pi_+, \pi_+)\) on an equal footing.

See Piérard et al. [16], Fig. 6, left.

Parameters:

• param1 (float) – the value of the first parameter, \(a\)

• param2 (float) – the value of the second parameter, \(b\)

Returns:

\(\pi_+\)

Return type:

float

static getRateNegPredictionsForIsoValuedNoSkillPerformances(param1: float, param2: float) → float

Returns the rate of predictions for the negative class \(\tau_-\) such that the performance ordering located at (param1, param2) puts all the performances corresponding to the prediction rates \((P(\hat{Y}=c_-), P(\hat{Y}=c_+))=(\tau_-, 1-\tau_-)\) on an equal footing.

See Piérard et al. [16], Fig. 6, right.

Parameters:

• param1 (float) – the value of the first parameter, \(a\)

• param2 (float) – the value of the second parameter, \(b\)

Returns:

\(\tau_-\)

Return type:

float

static getRatePosPredictionsForIsoValuedNoSkillPerformances(param1: float, param2: float) → float

Returns the rate of predictions for the positive class \(\tau_+\) such that the performance ordering located at (param1, param2) puts all the performances corresponding to the prediction rates \((P(\hat{Y}=c_-), P(\hat{Y}=c_+))=(1-\tau_+, \tau_+)\) on an equal footing.

See Piérard et al. [16], Fig. 6, right.

Parameters:

• param1 (float) – the value of the first parameter, \(a\)

• param2 (float) – the value of the second parameter, \(b\)

Returns:

\(\tau_+\)

Return type:

float

getValueParameter1(rankingScore) → float

getValueParameter2(rankingScore) → float

locateOrderingsPuttingNoSkillPerformancesOnAnEqualFootingForFixedClassPriors(priorPos: float) → BilinearCurve

The set of performance orderings induced by ranking scores that put all no-skill performances, for given class priors \((\pi_-, \pi_+)\), on an equal footing is given by

See Piérard et al. [16], Figure 6, left. # See Theorem 3 of future “paper 6”. # See Piérard et al. [16], Figure 8.

Parameters:

priorPos (float) – the prior of the positive class, \(\pi_+\)

Returns:

The locus (a curve).

Return type:

BilinearCurve

locateOrderingsPuttingNoSkillPerformancesOnAnEqualFootingForFixedPredictionRates(ratePos: float) → BilinearCurve

The set of performance orderings induced by ranking scores that put all no-skill performances, for given prediction rates \((\tau_-, \tau_+)\), on an equal footing is given by

See Piérard et al. [16], Figure 6, right. # See Theorem 4 of future “paper 6”.

Parameters:

ratePos (float) – the prediction rate for the positive class, \(\tau_+\)

Returns:

The locus (a curve).

Return type:

AbstractGeometricObject2D

Submodules

• sorbetto.parameterization.abstract_parameterization module
• sorbetto.parameterization.parameterization_default module