nonconformist.evaluation.ClassIcpCvHelper

class nonconformist.evaluation.ClassIcpCvHelper(icp, calibration_portion=0.25)

Helper class for running the cross_val_score evaluation method on IcpClassifiers.

See also

IcpRegCrossValHelper

Examples

>>> from sklearn.datasets import load_iris
>>> from sklearn.ensemble import RandomForestClassifier
>>> from nonconformist.icp import IcpClassifier
>>> from nonconformist.nc import ClassifierNc, MarginErrFunc
>>> from nonconformist.evaluation import ClassIcpCvHelper
>>> from nonconformist.evaluation import class_mean_errors
>>> from nonconformist.evaluation import cross_val_score
>>> data = load_iris()
>>> nc = ProbEstClassifierNc(RandomForestClassifier(), MarginErrFunc())
>>> icp = IcpClassifier(nc)
>>> icp_cv = ClassIcpCvHelper(icp)
>>> cross_val_score(icp_cv,
...                 data.data,
...                 data.target,
...                 iterations=2,
...                 folds=2,
...                 scoring_funcs=[class_mean_errors],
...                 significance_levels=[0.1])
...     
   class_mean_errors  fold  iter  significance
0           0.013333     0     0           0.1
1           0.080000     1     0           0.1
2           0.053333     0     1           0.1
3           0.080000     1     1           0.1

__init__(icp, calibration_portion=0.25)

get_params(deep=True)

Get parameters for this estimator.

Parameters:

deep : boolean, optional

If True, will return the parameters for this estimator and contained subobjects that are estimators.

Returns:

params : mapping of string to any

Parameter names mapped to their values.

set_params(**params)

Set the parameters of this estimator.

The method works on simple estimators as well as on nested objects (such as pipelines). The latter have parameters of the form <component>__<parameter> so that it’s possible to update each component of a nested object.

Returns:self