boost_loss package

class boost_loss.DebugLoss(loss_: LossBase)

Bases: LossBase

Calls LOG.debug() every time loss() or grad_hess() is called.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_: LossBase

class boost_loss.LossBase

Bases: object

Base class for loss functions. Inherit this class to implement custom loss function.

See also

Catboost

https

//catboost.ai/en/docs/concepts/python-usages-examples#user-defined-loss-function

LightGBM

https

//lightgbm.readthedocs.io/en/latest/Advanced-Topics.html#custom-objective-function

XGBoost

https

//xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html

Example

>>> from boost_loss.base import LossBase
>>> import numpy as np
>>> from numpy.typing import NDArray
>>>
>>> class L2Loss(LossBase):
>>>     def loss(self, y_true: NDArray, y_pred: NDArray) -> NDArray:
>>>         return (y_true - y_pred) ** 2
>>>     def grad(self, y_true: NDArray, y_pred: NDArray) -> NDArray: # dL/dy_pred
>>>         return -2 * (y_true - y_pred) # or (y_pred - y_true)
>>>     def hess(self, y_true: NDArray, y_pred: NDArray) -> NDArray: # d2L/dy_pred2
>>>         return 2 * np.ones_like(y_true) # or np.ones_like(y_true)
>>>
>>> from boost.sklearn import apply_custom_loss
>>> import lightgbm as lgb
>>> apply_custom_loss(lgb.LGBMRegressor(), L2Loss()).fit(X, y)

final calc_ders_range(preds: Sequence[float], targets: Sequence[float], weights: Sequence[float] | None = None) → list[tuple[float, float]]

Catboost-compatible interface

final eval_metric_lgb(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix, sample_weight: NDArray | lgb.Dataset | xgb.DMatrix | None = None) → tuple[str, float, bool]

LightGBM-compatible interface

final eval_metric_xgb_native(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix) → tuple[str, float]

XGBoost-native-api-compatible interface

final eval_metric_xgb_sklearn(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix, sample_weight: NDArray | lgb.Dataset | xgb.DMatrix | None = None) → float

XGBoost-sklearn-api-compatible interface

final evaluate(approxes: Sequence[float], target: Sequence[float], weight: Sequence[float] | None = None) → tuple[float, float]

Catboost-compatible interface

final classmethod from_callable(loss: Callable[[NDArray, NDArray], NDArray | float], grad: Callable[[NDArray, NDArray], NDArray], hess: Callable[[NDArray, NDArray], NDArray], name: str | None = None, is_higher_better: bool = False) → type[Self]

Create this class from loss, grad, and hess callables.

Parameters:

• loss (Callable[[NDArray, NDArray], NDArray | float]) – The loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available. (y_true, y_pred) -> loss

• grad (Callable[[NDArray, NDArray], NDArray]) – The 1st order derivative (gradient) of loss w.r.t. y_pred. (y_true, y_pred) -> grad

• hess (Callable[[NDArray, NDArray], NDArray]) – The 2nd order derivative (Hessian) of loss w.r.t. y_pred. (y_true, y_pred) -> hess

• name (str | None, optional) – The name of loss function. If None, it tries to infer from loss function, by default None

• is_higher_better (bool, optional) – Whether the result of loss function is better when it is higher, by default False

Returns:

The subclass of this class.

Return type:

type[Self]

Raises:

ValueError – If name is None and it can’t infer from loss function.

final get_final_error(error: float, weight: float | None = None) → float

Catboost-compatible interface

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

is_higher_better: bool = False

Whether the result of loss function is better when it is higher.

final is_max_optimal() → bool

Catboost-compatible interface

loss(y_true: NDArray, y_pred: NDArray) → NDArray | float

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

property name: str

Name of loss function.

Returns:

Snake case of class name. e.g. LogCoshLoss -> log_cosh_loss.

Return type:

str

class boost_loss.PrintLoss(loss_: LossBase)

Bases: LossBase

Prints every time loss() or grad_hess() is called.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_: LossBase

class boost_loss.ResumingLoss(losses: Sequence[LossBase], *, weights: Sequence[float] | None = None, interval: int = 1, random_state: int | None = None)

Bases: LossBase

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → NDArray | float

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

boost_loss.apply_custom_loss(estimator: TEstimator, loss: LossBase, *, copy: bool = True, copy_loss: bool = True, apply_objective: bool = True, apply_eval_metric: bool = True, target_transformer: BaseEstimator | Any | None = StandardScaler(), recursive: bool = True, recursive_strict: bool = False) → TEstimator | TransformedTargetRegressor

Apply custom loss to the estimator.

Parameters:

• estimator (TEstimator) – CatBoost, LGBMModel, or XGBModel

• loss (LossBase) – The custom loss to apply

• copy (bool, optional) – Whether to copy the estimator using sklearn.base.clone, by default True

• copy_loss (bool, optional) – Whether to copy the loss using copy.deepcopy, by default True

• apply_objective (bool, optional) – Whether to apply the custom loss to the estimator’s objective, by default True

• apply_eval_metric (bool, optional) – Whether to apply the custom loss to the estimator’s eval_metric, by default True

• target_transformer (BaseEstimator | Any | None, optional) – The target transformer to use, by default StandardScaler() (This option exists because some loss functions require the target to be normalized (i.e. LogCoshLoss))

• recursive (bool, optional) – Whether to recursively search for estimators inside the estimator and apply the custom loss to all of them, by default True

• recursive_strict (bool, optional) – Whether to recursively search for estimators inside the estimator’s attributes, lists, tuples, sets, and frozensets as well, by default False

Returns:

The estimator with the custom loss applied

Return type:

TEstimator | TransformedTargetRegressor

boost_loss.patch_catboost(estimator: CatBoost) → CatBoost

Patch CatBoost to return only the mean prediction in predict and the variance in predict_var to be consistent with other models. The patch will not apply if the estimator is cloned using sklearn.base.clone() and requires re-patching.

Parameters:

estimator (cb.CatBoost) – The CatBoost estimator to patch.

Returns:

The patched CatBoost estimator.

Return type:

cb.CatBoost

Subpackages

• boost_loss.regression package
  • AsymmetricCompositeLoss
    • AsymmetricCompositeLoss.grad_hess()
    • AsymmetricCompositeLoss.loss()
    • AsymmetricCompositeLoss.loss_pred_greater
    • AsymmetricCompositeLoss.loss_pred_less
  • AsymmetricLoss
    • AsymmetricLoss.loss_pred_greater
    • AsymmetricLoss.loss_pred_less
  • ExpectileLoss
    • ExpectileLoss.loss_pred_greater
    • ExpectileLoss.loss_pred_less
  • FairLoss
    • FairLoss.grad()
    • FairLoss.hess()
    • FairLoss.loss()
  • GammaLoss
    • GammaLoss.grad()
    • GammaLoss.hess()
    • GammaLoss.loss()
  • HuberLoss
    • HuberLoss.grad()
    • HuberLoss.hess()
    • HuberLoss.loss()
  • L1Loss
    • L1Loss.divide_n_grad
    • L1Loss.divide_n_loss
    • L1Loss.hess()
    • L1Loss.n
  • L2Loss
    • L2Loss.divide_n_grad
    • L2Loss.divide_n_loss
    • L2Loss.n
  • LNLoss
    • LNLoss.divide_n_grad
    • LNLoss.divide_n_loss
    • LNLoss.grad()
    • LNLoss.hess()
    • LNLoss.loss()
    • LNLoss.n
    • LNLoss.name
  • LogCoshLoss
    • LogCoshLoss.grad()
    • LogCoshLoss.hess()
    • LogCoshLoss.loss()
  • LogLoss
    • LogLoss.grad()
    • LogLoss.hess()
    • LogLoss.loss()
  • MAPELoss
    • MAPELoss.grad()
    • MAPELoss.hess()
    • MAPELoss.loss()
  • MSLELoss
    • MSLELoss.grad()
    • MSLELoss.hess()
    • MSLELoss.loss()
  • PoissonLoss
    • PoissonLoss.grad()
    • PoissonLoss.hess()
    • PoissonLoss.loss()
  • QuantileLoss
    • QuantileLoss.loss_pred_greater
    • QuantileLoss.loss_pred_less
  • SMAPELoss
    • SMAPELoss.grad()
    • SMAPELoss.hess()
    • SMAPELoss.loss()
  • TweedieLoss
    • TweedieLoss.grad()
    • TweedieLoss.hess()
    • TweedieLoss.loss()
  • VarianceEstimator
    • VarianceEstimator.fit()
    • VarianceEstimator.m_type
    • VarianceEstimator.predict()
    • VarianceEstimator.predict_raw()
    • VarianceEstimator.predict_var()
    • VarianceEstimator.set_predict_request()
    • VarianceEstimator.ts_
    • VarianceEstimator.var_type
  • Submodules
  • boost_loss.regression.asymmetric module
    • AsymmetricCompositeLoss
      • AsymmetricCompositeLoss.grad_hess()
      • AsymmetricCompositeLoss.loss()
      • AsymmetricCompositeLoss.loss_pred_greater
      • AsymmetricCompositeLoss.loss_pred_less
    • AsymmetricLoss
      • AsymmetricLoss.loss_pred_greater
      • AsymmetricLoss.loss_pred_less
    • ExpectileLoss
      • ExpectileLoss.loss_pred_greater
      • ExpectileLoss.loss_pred_less
    • QuantileLoss
      • QuantileLoss.loss_pred_greater
      • QuantileLoss.loss_pred_less
  • boost_loss.regression.regression module
    • FairLoss
      • FairLoss.grad()
      • FairLoss.hess()
      • FairLoss.loss()
    • GammaLoss
      • GammaLoss.grad()
      • GammaLoss.hess()
      • GammaLoss.loss()
    • HuberLoss
      • HuberLoss.grad()
      • HuberLoss.hess()
      • HuberLoss.loss()
    • L1Loss
      • L1Loss.divide_n_grad
      • L1Loss.divide_n_loss
      • L1Loss.hess()
      • L1Loss.n
    • L2Loss
      • L2Loss.divide_n_grad
      • L2Loss.divide_n_loss
      • L2Loss.n
    • LNLoss
      • LNLoss.divide_n_grad
      • LNLoss.divide_n_loss
      • LNLoss.grad()
      • LNLoss.hess()
      • LNLoss.loss()
      • LNLoss.n
      • LNLoss.name
    • LogCoshLoss
      • LogCoshLoss.grad()
      • LogCoshLoss.hess()
      • LogCoshLoss.loss()
    • LogLoss
      • LogLoss.grad()
      • LogLoss.hess()
      • LogLoss.loss()
    • MAPELoss
      • MAPELoss.grad()
      • MAPELoss.hess()
      • MAPELoss.loss()
    • MSLELoss
      • MSLELoss.grad()
      • MSLELoss.hess()
      • MSLELoss.loss()
    • PoissonLoss
      • PoissonLoss.grad()
      • PoissonLoss.hess()
      • PoissonLoss.loss()
    • SMAPELoss
      • SMAPELoss.grad()
      • SMAPELoss.hess()
      • SMAPELoss.loss()
    • TweedieLoss
      • TweedieLoss.grad()
      • TweedieLoss.hess()
      • TweedieLoss.loss()
  • boost_loss.regression.sklearn module
    • VarianceEstimator
      • VarianceEstimator.fit()
      • VarianceEstimator.m_type
      • VarianceEstimator.predict()
      • VarianceEstimator.predict_raw()
      • VarianceEstimator.predict_var()
      • VarianceEstimator.set_predict_request()
      • VarianceEstimator.ts_
      • VarianceEstimator.var_type

Submodules

boost_loss.base module

class boost_loss.base.LossBase

Bases: object

Base class for loss functions. Inherit this class to implement custom loss function.

See also

Catboost

https

//catboost.ai/en/docs/concepts/python-usages-examples#user-defined-loss-function

LightGBM

https

//lightgbm.readthedocs.io/en/latest/Advanced-Topics.html#custom-objective-function

XGBoost

https

//xgboost.readthedocs.io/en/latest/tutorials/custom_metric_obj.html

Example

>>> from boost_loss.base import LossBase
>>> import numpy as np
>>> from numpy.typing import NDArray
>>>
>>> class L2Loss(LossBase):
>>>     def loss(self, y_true: NDArray, y_pred: NDArray) -> NDArray:
>>>         return (y_true - y_pred) ** 2
>>>     def grad(self, y_true: NDArray, y_pred: NDArray) -> NDArray: # dL/dy_pred
>>>         return -2 * (y_true - y_pred) # or (y_pred - y_true)
>>>     def hess(self, y_true: NDArray, y_pred: NDArray) -> NDArray: # d2L/dy_pred2
>>>         return 2 * np.ones_like(y_true) # or np.ones_like(y_true)
>>>
>>> from boost.sklearn import apply_custom_loss
>>> import lightgbm as lgb
>>> apply_custom_loss(lgb.LGBMRegressor(), L2Loss()).fit(X, y)

final calc_ders_range(preds: Sequence[float], targets: Sequence[float], weights: Sequence[float] | None = None) → list[tuple[float, float]]

Catboost-compatible interface

final eval_metric_lgb(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix, sample_weight: NDArray | lgb.Dataset | xgb.DMatrix | None = None) → tuple[str, float, bool]

LightGBM-compatible interface

final eval_metric_xgb_native(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix) → tuple[str, float]

XGBoost-native-api-compatible interface

final eval_metric_xgb_sklearn(y_true: NDArray | lgb.Dataset | xgb.DMatrix, y_pred: NDArray | lgb.Dataset | xgb.DMatrix, sample_weight: NDArray | lgb.Dataset | xgb.DMatrix | None = None) → float

XGBoost-sklearn-api-compatible interface

final evaluate(approxes: Sequence[float], target: Sequence[float], weight: Sequence[float] | None = None) → tuple[float, float]

Catboost-compatible interface

final classmethod from_callable(loss: Callable[[NDArray, NDArray], NDArray | float], grad: Callable[[NDArray, NDArray], NDArray], hess: Callable[[NDArray, NDArray], NDArray], name: str | None = None, is_higher_better: bool = False) → type[Self]

Create this class from loss, grad, and hess callables.

Parameters:

• loss (Callable[[NDArray, NDArray], NDArray | float]) – The loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available. (y_true, y_pred) -> loss

• grad (Callable[[NDArray, NDArray], NDArray]) – The 1st order derivative (gradient) of loss w.r.t. y_pred. (y_true, y_pred) -> grad

• hess (Callable[[NDArray, NDArray], NDArray]) – The 2nd order derivative (Hessian) of loss w.r.t. y_pred. (y_true, y_pred) -> hess

• name (str | None, optional) – The name of loss function. If None, it tries to infer from loss function, by default None

• is_higher_better (bool, optional) – Whether the result of loss function is better when it is higher, by default False

Returns:

The subclass of this class.

Return type:

type[Self]

Raises:

ValueError – If name is None and it can’t infer from loss function.

final get_final_error(error: float, weight: float | None = None) → float

Catboost-compatible interface

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

is_higher_better: bool = False

Whether the result of loss function is better when it is higher.

final is_max_optimal() → bool

Catboost-compatible interface

loss(y_true: NDArray, y_pred: NDArray) → NDArray | float

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

property name: str

Name of loss function.

Returns:

Snake case of class name. e.g. LogCoshLoss -> log_cosh_loss.

Return type:

str

boost_loss.debug module

class boost_loss.debug.DebugLoss(loss_: LossBase)

Bases: LossBase

Calls LOG.debug() every time loss() or grad_hess() is called.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_: LossBase

class boost_loss.debug.PrintLoss(loss_: LossBase)

Bases: LossBase

Prints every time loss() or grad_hess() is called.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_: LossBase

boost_loss.resuming module

class boost_loss.resuming.ResumingLoss(losses: Sequence[LossBase], *, weights: Sequence[float] | None = None, interval: int = 1, random_state: int | None = None)

Bases: LossBase

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → NDArray | float

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

boost_loss.sklearn module

boost_loss.sklearn.apply_custom_loss(estimator: TEstimator, loss: LossBase, *, copy: bool = True, copy_loss: bool = True, apply_objective: bool = True, apply_eval_metric: bool = True, target_transformer: None = StandardScaler(), recursive: bool = True, recursive_strict: bool = False) → TEstimator

boost_loss.sklearn.apply_custom_loss(estimator: TEstimator, loss: LossBase, *, copy: bool = True, copy_loss: bool = True, apply_objective: bool = True, apply_eval_metric: bool = True, target_transformer: BaseEstimator = StandardScaler(), recursive: bool = True, recursive_strict: bool = False) → TransformedTargetRegressor

Apply custom loss to the estimator.

Parameters:

• estimator (TEstimator) – CatBoost, LGBMModel, or XGBModel

• loss (LossBase) – The custom loss to apply

• copy (bool, optional) – Whether to copy the estimator using sklearn.base.clone, by default True

• copy_loss (bool, optional) – Whether to copy the loss using copy.deepcopy, by default True

• apply_objective (bool, optional) – Whether to apply the custom loss to the estimator’s objective, by default True

• apply_eval_metric (bool, optional) – Whether to apply the custom loss to the estimator’s eval_metric, by default True

• target_transformer (BaseEstimator | Any | None, optional) – The target transformer to use, by default StandardScaler() (This option exists because some loss functions require the target to be normalized (i.e. LogCoshLoss))

• recursive (bool, optional) – Whether to recursively search for estimators inside the estimator and apply the custom loss to all of them, by default True

• recursive_strict (bool, optional) – Whether to recursively search for estimators inside the estimator’s attributes, lists, tuples, sets, and frozensets as well, by default False

Returns:

The estimator with the custom loss applied

Return type:

TEstimator | TransformedTargetRegressor

boost_loss.sklearn.patch(estimator: TAny, *, copy: bool = True, recursive: bool = True, recursive_strict: bool = False) → TAny

Patch estimator if it is supported. (patch_ngboost and patch_catboost.) The patch will not apply if the estimator is cloned using sklearn.base.clone() and requires re-patching.

Parameters:

• estimator (TAny) – The estimator to patch.

• copy (bool, optional) – Whether to copy the estimator before patching, by default True

• recursive (bool, optional) – Whether to recursively patch the estimator, by default True

• recursive_strict (bool, optional) – Whether to recursively patch the estimator’s attributes, lists, tuples, sets, and frozensets as well, by default False

Returns:

The patched estimator.

Return type:

TAny

boost_loss.sklearn.patch_catboost(estimator: CatBoost) → CatBoost

Patch CatBoost to return only the mean prediction in predict and the variance in predict_var to be consistent with other models. The patch will not apply if the estimator is cloned using sklearn.base.clone() and requires re-patching.

Parameters:

estimator (cb.CatBoost) – The CatBoost estimator to patch.

Returns:

The patched CatBoost estimator.

Return type:

cb.CatBoost

boost_loss.torch module

class boost_loss.torch.TorchLossBase

Bases: LossBase

Calculate gradient and hessian using torch.autograd.grad. One of loss_torch and grad_torch must be implemented.

Inspired by https://github.com/TomerRonen34/treeboost_autograd/blob/main/treeboost_autograd/pytorch_objective.py # noqa: E501

final classmethod from_callable_torch(loss: Callable[[torch.Tensor, torch.Tensor], torch.Tensor], name: str | None = None, is_higher_better: bool = False) → type[Self]

Create a loss class from a callable.

Parameters:

• loss (Callable[[torch.Tensor, torch.Tensor], torch.Tensor]) – The loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available. (y_true, y_pred) -> loss

• name (str | None, optional) – The name of loss function. If None, it tries to infer from loss function, by default None

• is_higher_better (bool, optional) – Whether the result of loss function is better when it is higher, by default False

Returns:

The subclass of this class.

Return type:

type[Self]

Raises:

ValueError – If name is None and it can’t infer from loss function.

final grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

grad_torch(y_true: Tensor, y_pred: Tensor) → Tensor

The 1st order derivative of loss w.r.t. y_pred.

Parameters:

• y_true (torch.Tensor) – The true target values.

• y_pred (torch.Tensor) – The predicted target values.

Returns:

The gradient of loss w.r.t. y_pred. 1-dim tensor of shape (n_samples,).

Return type:

torch.Tensor

Raises:

NotImplementedError – If not implemented.

final loss(y_true: NDArray, y_pred: NDArray) → NDArray | float

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (NDArray) – The true target values.

• y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_torch(y_true: Tensor, y_pred: Tensor) → Tensor

The loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

• y_true (torch.Tensor) – The true target values.

• y_pred (torch.Tensor) – The predicted target values.

Returns:

0-dim or 1-dim tensor of shape (n_samples,). Return 1-dim tensor if possible to utilize weights in the dataset if available.

Return type:

torch.Tensor

Raises:

NotImplementedError – If not implemented.