boost_loss.regression package

class boost_loss.regression.AsymmetricCompositeLoss(loss_pred_less: LossBase, loss_pred_greater: LossBase)[source]

Bases: LossBase

Asymmetric composite loss function. The loss function is loss_pred_less if y_true < y_pred, otherwise loss_pred_greater.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]][source]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray[source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.AsymmetricLoss(loss: LossBase, t: float = 0.5)[source]

Bases: AsymmetricCompositeLoss

Asymmetric loss function. The loss function is loss * (1 - t) if y_true < y_pred, otherwise loss * t. Generalized from quantile loss (pinball loss, check loss, etc.) and expectile loss.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.ExpectileLoss(t: float = 0.5)[source]

Bases: AsymmetricLoss

[Expectile](https://sites.google.com/site/csphilipps/expectiles) loss function. - Expectile is a conditional mean if observations in [μ_τ, ∞) are τ/(1 - τ) times more likely than the original distribution. - Expectiles are not always quantiles of F. - Expectile loss is more smooth than quantile loss.

\[\tau \int_{-\infty}^{\mu_\tau} (y - \mu_\tau) dF(y) = (1 - \tau) \int_{\mu_\tau}^{\infty} (y - \mu_\tau) dF(y)\]

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.FairLoss(c: float = 1.0)[source]

Bases: LossBase

Fair loss = c^2/2 * (abs(y_true - y_pred) - c * log(1 + abs(y_true - y_pred)/c))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.GammaLoss[source]

Bases: LossBase

Gamma loss = y_true / y_pred - log(y_true / y_pred) - 1

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.HuberLoss(delta: float = 1.0)[source]

Bases: LossBase

Huber loss = 0.5 (y_true - y_pred)^2 if |y_true - y_pred| <= delta else delta * (|y_true - y_pred| - 0.5 * delta)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.L1Loss(*, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LNLoss

L1 loss = |y_true - y_pred|.

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

n: float: The exponent of the loss.

class boost_loss.regression.L2Loss(*, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LNLoss

L2 loss = |y_true - y_pred|^2

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

n: float: The exponent of the loss.

class boost_loss.regression.LNLoss(n: float, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LossBase

LNLoss = |y_true - y_pred|^n - x < 1 is not recommended because the loss is not convex. - x >> 2 is not recommended because the gradient is too steep.

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

n: float: The exponent of the loss.

property name: str

Name of loss function.

Returns:: Snake case of class name. e.g. LogCoshLoss -> log_cosh_loss.
Return type:: str

class boost_loss.regression.LogCoshLoss[source]

Bases: LossBase

LogCosh loss = log(cosh(y_true - y_pred))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.LogLoss[source]

Bases: LossBase

Log loss = log(1 + exp(-y_true * y_pred))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.MAPELoss[source]

Bases: LossBase

Mean absolute percentage error loss = abs(y_true - y_pred) / y_true

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.MSLELoss[source]

Bases: LossBase

Mean squared logarithmic error loss = (log(1 + y_true) - log(1 + y_pred))^2

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.PoissonLoss[source]

Bases: LossBase

Poisson loss = y_pred - y_true * log(y_pred)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.QuantileLoss(t: float = 0.5)[source]

Bases: AsymmetricLoss

[Quantile](https://en.wikipedia.org/wiki/Quantile) loss function.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.SMAPELoss[source]

Bases: LossBase

Symmetric mean absolute percentage error loss = abs(y_true - y_pred) / ((abs(y_true) + abs(y_pred))/2)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.TweedieLoss(p: float = 1.5)[source]

Bases: LossBase

Tweedie loss = -y_true * y_pred^(2-p) / (2-p) + y_pred^(1-p) / (1-p)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.VarianceEstimator(estimator: Any, loss: LossBase, *, ts: int | Sequence[float], n_jobs: int | None = 1, verbose: int = 0, random_state: int | None = None, m_type: Literal['mean', 'median'] = 'mean', var_type: Literal['var', 'std', 'range', 'mae', 'mse'] = 'std', apply_objective: bool = True, apply_eval_metric: bool = True, target_transformer: BaseEstimator | Any | None = None, recursive: bool = True, recursive_strict: bool = False)[source]

Bases: BaseEstimator

Estimator that estimates the distribution by simply using multiple estimators with different t. Compared to [NGBoost](https://stanfordmlgroup.github.io/projects/ngboost/) or [CatBoost’s Uncertainty](https://catboost.ai/en/docs/references/uncertainty), this estimator is much slower and does not support “natural gradient”, but does not require any assumption on the distribution.

Note that NGBoost supports [any user-defineddistribution](https://stanfordmlgroup.github.io/ngboost/5-dev.html) # noqa but it has to be defined beforehand.

NGBoost requires mean estimator and log standard deviation estimator to be trained simultaneously, which is very difficult to implement in sklearn / lightgbm / xgboost. (Need to start and stop fitting per iteration) Consider change Base parameter in NGBoost. (See https://github.com/stanfordmlgroup/ngboost/issues/250)

fit(X: Any, y: Any, **fit_params: Any) → Self[source]

Fit each estimator with different t.

Parameters:

X (Any) – The training input samples.
y (Any) – The target values.

Returns:

Fitted estimator.

Return type:

Self

Raises:

RuntimeError – Raises if joblib fails to return the results.

m_type: Literal['mean', 'median']

predict(X: Any, type_: Literal['mean', 'median', 'var', 'std', 'range', 'mae', 'mse'] | None = None, return_std: Literal[False] = False, **predict_params: Any) → ndarray[Any, dtype[Any]][source]

predict(X: Any, type_: tuple[Literal['mean', 'median'], Literal['var', 'std', 'range', 'mae', 'mse']] | None = None, return_std: Literal[True] = False, **predict_params: Any) → tuple[numpy.ndarray[Any, numpy.dtype[Any]], numpy.ndarray[Any, numpy.dtype[Any]]]

Returns predictions of the ensemble.

Parameters:

X (Any) – X
type (Literal['mean', 'median', 'var', 'std', 'range', 'mae', 'mse'], optional) – Type of the prediction, by default None If None, self.m_type is used.
return_std (bool, optional) – Whether to return a tuple of (predictions, standard deviation), by default False
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Predictions of the ensemble with shape (n_samples,)

Return type:

NDArray[Any]

Raises:

ValueError – When type_ is not supported.

predict_raw(X: Any, **predict_params: Any) → ndarray[Any, dtype[Any]][source]

Returns raw predictions of each estimator.

Parameters:

X (Any) – X
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Raw predictions of each estimator with shape (n_estimators, n_samples)

Return type:

NDArray[Any]

predict_var(X: Any, type_: Literal['var', 'std', 'range', 'mae', 'mse'] | None = None, **predict_params: Any) → NDArray[Any][source]

Returns variance of the ensemble.

Parameters:

X (Any) – X
type (Literal['var', 'std', 'range', 'mae', 'mse'], optional) – Type of the variance, by default None If None, self.var_type is used.
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Variance of the ensemble with shape (n_samples,)

Return type:

NDArray[Any]

set_predict_request(*, return_std: bool | None | str = '$UNCHANGED$', type_: bool | None | str = '$UNCHANGED$') → VarianceEstimator

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to predict.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

return_std (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_std parameter in predict.
type (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for type_ parameter in predict.

Returns:

self – The updated object.

Return type:

object

ts_: Sequence[float]

var_type: Literal['var', 'std', 'range', 'mae', 'mse']

Submodules

boost_loss.regression.asymmetric module

class boost_loss.regression.asymmetric.AsymmetricCompositeLoss(loss_pred_less: LossBase, loss_pred_greater: LossBase)[source]

Bases: LossBase

Asymmetric composite loss function. The loss function is loss_pred_less if y_true < y_pred, otherwise loss_pred_greater.

grad_hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → tuple[numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]], numpy.ndarray[typing.Any, numpy.dtype[+ScalarType]]][source]

Gradient and hessian of loss function. Override this method if you want to calculate both gradient and hessian at the same time.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient and hessian of loss function. 1-D array with shape (n_samples,).

Return type:

tuple[NDArray, NDArray]

loss(y_true: NDArray, y_pred: NDArray) → float | NDArray[source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.asymmetric.AsymmetricLoss(loss: LossBase, t: float = 0.5)[source]

Bases: AsymmetricCompositeLoss

Asymmetric loss function. The loss function is loss * (1 - t) if y_true < y_pred, otherwise loss * t. Generalized from quantile loss (pinball loss, check loss, etc.) and expectile loss.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.asymmetric.ExpectileLoss(t: float = 0.5)[source]

Bases: AsymmetricLoss

[Expectile](https://sites.google.com/site/csphilipps/expectiles) loss function. - Expectile is a conditional mean if observations in [μ_τ, ∞) are τ/(1 - τ) times more likely than the original distribution. - Expectiles are not always quantiles of F. - Expectile loss is more smooth than quantile loss.

\[\tau \int_{-\infty}^{\mu_\tau} (y - \mu_\tau) dF(y) = (1 - \tau) \int_{\mu_\tau}^{\infty} (y - \mu_\tau) dF(y)\]

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

class boost_loss.regression.asymmetric.QuantileLoss(t: float = 0.5)[source]

Bases: AsymmetricLoss

[Quantile](https://en.wikipedia.org/wiki/Quantile) loss function.

loss_pred_greater: LossBase: The loss function if y_true >= y_pred.

loss_pred_less: LossBase: The loss function if y_true < y_pred.

boost_loss.regression.regression module

class boost_loss.regression.regression.FairLoss(c: float = 1.0)[source]

Bases: LossBase

Fair loss = c^2/2 * (abs(y_true - y_pred) - c * log(1 + abs(y_true - y_pred)/c))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.GammaLoss[source]

Bases: LossBase

Gamma loss = y_true / y_pred - log(y_true / y_pred) - 1

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.HuberLoss(delta: float = 1.0)[source]

Bases: LossBase

Huber loss = 0.5 (y_true - y_pred)^2 if |y_true - y_pred| <= delta else delta * (|y_true - y_pred| - 0.5 * delta)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.L1Loss(*, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LNLoss

L1 loss = |y_true - y_pred|.

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

n: float: The exponent of the loss.

class boost_loss.regression.regression.L2Loss(*, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LNLoss

L2 loss = |y_true - y_pred|^2

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

n: float: The exponent of the loss.

class boost_loss.regression.regression.LNLoss(n: float, divide_n_loss: bool = False, divide_n_grad: bool = True)[source]

Bases: LossBase

LNLoss = |y_true - y_pred|^n - x < 1 is not recommended because the loss is not convex. - x >> 2 is not recommended because the gradient is too steep.

divide_n_grad: bool: Whether to divide the gradient by n. Generally True is used.

divide_n_loss: bool: Whether to divide the loss by n. Generally False is used.

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

n: float: The exponent of the loss.

property name: str

Name of loss function.

Returns:: Snake case of class name. e.g. LogCoshLoss -> log_cosh_loss.
Return type:: str

class boost_loss.regression.regression.LogCoshLoss[source]

Bases: LossBase

LogCosh loss = log(cosh(y_true - y_pred))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.LogLoss[source]

Bases: LossBase

Log loss = log(1 + exp(-y_true * y_pred))

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.MAPELoss[source]

Bases: LossBase

Mean absolute percentage error loss = abs(y_true - y_pred) / y_true

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.MSLELoss[source]

Bases: LossBase

Mean squared logarithmic error loss = (log(1 + y_true) - log(1 + y_pred))^2

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.PoissonLoss[source]

Bases: LossBase

Poisson loss = y_pred - y_true * log(y_pred)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.SMAPELoss[source]

Bases: LossBase

Symmetric mean absolute percentage error loss = abs(y_true - y_pred) / ((abs(y_true) + abs(y_pred))/2)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

class boost_loss.regression.regression.TweedieLoss(p: float = 1.5)[source]

Bases: LossBase

Tweedie loss = -y_true * y_pred^(2-p) / (2-p) + y_pred^(1-p) / (1-p)

grad(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 1st order derivative (gradient) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The gradient of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

hess(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

The 2nd order derivative (hessian) of loss w.r.t. y_pred.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The hessian of loss function. 1-D array with shape (n_samples,).

Return type:

NDArray

Raises:

NotImplementedError – If not implemented.

loss(y_true: ndarray[Any, dtype[ScalarType]], y_pred: ndarray[Any, dtype[ScalarType]]) → ndarray[Any, dtype[ScalarType]][source]

Loss function. If 1-D array is returned, the mean of array is calculated. Return 1-D array if possible in order to utilize weights in the dataset if available.

Parameters:

y_true (NDArray) – The true target values.
y_pred (NDArray) – The predicted target values.

Returns:

The loss function. 1-D array with shape (n_samples,) or float.

Return type:

NDArray | float

Raises:

NotImplementedError – If not implemented.

boost_loss.regression.sklearn module

class boost_loss.regression.sklearn.VarianceEstimator(estimator: Any, loss: LossBase, *, ts: int | Sequence[float], n_jobs: int | None = 1, verbose: int = 0, random_state: int | None = None, m_type: Literal['mean', 'median'] = 'mean', var_type: Literal['var', 'std', 'range', 'mae', 'mse'] = 'std', apply_objective: bool = True, apply_eval_metric: bool = True, target_transformer: BaseEstimator | Any | None = None, recursive: bool = True, recursive_strict: bool = False)[source]

Bases: BaseEstimator

Estimator that estimates the distribution by simply using multiple estimators with different t. Compared to [NGBoost](https://stanfordmlgroup.github.io/projects/ngboost/) or [CatBoost’s Uncertainty](https://catboost.ai/en/docs/references/uncertainty), this estimator is much slower and does not support “natural gradient”, but does not require any assumption on the distribution.

Note that NGBoost supports [any user-defineddistribution](https://stanfordmlgroup.github.io/ngboost/5-dev.html) # noqa but it has to be defined beforehand.

NGBoost requires mean estimator and log standard deviation estimator to be trained simultaneously, which is very difficult to implement in sklearn / lightgbm / xgboost. (Need to start and stop fitting per iteration) Consider change Base parameter in NGBoost. (See https://github.com/stanfordmlgroup/ngboost/issues/250)

fit(X: Any, y: Any, **fit_params: Any) → Self[source]

Fit each estimator with different t.

Parameters:

X (Any) – The training input samples.
y (Any) – The target values.

Returns:

Fitted estimator.

Return type:

Self

Raises:

RuntimeError – Raises if joblib fails to return the results.

m_type: Literal['mean', 'median']

predict(X: Any, type_: Literal['mean', 'median', 'var', 'std', 'range', 'mae', 'mse'] | None = None, return_std: Literal[False] = False, **predict_params: Any) → ndarray[Any, dtype[Any]][source]

predict(X: Any, type_: tuple[Literal['mean', 'median'], Literal['var', 'std', 'range', 'mae', 'mse']] | None = None, return_std: Literal[True] = False, **predict_params: Any) → tuple[numpy.ndarray[Any, numpy.dtype[Any]], numpy.ndarray[Any, numpy.dtype[Any]]]

Returns predictions of the ensemble.

Parameters:

X (Any) – X
type (Literal['mean', 'median', 'var', 'std', 'range', 'mae', 'mse'], optional) – Type of the prediction, by default None If None, self.m_type is used.
return_std (bool, optional) – Whether to return a tuple of (predictions, standard deviation), by default False
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Predictions of the ensemble with shape (n_samples,)

Return type:

NDArray[Any]

Raises:

ValueError – When type_ is not supported.

predict_raw(X: Any, **predict_params: Any) → ndarray[Any, dtype[Any]][source]

Returns raw predictions of each estimator.

Parameters:

X (Any) – X
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Raw predictions of each estimator with shape (n_estimators, n_samples)

Return type:

NDArray[Any]

predict_var(X: Any, type_: Literal['var', 'std', 'range', 'mae', 'mse'] | None = None, **predict_params: Any) → NDArray[Any][source]

Returns variance of the ensemble.

Parameters:

X (Any) – X
type (Literal['var', 'std', 'range', 'mae', 'mse'], optional) – Type of the variance, by default None If None, self.var_type is used.
**predict_params (Any) – The parameters to be passed to predict method of each estimator.

Returns:

Variance of the ensemble with shape (n_samples,)

Return type:

NDArray[Any]

set_predict_request(*, return_std: bool | None | str = '$UNCHANGED$', type_: bool | None | str = '$UNCHANGED$') → VarianceEstimator

Request metadata passed to the predict method.

Note that this method is only relevant if enable_metadata_routing=True (see sklearn.set_config()). Please see User Guide on how the routing mechanism works.

The options for each parameter are:

True: metadata is requested, and passed to predict if provided. The request is ignored if metadata is not provided.
False: metadata is not requested and the meta-estimator will not pass it to predict.
None: metadata is not requested, and the meta-estimator will raise an error if the user provides it.
str: metadata should be passed to the meta-estimator with this given alias instead of the original name.

The default (sklearn.utils.metadata_routing.UNCHANGED) retains the existing request. This allows you to change the request for some parameters and not others.

New in version 1.3.

Note

This method is only relevant if this estimator is used as a sub-estimator of a meta-estimator, e.g. used inside a Pipeline. Otherwise it has no effect.

Parameters:

return_std (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for return_std parameter in predict.
type (str, True, False, or None, default=sklearn.utils.metadata_routing.UNCHANGED) – Metadata routing for type_ parameter in predict.

Returns:

self – The updated object.

Return type:

object

ts_: Sequence[float]

var_type: Literal['var', 'std', 'range', 'mae', 'mse']