skorch
is designed to maximize interoperability between sklearn
and pytorch
. The aim is to keep 99% of the flexibility of pytorch
while being able to leverage most features of sklearn
. Below, we show the basic usage of skorch
and how it can be combined with sklearn
.
This notebook shows you how to use the basic functionality of skorch
.
import torch
from torch import nn
import torch.nn.functional as F
torch.manual_seed(0);
We load a toy classification task from sklearn
.
import numpy as np
from sklearn.datasets import make_classification
X, y = make_classification(1000, 20, n_informative=10, random_state=0)
X = X.astype(np.float32)
X.shape, y.shape, y.mean()
((1000, 20), (1000,), 0.5)
pytorch
classification module
¶We define a vanilla neural network with two hidden layers. The output layer should have 2 output units since there are two classes. In addition, it should have a softmax nonlinearity, because later, when calling predict_proba
, the output from the forward
call will be used.
class ClassifierModule(nn.Module):
def __init__(
self,
num_units=10,
nonlin=F.relu,
dropout=0.5,
):
super(ClassifierModule, self).__init__()
self.num_units = num_units
self.nonlin = nonlin
self.dropout = dropout
self.dense0 = nn.Linear(20, num_units)
self.nonlin = nonlin
self.dropout = nn.Dropout(dropout)
self.dense1 = nn.Linear(num_units, 10)
self.output = nn.Linear(10, 2)
def forward(self, X, **kwargs):
X = self.nonlin(self.dense0(X))
X = self.dropout(X)
X = F.relu(self.dense1(X))
X = F.softmax(self.output(X), dim=-1)
return X
We use NeuralNetClassifier
because we're dealing with a classifcation task. The first argument should be the pytorch module
. As additional arguments, we pass the number of epochs and the learning rate (lr
), but those are optional.
Note: To use the cuda backend, pass use_cuda=True
as an additional argument.
from skorch.net import NeuralNetClassifier
net = NeuralNetClassifier(
ClassifierModule,
max_epochs=20,
lr=0.1,
# use_cuda=True, # uncomment this to train with CUDA
)
As in sklearn
, we call fit
passing the input data X
and the targets y
. By default, NeuralNetClassifier
makes a StratifiedKFold
split on the data (80/20) to track the validation loss. This is shown, as well as the train loss and the accuracy on the validation set.
pdb on
Automatic pdb calling has been turned ON
net.fit(X, y)
epoch train_loss valid_acc valid_loss dur ------- ------------ ----------- ------------ ------ 1 0.6868 0.6000 0.6740 0.0793 2 0.6706 0.6400 0.6617 0.0686 3 0.6637 0.6650 0.6504 0.0541 4 0.6548 0.7000 0.6418 0.0535 5 0.6340 0.7100 0.6272 0.0539 6 0.6219 0.7150 0.6124 0.0574 7 0.6058 0.7100 0.5980 0.0530 8 0.5964 0.7200 0.5875 0.0646 9 0.5901 0.7100 0.5760 0.0572 10 0.5716 0.7250 0.5651 0.0460 11 0.5633 0.7250 0.5580 0.0471 12 0.5652 0.7300 0.5529 0.0453 13 0.5462 0.7350 0.5426 0.0500 14 0.5407 0.7300 0.5407 0.0448 15 0.5360 0.7300 0.5373 0.0464 16 0.5517 0.7400 0.5328 0.0448 17 0.5351 0.7450 0.5277 0.0460 18 0.5280 0.7400 0.5260 0.0530 19 0.5148 0.7450 0.5264 0.0583 20 0.5309 0.7400 0.5210 0.0740
<skorch.net.NeuralNetClassifier at 0x7fb203e05668>
Also, as in sklearn
, you may call predict
or predict_proba
on the fitted model.
y_pred = net.predict(X[:5])
y_pred
array([0, 0, 0, 0, 0])
y_proba = net.predict_proba(X[:5])
y_proba
array([[ 0.54967159, 0.45032838], [ 0.7842356 , 0.2157644 ], [ 0.67652136, 0.32347867], [ 0.88522649, 0.1147735 ], [ 0.68577141, 0.31422859]], dtype=float32)
from sklearn.datasets import make_regression
X_regr, y_regr = make_regression(1000, 20, n_informative=10, random_state=0)
X_regr = X_regr.astype(np.float32)
y_regr = y_regr.astype(np.float32) / 100
y_regr = y_regr.reshape(-1, 1)
X_regr.shape, y_regr.shape, y_regr.min(), y_regr.max()
((1000, 20), (1000, 1), -6.4901485, 6.1545048)
Note: Regression currently requires the target to be 2-dimensional, hence the need to reshape. This should be fixed with an upcoming version of pytorch.
pytorch
regression module
¶Again, define a vanilla neural network with two hidden layers. The main difference is that the output layer only has one unit and does not apply a softmax nonlinearity.
class RegressorModule(nn.Module):
def __init__(
self,
num_units=10,
nonlin=F.relu,
):
super(RegressorModule, self).__init__()
self.num_units = num_units
self.nonlin = nonlin
self.dense0 = nn.Linear(20, num_units)
self.nonlin = nonlin
self.dense1 = nn.Linear(num_units, 10)
self.output = nn.Linear(10, 1)
def forward(self, X, **kwargs):
X = self.nonlin(self.dense0(X))
X = F.relu(self.dense1(X))
X = self.output(X)
return X
Training a regressor is almost the same as training a classifier. Mainly, we use NeuralNetRegressor
instead of NeuralNetClassifier
(this is the same terminology as in sklearn
).
from skorch.net import NeuralNetRegressor
net_regr = NeuralNetRegressor(
RegressorModule,
max_epochs=20,
lr=0.1,
# use_cuda=True, # uncomment this to train with CUDA
)
net_regr.fit(X_regr, y_regr)
epoch train_loss valid_loss dur ------- ------------ ------------ ------ 1 4.6059 3.5860 0.0264 2 3.5021 1.3814 0.0421 3 1.1019 0.5334 0.0436 4 0.7071 0.2994 0.0414 5 0.5654 0.4141 0.0248 6 0.3179 0.1574 0.0242 7 0.2476 0.1906 0.0269 8 0.1302 0.1049 0.0250 9 0.1373 0.1124 0.0240 10 0.0728 0.0737 0.0265 11 0.0839 0.0727 0.0247 12 0.0435 0.0513 0.0267 13 0.0508 0.0483 0.0268 14 0.0279 0.0371 0.0261 15 0.0322 0.0335 0.0275 16 0.0193 0.0282 0.0260 17 0.0224 0.0247 0.0257 18 0.0148 0.0221 0.0271 19 0.0167 0.0198 0.0264 20 0.0122 0.0182 0.0268
<skorch.net.NeuralNetRegressor at 0x7fb203dcd390>
You may call predict
or predict_proba
on the fitted model. For regressions, both methods return the same value.
y_pred = net_regr.predict(X_regr[:5])
y_pred
array([[ 0.52162153], [-1.50998139], [-0.90007448], [-0.08845913], [-0.52214217]], dtype=float32)
Save and load either the whole model by using pickle or just the learned model parameters by calling save_params
and load_params
.
import pickle
file_name = '/tmp/mymodel.pkl'
with open(file_name, 'wb') as f:
pickle.dump(net, f)
/home/bbossan_dev/anaconda3/envs/skorch/lib/python3.6/site-packages/torch/serialization.py:158: UserWarning: Couldn't retrieve source code for container of type ClassifierModule. It won't be checked for correctness upon loading. "type " + obj.__name__ + ". It won't be checked "
with open(file_name, 'rb') as f:
new_net = pickle.load(f)
This only saves and loads the proper module
parameters, meaning that hyperparameters such as lr
and max_epochs
are not saved. Therefore, to load the model, we have to re-initialize it beforehand.
net.save_params(file_name) # a file handler also works
# first initialize the model
new_net = NeuralNetClassifier(
ClassifierModule,
max_epochs=20,
lr=0.1,
).initialize()
new_net.load_params(file_name)
sklearn Pipeline
¶It is possible to put the NeuralNetClassifier
inside an sklearn Pipeline
, as you would with any sklearn
classifier.
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler
pipe = Pipeline([
('scale', StandardScaler()),
('net', net),
])
pipe.fit(X, y)
Re-initializing module! epoch train_loss valid_acc valid_loss dur ------- ------------ ----------- ------------ ------ 1 0.6891 0.5550 0.6853 0.1064 2 0.6826 0.5600 0.6825 0.0576 3 0.6873 0.5900 0.6801 0.0514 4 0.6797 0.6000 0.6776 0.0448 5 0.6772 0.6150 0.6751 0.0434 6 0.6748 0.6200 0.6723 0.0429 7 0.6682 0.6200 0.6691 0.0429 8 0.6645 0.6200 0.6654 0.0473 9 0.6623 0.6300 0.6613 0.0524 10 0.6464 0.6200 0.6555 0.0612 11 0.6471 0.6300 0.6491 0.0490 12 0.6449 0.6600 0.6424 0.0493 13 0.6285 0.6500 0.6341 0.0520 14 0.6265 0.6500 0.6261 0.0487 15 0.6252 0.6600 0.6193 0.0341 16 0.6148 0.6750 0.6102 0.0282 17 0.6039 0.6850 0.6017 0.0475 18 0.5979 0.6900 0.5949 0.0435 19 0.5794 0.7000 0.5849 0.0440 20 0.5596 0.7050 0.5758 0.0484
Pipeline(memory=None, steps=[('scale', StandardScaler(copy=True, with_mean=True, with_std=True)), ('net', <skorch.net.NeuralNetClassifier object at 0x7fb203e05668>)])
y_proba = pipe.predict_proba(X[:5])
y_proba
array([[ 0.39650354, 0.60349649], [ 0.73950195, 0.26049808], [ 0.72104084, 0.27895918], [ 0.71111423, 0.2888858 ], [ 0.66332674, 0.33667326]], dtype=float32)
To save the whole pipeline, including the pytorch module, use pickle
.
Adding a new callback to the model is straightforward. Below we show how to add a new callback that determines the area under the ROC (AUC) score.
from skorch.callbacks import EpochScoring
There is a scoring callback in skorch, EpochScoring
, which we use for this. We have to specify which score to calculate. We have 3 choices:
sklearn
metric. For a list of all existing scores, look here.None
: If you implement your own .score
method on your neural net, passing scoring=None
will tell skorch
to use that.func(model, X, y) -> score
, which is then used.Note that this works exactly the same as scoring in sklearn
does.
For our case here, since sklearn
already implements AUC, we just pass the correct string 'roc_auc'
. We should also tell the callback that higher scores are better (to get the correct colors printed below -- by default, lower scores are assumed to be better). Furthermore, we may specify a name
argument for EpochScoring
, and whether to use training data (by setting on_train=True
) or validation data (which is the default).
auc = EpochScoring(scoring='roc_auc', lower_is_better=False)
Finally, we pass the scoring callback to the callbacks
parameter as a list and then call fit
. Notice that we get the printed scores and color highlighting for free.
net = NeuralNetClassifier(
ClassifierModule,
max_epochs=20,
lr=0.1,
callbacks=[auc],
)
net.fit(X, y)
epoch roc_auc train_loss valid_acc valid_loss dur ------- --------- ------------ ----------- ------------ ------ 1 0.5911 0.7204 0.5000 0.6948 0.0590 2 0.6524 0.6925 0.5300 0.6881 0.0502 3 0.6700 0.6867 0.6000 0.6857 0.0321 4 0.6854 0.6820 0.6400 0.6832 0.0364 5 0.6829 0.6801 0.6050 0.6812 0.0377 6 0.6757 0.6742 0.6100 0.6796 0.0541 7 0.6808 0.6762 0.6100 0.6776 0.0528 8 0.6759 0.6576 0.6350 0.6747 0.0317 9 0.6813 0.6661 0.6350 0.6707 0.0525 10 0.6903 0.6548 0.6450 0.6655 0.0467 11 0.6929 0.6500 0.6400 0.6611 0.0495 12 0.6920 0.6445 0.6500 0.6571 0.0314 13 0.7095 0.6372 0.6650 0.6509 0.0390 14 0.7155 0.6288 0.6700 0.6446 0.0532 15 0.7265 0.6268 0.6700 0.6390 0.0494 16 0.7398 0.6150 0.6900 0.6308 0.0609 17 0.7487 0.6221 0.7000 0.6246 0.0540 18 0.7473 0.6168 0.7250 0.6187 0.0529 19 0.7588 0.5945 0.7400 0.6100 0.0522 20 0.7664 0.6000 0.7650 0.6026 0.0524
<skorch.net.NeuralNetClassifier at 0x7fb203db76a0>
For information on how to write custom callbacks, have a look at the Advanced_Usage notebook.
GridSearchCV
¶The NeuralNet
class allows to directly access parameters of the pytorch module
by using the module__
prefix. So e.g. if you defined the module
to have a num_units
parameter, you can set it via the module__num_units
argument. This is exactly the same logic that allows to access estimator parameters in sklearn Pipeline
s and FeatureUnion
s.
This feature is useful in several ways. For one, it allows to set those parameters in the model definition. Furthermore, it allows you to set parameters in an sklearn GridSearchCV
as shown below.
In addition to the parameters prefixed by module__
, you may access a couple of other attributes, such as those of the optimizer by using the optimizer__
prefix (again, see below). All those special prefixes are stored in the prefixes_
attribute:
print(', '.join(net.prefixes_))
module, iterator_train, iterator_valid, optimizer, criterion, callbacks, dataset
Below we show how to perform a grid search over the learning rate (lr
), the module's number of hidden units (module__num_units
), the module's dropout rate (module__dropout
), and whether the SGD optimizer should use Nesterov momentum or not (optimizer__nesterov
).
from sklearn.model_selection import GridSearchCV
net = NeuralNetClassifier(
ClassifierModule,
max_epochs=20,
lr=0.1,
verbose=0,
optimizer__momentum=0.9,
)
params = {
'lr': [0.05, 0.1],
'module__num_units': [10, 20],
'module__dropout': [0, 0.5],
'optimizer__nesterov': [False, True],
}
gs = GridSearchCV(net, params, refit=False, cv=3, scoring='accuracy', verbose=2)
gs.fit(X, y)
Fitting 3 folds for each of 16 candidates, totalling 48 fits [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 0.9s [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False
[Parallel(n_jobs=1)]: Done 1 out of 1 | elapsed: 0.9s remaining: 0.0s
[CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 1.1s [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 2.4s [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 2.4s [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 1.0s [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 1.0s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 0.9s [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.9s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 1.0s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 1.0s [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 0.9s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 1.0s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.05, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.9s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 0.9s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.7s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.7s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 0.9s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=10, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=False, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.8s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.9s [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True [CV] lr=0.1, module__dropout=0.5, module__num_units=20, optimizer__nesterov=True, total= 0.9s
[Parallel(n_jobs=1)]: Done 48 out of 48 | elapsed: 44.6s finished
GridSearchCV(cv=3, error_score='raise', estimator=<skorch.net.NeuralNetClassifier object at 0x7fb203dc3320>, fit_params=None, iid=True, n_jobs=1, param_grid={'lr': [0.05, 0.1], 'module__num_units': [10, 20], 'module__dropout': [0, 0.5], 'optimizer__nesterov': [False, True]}, pre_dispatch='2*n_jobs', refit=False, return_train_score=True, scoring='accuracy', verbose=2)
print(gs.best_score_, gs.best_params_)
0.856 {'lr': 0.1, 'module__dropout': 0, 'module__num_units': 20, 'optimizer__nesterov': False}
Of course, we could further nest the NeuralNetClassifier
within an sklearn Pipeline
, in which case we just prefix the parameter by the name of the net (e.g. net__module__num_units
).