下ごしらえ
from sklearn.datasets import load_digits
digits=load_digits()
from sklearn.cross_validation import train_test_split
data_train,data_test,target_train,target_test=train_test_split(
digits.data,digits.target,train_size=0.95)
from autosklearn.classification import AutoSklearnClassifier
sklearnのAPIは基本的には共通。モデル間の比較もしやすいはず。
当然、全部まとめてやりたいって考えが出てくる。
説明はPyLadysTokyoのスライドが詳しい。
中身は私もよくわかっていない。。。
clf = AutoSklearnClassifier(time_left_for_this_task=360, per_run_time_limit=36)
# 時間は短くしてあります。
いつもどおりfit。
outputがいっぱい出る。
clf.fit(data_train,target_train)
[INFO] [2016-05-21 11:53:09,780:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Starting to create dummy predictions. [INFO] [2016-05-21 11:53:09,975:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Finished creating dummy prediction 1/2. [INFO] [2016-05-21 11:53:10,171:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Finished creating dummy prediction 2/2. [INFO] [2016-05-21 11:53:14,572:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Start Ensemble with 355.19sec time left [INFO] [2016-05-21 11:53:14,591:AutoML(1):6be0c6791d6c4877e53f8a237238200e] Start SMAC with 355.17sec time left [ERROR] [2016-05-21 11:53:14,773:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:53:14,785:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Training default configurations on a subset of 563/1707 data points. [INFO] [2016-05-21 11:53:14,969:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.003355 Members: [0] Weights: [ 1.] Identifiers: (1, 2)[INFO] [2016-05-21 11:53:14,973:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 3 on SUBSET with size 563 and time limit 18s. [INFO] [2016-05-21 11:53:14,974:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: random_forest classifier:random_forest:bootstrap, Value: True classifier:random_forest:criterion, Value: gini classifier:random_forest:max_depth, Constant: None classifier:random_forest:max_features, Value: 1.0 classifier:random_forest:max_leaf_nodes, Constant: None classifier:random_forest:min_samples_leaf, Value: 1 classifier:random_forest:min_samples_split, Value: 2 classifier:random_forest:min_weight_fraction_leaf, Constant: 0.0 classifier:random_forest:n_estimators, Constant: 100 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.01 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: no_preprocessing rescaling:__choice__, Value: min/max [INFO] [2016-05-21 11:53:14,976:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.003355 [INFO] [2016-05-21 11:53:14,978:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 0.207753 seconds [INFO] [2016-05-21 11:53:14,981:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (2)!. [INFO] [2016-05-21 11:53:14,988:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (2)! [INFO] [2016-05-21 11:53:16,042:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 3. configuration on SUBSET. Duration 1.017266; loss 0.047365; status 1; additional run info: ;duration: 1.0172662734985352;num_run:00003 [INFO] [2016-05-21 11:53:16,164:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000 [INFO] [2016-05-21 11:53:16,168:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 4 on SUBSET with size 563 and time limit 18s. [INFO] [2016-05-21 11:53:16,169:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: sgd classifier:sgd:alpha, Value: 0.0001 classifier:sgd:average, Value: True classifier:sgd:eta0, Value: 0.01 classifier:sgd:fit_intercept, Constant: True classifier:sgd:learning_rate, Value: optimal classifier:sgd:loss, Value: hinge classifier:sgd:n_iter, Value: 5 classifier:sgd:penalty, Value: l2 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.1 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: no_preprocessing rescaling:__choice__, Value: min/max [INFO] [2016-05-21 11:53:16,580:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 4. configuration on SUBSET. Duration 0.218824; loss 0.063154; status 1; additional run info: ;duration: 0.2188243865966797;num_run:00004 [INFO] [2016-05-21 11:53:16,582:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000 [INFO] [2016-05-21 11:53:16,586:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 5 on SUBSET with size 563 and time limit 18s. [INFO] [2016-05-21 11:53:16,588:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: extra_trees classifier:extra_trees:bootstrap, Value: False classifier:extra_trees:criterion, Value: gini classifier:extra_trees:max_depth, Constant: None classifier:extra_trees:max_features, Value: 1.0 classifier:extra_trees:min_samples_leaf, Value: 5 classifier:extra_trees:min_samples_split, Value: 5 classifier:extra_trees:min_weight_fraction_leaf, Constant: 0.0 classifier:extra_trees:n_estimators, Constant: 100 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.1 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: select_percentile_classification preprocessor:select_percentile_classification:percentile, Value: 2.0 preprocessor:select_percentile_classification:score_func, Value: chi2 rescaling:__choice__, Value: min/max [ERROR] [2016-05-21 11:53:17,167:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:53:18,769:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 5. configuration on SUBSET. Duration 1.807020; loss 0.708506; status 1; additional run info: ;duration: 1.8070201873779297;num_run:00005 [INFO] [2016-05-21 11:53:18,771:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000 [INFO] [2016-05-21 11:53:18,774:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 6 on SUBSET with size 563 and time limit 18s. [INFO] [2016-05-21 11:53:18,776:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: gaussian_nb imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.1 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: no_preprocessing rescaling:__choice__, Value: standardize [INFO] [2016-05-21 11:53:18,967:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.952635 6: 0.952635 7: 0.956582 8: 0.956582 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.962502 13: 0.962502 14: 0.962502 15: 0.958555 16: 0.958555 17: 0.958555 18: 0.958555 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.962502 24: 0.962502 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.962502 29: 0.962502 30: 0.962502 31: 0.960529 32: 0.962502 33: 0.962502 34: 0.962502 35: 0.962502 36: 0.962502 37: 0.962502 38: 0.962502 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502 Members: [1, 1, 1, 1, 1, 1, 0, 2, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 2, 1, 0, 1, 1, 1, 1, 1, 0, 2, 1, 1, 1, 1, 0, 1, 1, 1, 1, 1, 1, 1, 1, 0, 2, 1, 1] Weights: [ 0.14 0.78 0.08] Identifiers: (1, 2) (1, 3) (1, 4) [INFO] [2016-05-21 11:53:18,972:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502 [INFO] [2016-05-21 11:53:18,973:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 1.808918 seconds [INFO] [2016-05-21 11:53:18,976:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (4)!. [INFO] [2016-05-21 11:53:18,977:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (4)! [ERROR] [2016-05-21 11:53:18,981:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:53:19,166:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 6. configuration on SUBSET. Duration 0.192416; loss 0.211170; status 1; additional run info: ;duration: 0.19241595268249512;num_run:00006 [INFO] [2016-05-21 11:53:19,169:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished SUBSET training sucessfully with ratio 0.330000 [INFO] [2016-05-21 11:53:19,170:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Metadata directory: /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/autosklearn/metalearning/files/acc_metric_multiclass.classification_dense [INFO] [2016-05-21 11:53:20,989:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.954608 6: 0.954608 7: 0.954608 8: 0.954608 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.960529 13: 0.960529 14: 0.960529 15: 0.960529 16: 0.960529 17: 0.960529 18: 0.960529 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.960529 24: 0.960529 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.960529 29: 0.962502 30: 0.962502 31: 0.962502 32: 0.962502 33: 0.962502 34: 0.962502 35: 0.962502 36: 0.962502 37: 0.962502 38: 0.962502 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502 Members: [1, 1, 1, 1, 1, 3, 1, 1, 0, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 0, 2, 3, 3, 0, 1, 1, 1, 1, 1, 1, 2, 0, 1, 1, 0, 1, 1, 1, 1, 2, 0, 1, 1, 1, 1] Weights: [ 0.12 0.72 0.06 0.1 ] Identifiers: (1, 2) (1, 3) (1, 4) (1, 5) [INFO] [2016-05-21 11:53:20,993:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502 [INFO] [2016-05-21 11:53:20,994:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 2.014419 seconds [INFO] [2016-05-21 11:53:21,165:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (5)!. [INFO] [2016-05-21 11:53:21,166:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (5)! [ERROR] [2016-05-21 11:53:21,171:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:53:22,796:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Start calculating metafeatures for 6be0c6791d6c4877e53f8a237238200e
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/numpy/lib/nanfunctions.py:1147: RuntimeWarning: Degrees of freedom <= 0 for slice. warnings.warn("Degrees of freedom <= 0 for slice.", RuntimeWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/numpy/lib/nanfunctions.py:675: RuntimeWarning: Mean of empty slice warnings.warn("Mean of empty slice", RuntimeWarning)
[INFO] [2016-05-21 11:53:22,979:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Calculating Metafeatures (categorical attributes) took 0.18 [INFO] [2016-05-21 11:53:23,588:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.952635 1: 0.952635 2: 0.952635 3: 0.952635 4: 0.952635 5: 0.954608 6: 0.954608 7: 0.954608 8: 0.954608 9: 0.956582 10: 0.956582 11: 0.958555 12: 0.960529 13: 0.960529 14: 0.960529 15: 0.960529 16: 0.960529 17: 0.960529 18: 0.960529 19: 0.960529 20: 0.960529 21: 0.960529 22: 0.960529 23: 0.960529 24: 0.960529 25: 0.962502 26: 0.962502 27: 0.962502 28: 0.962502 29: 0.962502 30: 0.962502 31: 0.962502 32: 0.962502 33: 0.962502 34: 0.964476 35: 0.962502 36: 0.962502 37: 0.960529 38: 0.960529 39: 0.962502 40: 0.962502 41: 0.962502 42: 0.962502 43: 0.962502 44: 0.962502 45: 0.962502 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502 Members: [1, 1, 1, 1, 1, 3, 1, 1, 0, 1, 1, 1, 3, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 3, 0, 2, 3, 3, 4, 1, 1, 1, 1, 1, 2, 0, 0, 1, 1, 1, 1, 3, 1, 1, 1, 3, 1, 1, 1, 3] Weights: [ 0.08 0.7 0.04 0.16 0.02] Identifiers: (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) [INFO] [2016-05-21 11:53:23,593:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502 [INFO] [2016-05-21 11:53:23,594:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 2.425635 seconds [INFO] [2016-05-21 11:53:23,597:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (6)!. [INFO] [2016-05-21 11:53:23,598:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (6)! [INFO] [2016-05-21 11:53:23,771:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Calculating Metafeatures (encoded attributes) took 0.77sec
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/validation.py:386: DeprecationWarning: Passing 1d arrays as data is deprecated in 0.17 and willraise ValueError in 0.19. Reshape your data either using X.reshape(-1, 1) if your data has a single feature or X.reshape(1, -1) if it contains a single sample. DeprecationWarning)
[INFO] [2016-05-21 11:53:24,211:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Time left for 6be0c6791d6c4877e53f8a237238200e after finding initial configurations: 340.55sec [INFO] [2016-05-21 11:53:24,215:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Metafeatures for dataset 6be0c6791d6c4877e53f8a237238200euiaeo KurtosisMean: 79.9312077309 ClassProbabilityMin: 0.09431751611013474 SkewnessMean: 3.70700734283 ClassEntropy: 3.32137035936 NumberOfCategoricalFeatures: 0 KurtosisSTD: 249.225704121 SymbolsMean: 0 SymbolsSTD: 0 PercentageOfFeaturesWithMissingValues: 0.0 NumberOfInstancesWithMissingValues: 0.0 InverseDatasetRatio: 26.671875 NumberOfFeatures: 64.0 RatioNumericalToNominal: 0.0 LogInverseDatasetRatio: 3.28360963943 NumberOfFeaturesWithMissingValues: 0.0 KurtosisMin: -3.0 NumberOfNumericFeatures: 64 SymbolsSum: 0.0 SkewnessSTD: 7.95909987575 LogNumberOfInstances: 7.44249272279 NumberOfClasses: 10.0 SkewnessMax: 39.5696193954 SymbolsMin: 0 ClassProbabilitySTD: 0.00277323969034 ClassProbabilityMax: 0.10369068541300527 PercentageOfInstancesWithMissingValues: 0.0 LandmarkRandomNodeLearner: 0.102491005855 ClassProbabilityMean: 0.1 PercentageOfMissingValues: 0.0 LogDatasetRatio: -3.28360963943 NumberOfMissingValues: 0.0 SymbolsMax: 0 NumberOfInstances: 1707.0 RatioNominalToNumerical: 0.0 DatasetRatio: 0.03749267721148213 SkewnessMin: -1.33853375774 KurtosisMax: 1601.02833276 LogNumberOfFeatures: 4.15888308336 [INFO] [2016-05-21 11:53:24,224:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] ['3_acc', '971_acc', '718_acc', '180_acc', '1018_acc', '797_acc', '401_acc', '1036_acc', '184_acc', '300_acc', '897_acc', '46_acc', '722_acc', '30_acc', '934_acc', '723_acc', '351_acc', '903_acc', '912_acc', '1112_acc', '354_acc', '913_acc', '21_acc', '1139_acc', '1021_acc', '36_acc', '803_acc', '837_acc', '847_acc', '396_acc', '12_acc', '843_acc', '392_acc', '833_acc', '1120_acc', '737_acc', '16_acc', '31_acc', '1000_acc', '1049_acc', '813_acc', '904_acc', '735_acc', '44_acc', '1040_acc', '1020_acc', '846_acc', '38_acc', '741_acc', '182_acc', '293_acc', '1116_acc', '966_acc', '60_acc', '980_acc', '772_acc', '727_acc', '1041_acc', '761_acc', '554_acc', '57_acc', '185_acc', '1053_acc', '715_acc', '32_acc', '1119_acc', '1069_acc', '395_acc', '1128_acc', '979_acc', '181_acc', '849_acc', '910_acc', '23_acc', '6_acc', '1114_acc', '1142_acc', '819_acc', '1002_acc', '978_acc', '1056_acc', '393_acc', '799_acc', '871_acc', '914_acc', '740_acc', '977_acc', '976_acc', '959_acc', '752_acc', '390_acc', '398_acc', '881_acc', '1138_acc', '1166_acc', '917_acc', '823_acc', '273_acc', '866_acc', '399_acc', '28_acc', '22_acc', '1130_acc', '389_acc', '6be0c6791d6c4877e53f8a237238200euiaeo', '1134_acc', '14_acc', '958_acc', '1146_acc', '821_acc', '1111_acc', '822_acc', '930_acc', '179_acc', '26_acc', '901_acc', '993_acc', '845_acc', '807_acc', '995_acc', '1019_acc', '751_acc', '1161_acc', '962_acc', '816_acc', '743_acc', '679_acc', '734_acc', '953_acc', '24_acc', '720_acc', '806_acc', '991_acc', '391_acc', '923_acc', '728_acc', '18_acc', '1067_acc', '1050_acc', '1068_acc', '357_acc'] [INFO] [2016-05-21 11:55:50,225:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 7. configuration (default configuration) with time limit 36s. [INFO] [2016-05-21 11:55:50,228:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: random_forest classifier:random_forest:bootstrap, Value: True classifier:random_forest:criterion, Value: gini classifier:random_forest:max_depth, Constant: None classifier:random_forest:max_features, Value: 1.0 classifier:random_forest:max_leaf_nodes, Constant: None classifier:random_forest:min_samples_leaf, Value: 1 classifier:random_forest:min_samples_split, Value: 2 classifier:random_forest:min_weight_fraction_leaf, Constant: 0.0 classifier:random_forest:n_estimators, Constant: 100 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.01 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: no_preprocessing rescaling:__choice__, Value: min/max [INFO] [2016-05-21 11:55:52,170:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 7. configuration. Duration 1.624692; loss 0.035524; status 1; additional run info: ;duration: 1.6246922016143799;num_run:00007 [INFO] [2016-05-21 11:55:52,172:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 8. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:55:52,176:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: libsvm_svc classifier:libsvm_svc:C, Value: 2.820950683561126 classifier:libsvm_svc:coef0, Value: 0.7525658406326312 classifier:libsvm_svc:degree, Value: 5 classifier:libsvm_svc:gamma, Value: 1.924396413436426 classifier:libsvm_svc:kernel, Value: poly classifier:libsvm_svc:max_iter, Constant: -1 classifier:libsvm_svc:shrinking, Value: True classifier:libsvm_svc:tol, Value: 0.00037682073690554086 imputation:strategy, Value: mean one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: select_percentile_classification preprocessor:select_percentile_classification:percentile, Value: 83.60106850579331 preprocessor:select_percentile_classification:score_func, Value: f_classif rescaling:__choice__, Value: min/max
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/feature_selection/univariate_selection.py:113: UserWarning: Features [ 0 32 39 56] are constant. UserWarning)
[ERROR] [2016-05-21 11:55:52,388:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning)
[INFO] [2016-05-21 11:55:53,001:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 8. configuration. Duration 0.778786; loss 0.011841; status 1; additional run info: ;duration: 0.7787859439849854;num_run:00008 [INFO] [2016-05-21 11:55:53,165:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 9. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:55:53,167:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: liblinear_svc classifier:liblinear_svc:C, Value: 0.5815868776595028 classifier:liblinear_svc:dual, Constant: False classifier:liblinear_svc:fit_intercept, Constant: True classifier:liblinear_svc:intercept_scaling, Constant: 1 classifier:liblinear_svc:loss, Value: squared_hinge classifier:liblinear_svc:multi_class, Constant: ovr classifier:liblinear_svc:penalty, Value: l2 classifier:liblinear_svc:tol, Value: 0.05261321190581748 imputation:strategy, Value: most_frequent one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: polynomial preprocessor:polynomial:degree, Value: 2 preprocessor:polynomial:include_bias, Value: True preprocessor:polynomial:interaction_only, Value: False rescaling:__choice__, Value: none
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19 " 0.19", DeprecationWarning)
[INFO] [2016-05-21 11:55:56,196:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.964476 1: 0.964476 2: 0.964476 3: 0.966450 4: 0.966450 5: 0.970397 6: 0.970397 7: 0.968423 8: 0.970397 9: 0.972370 10: 0.972370 11: 0.972370 12: 0.970397 13: 0.970397 14: 0.970397 15: 0.970397 16: 0.970397 17: 0.970397 18: 0.970397 19: 0.970397 20: 0.968423 21: 0.966450 22: 0.964476 23: 0.966450 24: 0.966450 25: 0.966450 26: 0.966450 27: 0.966450 28: 0.966450 29: 0.966450 30: 0.966450 31: 0.966450 32: 0.966450 33: 0.966450 34: 0.966450 35: 0.966450 36: 0.966450 37: 0.966450 38: 0.966450 39: 0.966450 40: 0.966450 41: 0.964476 42: 0.962502 43: 0.960529 44: 0.960529 45: 0.960529 46: 0.962502 47: 0.962502 48: 0.962502 49: 0.962502 Members: [5, 5, 5, 1, 5, 5, 5, 5, 4, 1, 1, 5, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 1, 5, 5, 1, 5, 3, 4, 2, 1, 5, 1, 5, 5, 0, 5, 5, 1, 4, 1, 0, 0, 1, 1, 1, 1, 1, 1, 1] Weights: [ 0.1 0.48 0.02 0.02 0.06 0.32] Identifiers: (1, 2) (1, 3) (1, 4) (1, 5) (1, 6) (1, 7) [INFO] [2016-05-21 11:55:56,365:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.962502 [INFO] [2016-05-21 11:55:56,366:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 3.981098 seconds [INFO] [2016-05-21 11:55:56,370:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (7)!. [INFO] [2016-05-21 11:55:56,371:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (7)! [ERROR] [2016-05-21 11:55:56,377:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:55:58,583:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 9. configuration. Duration 5.013263; loss 0.011841; status 1; additional run info: ;duration: 5.013262987136841;num_run:00009 [INFO] [2016-05-21 11:55:58,586:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 10. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:55:58,588:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: adaboost classifier:adaboost:algorithm, Value: SAMME classifier:adaboost:learning_rate, Value: 1.2306208006800998 classifier:adaboost:max_depth, Value: 6 classifier:adaboost:n_estimators, Value: 499 imputation:strategy, Value: median one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: extra_trees_preproc_for_classification preprocessor:extra_trees_preproc_for_classification:bootstrap, Value: True preprocessor:extra_trees_preproc_for_classification:criterion, Value: entropy preprocessor:extra_trees_preproc_for_classification:max_depth, Constant: None preprocessor:extra_trees_preproc_for_classification:max_features, Value: 3.5347851525007146 preprocessor:extra_trees_preproc_for_classification:min_samples_leaf, Value: 6 preprocessor:extra_trees_preproc_for_classification:min_samples_split, Value: 8 preprocessor:extra_trees_preproc_for_classification:min_weight_fraction_leaf, Constant: 0.0 preprocessor:extra_trees_preproc_for_classification:n_estimators, Constant: 100 rescaling:__choice__, Value: standardize [INFO] [2016-05-21 11:56:00,765:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159 Members: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6] Weights: [ 0.2 0. 0. 0. 0. 0.18 0.62] Identifiers: (1, 2) (1, 7) (1, 8) [INFO] [2016-05-21 11:56:00,770:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159 [INFO] [2016-05-21 11:56:00,771:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.396333 seconds [INFO] [2016-05-21 11:56:00,774:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (8)!. [INFO] [2016-05-21 11:56:00,775:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (8)! [ERROR] [2016-05-21 11:56:00,781:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning)
[INFO] [2016-05-21 11:56:04,798:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159 Members: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6] Weights: [ 0.2 0. 0. 0. 0. 0.18 0.62 0. ] Identifiers: (1, 2) (1, 7) (1, 8) [INFO] [2016-05-21 11:56:04,965:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159 [INFO] [2016-05-21 11:56:04,967:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.187559 seconds [INFO] [2016-05-21 11:56:04,968:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning)
[INFO] [2016-05-21 11:56:12,565:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 10. configuration. Duration 13.453381; loss 0.029603; status 1; additional run info: ;duration: 13.453380584716797;num_run:00010 [INFO] [2016-05-21 11:56:12,568:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 11. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:56:12,569:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: sgd classifier:sgd:alpha, Value: 0.0002011208878338422 classifier:sgd:average, Value: False classifier:sgd:eta0, Value: 0.0812309539678548 classifier:sgd:fit_intercept, Constant: True classifier:sgd:l1_ratio, Value: 0.49756924007684844 classifier:sgd:learning_rate, Value: invscaling classifier:sgd:loss, Value: hinge classifier:sgd:n_iter, Value: 554 classifier:sgd:penalty, Value: elasticnet classifier:sgd:power_t, Value: 0.2567923580475304 imputation:strategy, Value: most_frequent one_hot_encoding:minimum_fraction, Value: 0.0007915092939712602 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: nystroem_sampler preprocessor:nystroem_sampler:coef0, Value: -0.6384046151685252 preprocessor:nystroem_sampler:degree, Value: 3 preprocessor:nystroem_sampler:gamma, Value: 0.22795987796995726 preprocessor:nystroem_sampler:kernel, Value: poly preprocessor:nystroem_sampler:n_components, Value: 3207 rescaling:__choice__, Value: none
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/kernel_approximation.py:463: UserWarning: n_components > n_samples. This is not possible. n_components was set to n_samples, which results in inefficient evaluation of the full kernel. warnings.warn("n_components > n_samples. This is not possible.\n" You are already timing task: index_run6
[ERROR] [2016-05-21 11:56:13,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:56:21,963:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.988159 1: 0.988159 2: 0.988159 3: 0.988159 4: 0.988159 5: 0.988159 6: 0.988159 7: 0.988159 8: 0.988159 9: 0.988159 10: 0.988159 11: 0.988159 12: 0.988159 13: 0.988159 14: 0.988159 15: 0.988159 16: 0.988159 17: 0.988159 18: 0.988159 19: 0.988159 20: 0.988159 21: 0.988159 22: 0.988159 23: 0.988159 24: 0.988159 25: 0.988159 26: 0.988159 27: 0.988159 28: 0.988159 29: 0.988159 30: 0.988159 31: 0.988159 32: 0.988159 33: 0.988159 34: 0.988159 35: 0.988159 36: 0.988159 37: 0.988159 38: 0.988159 39: 0.988159 40: 0.988159 41: 0.988159 42: 0.988159 43: 0.988159 44: 0.988159 45: 0.988159 46: 0.988159 47: 0.988159 48: 0.988159 49: 0.988159 Members: [6, 6, 6, 6, 0, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 6, 0, 6, 5, 6, 0, 6, 6, 5, 6, 0, 6, 6, 6, 0, 5, 6, 6, 6, 0, 5, 6, 6] Weights: [ 0.2 0. 0. 0. 0. 0.18 0.62 0. 0. ] Identifiers: (1, 2) (1, 7) (1, 8) [INFO] [2016-05-21 11:56:22,166:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.988159 [INFO] [2016-05-21 11:56:22,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 9.002641 seconds [INFO] [2016-05-21 11:56:22,170:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change. [INFO] [2016-05-21 11:56:49,615:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 11. configuration. Duration 37.038066; loss 2.000000; status 2; additional run info: [INFO] [2016-05-21 11:56:49,618:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 12. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:56:49,763:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: proj_logit classifier:proj_logit:max_epochs, Value: 2 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.0011870322904213703 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: no_preprocessing rescaling:__choice__, Value: none [INFO] [2016-05-21 11:56:50,385:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 12. configuration. Duration 0.387105; loss 0.055260; status 1; additional run info: ;duration: 0.38710546493530273;num_run:00012 [INFO] [2016-05-21 11:56:50,388:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 13. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:56:50,389:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: adaboost classifier:adaboost:algorithm, Value: SAMME classifier:adaboost:learning_rate, Value: 1.0081104516473922 classifier:adaboost:max_depth, Value: 6 classifier:adaboost:n_estimators, Value: 468 imputation:strategy, Value: mean one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: liblinear_svc_preprocessor preprocessor:liblinear_svc_preprocessor:C, Value: 1.1828431725901418 preprocessor:liblinear_svc_preprocessor:dual, Constant: False preprocessor:liblinear_svc_preprocessor:fit_intercept, Constant: True preprocessor:liblinear_svc_preprocessor:intercept_scaling, Constant: 1 preprocessor:liblinear_svc_preprocessor:loss, Value: squared_hinge preprocessor:liblinear_svc_preprocessor:multi_class, Constant: ovr preprocessor:liblinear_svc_preprocessor:penalty, Constant: l1 preprocessor:liblinear_svc_preprocessor:tol, Value: 0.0022792606924326923 rescaling:__choice__, Value: min/max
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) You are already timing task: index_run6
[ERROR] [2016-05-21 11:56:52,240:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:56:57,164:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.988159 1: 0.990132 2: 0.990132 3: 0.992106 4: 0.992106 5: 0.992106 6: 0.992106 7: 0.992106 8: 0.992106 9: 0.992106 10: 0.992106 11: 0.992106 12: 0.992106 13: 0.992106 14: 0.992106 15: 0.992106 16: 0.992106 17: 0.992106 18: 0.992106 19: 0.992106 20: 0.992106 21: 0.992106 22: 0.992106 23: 0.992106 24: 0.992106 25: 0.992106 26: 0.992106 27: 0.992106 28: 0.992106 29: 0.992106 30: 0.992106 31: 0.992106 32: 0.992106 33: 0.992106 34: 0.992106 35: 0.992106 36: 0.992106 37: 0.992106 38: 0.992106 39: 0.992106 40: 0.992106 41: 0.992106 42: 0.992106 43: 0.992106 44: 0.992106 45: 0.992106 46: 0.992106 47: 0.992106 48: 0.992106 49: 0.992106 Members: [6, 9, 1, 7, 8, 8, 7, 1, 7, 0, 1, 6, 7, 1, 1, 1, 7, 0, 1, 7, 1, 7, 1, 6, 7, 0, 1, 7, 1, 1, 7, 1, 5, 7, 1, 5, 7, 5, 7, 1, 7, 1, 6, 0, 1, 7, 1, 5, 7, 1] Weights: [ 0.08 0.38 0. 0. 0. 0.08 0.08 0.32 0.04 0.02] Identifiers: (1, 2) (1, 3) (1, 7) (1, 8) (1, 9) (1, 10) (1, 12) [INFO] [2016-05-21 11:56:57,169:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.992106 [INFO] [2016-05-21 11:56:57,170:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 4.932312 seconds [INFO] [2016-05-21 11:56:57,173:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (11)!. [INFO] [2016-05-21 11:56:57,175:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (11)!
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning)
[INFO] [2016-05-21 11:57:08,021:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 13. configuration. Duration 17.245130; loss 0.017762; status 1; additional run info: ;duration: 17.24513030052185;num_run:00013 [INFO] [2016-05-21 11:57:08,024:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 14. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:57:08,025:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: lda classifier:lda:n_components, Value: 242 classifier:lda:shrinkage, Value: None classifier:lda:tol, Value: 0.0011339574479631279 imputation:strategy, Value: most_frequent one_hot_encoding:minimum_fraction, Value: 0.004889144803037605 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: gem preprocessor:gem:N, Value: 8 preprocessor:gem:precond, Value: 0.39344596751061517 rescaling:__choice__, Value: min/max [ERROR] [2016-05-21 11:57:09,368:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:57:20,786:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 14. configuration. Duration 12.396692; loss 0.009868; status 1; additional run info: ;duration: 12.396692276000977;num_run:00014 [INFO] [2016-05-21 11:57:20,788:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 15. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:57:20,790:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: libsvm_svc classifier:libsvm_svc:C, Value: 2099.097215335698 classifier:libsvm_svc:coef0, Value: 0.32850098053260424 classifier:libsvm_svc:degree, Value: 3 classifier:libsvm_svc:gamma, Value: 2.809557625370355 classifier:libsvm_svc:kernel, Value: poly classifier:libsvm_svc:max_iter, Constant: -1 classifier:libsvm_svc:shrinking, Value: False classifier:libsvm_svc:tol, Value: 0.003213729117706896 imputation:strategy, Value: most_frequent one_hot_encoding:minimum_fraction, Value: 0.0033839910940432396 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: select_rates preprocessor:select_rates:alpha, Value: 0.08594367101091721 preprocessor:select_rates:mode, Value: fpr preprocessor:select_rates:score_func, Value: chi2 rescaling:__choice__, Value: min/max [INFO] [2016-05-21 11:57:20,992:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.988159 1: 0.990132 2: 0.990132 3: 0.992106 4: 0.992106 5: 0.992106 6: 0.992106 7: 0.992106 8: 0.992106 9: 0.992106 10: 0.992106 11: 0.992106 12: 0.992106 13: 0.992106 14: 0.992106 15: 0.992106 16: 0.992106 17: 0.992106 18: 0.992106 19: 0.992106 20: 0.992106 21: 0.992106 22: 0.992106 23: 0.992106 24: 0.992106 25: 0.992106 26: 0.992106 27: 0.992106 28: 0.992106 29: 0.992106 30: 0.992106 31: 0.992106 32: 0.992106 33: 0.992106 34: 0.992106 35: 0.992106 36: 0.992106 37: 0.992106 38: 0.992106 39: 0.992106 40: 0.992106 41: 0.992106 42: 0.992106 43: 0.992106 44: 0.992106 45: 0.992106 46: 0.992106 47: 0.992106 48: 0.992106 49: 0.992106 Members: [6, 9, 1, 7, 8, 8, 7, 1, 7, 0, 1, 6, 7, 1, 1, 1, 7, 0, 1, 7, 1, 7, 1, 6, 7, 0, 1, 7, 1, 1, 7, 1, 5, 7, 1, 5, 7, 5, 7, 1, 7, 1, 6, 0, 1, 7, 1, 5, 7, 1] Weights: [ 0.08 0.38 0. 0. 0. 0.08 0.08 0.32 0.04 0.02 0. ] Identifiers: (1, 2) (1, 3) (1, 7) (1, 8) (1, 9) (1, 10) (1, 12) [INFO] [2016-05-21 11:57:20,998:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.992106 [INFO] [2016-05-21 11:57:21,164:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 11.798627 seconds [INFO] [2016-05-21 11:57:21,165:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19 " 0.19", DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning)
[INFO] [2016-05-21 11:57:22,002:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 15. configuration. Duration 0.992510; loss 0.013815; status 1; additional run info: ;duration: 0.9925096035003662;num_run:00015 [INFO] [2016-05-21 11:57:22,004:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 16. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:57:22,006:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: liblinear_svc classifier:liblinear_svc:C, Value: 62.21750028494345 classifier:liblinear_svc:dual, Constant: False classifier:liblinear_svc:fit_intercept, Constant: True classifier:liblinear_svc:intercept_scaling, Constant: 1 classifier:liblinear_svc:loss, Value: squared_hinge classifier:liblinear_svc:multi_class, Constant: ovr classifier:liblinear_svc:penalty, Value: l2 classifier:liblinear_svc:tol, Value: 0.0002178180133457927 imputation:strategy, Value: most_frequent one_hot_encoding:minimum_fraction, Value: 0.028777961041830738 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: polynomial preprocessor:polynomial:degree, Value: 3 preprocessor:polynomial:include_bias, Value: False preprocessor:polynomial:interaction_only, Value: False rescaling:__choice__, Value: none
You are already timing task: index_run7
[ERROR] [2016-05-21 11:57:23,175:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:57:29,391:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079 Members: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7] Weights: [ 0.02 0.32 0. 0.02 0. 0.08 0.04 0.34 0.16 0. 0. 0.02 0. ] Identifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14) [INFO] [2016-05-21 11:57:29,397:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079 [INFO] [2016-05-21 11:57:29,566:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 6.393615 seconds [INFO] [2016-05-21 11:57:29,569:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many validation set predictions (0)as ensemble predictions (14)!. [INFO] [2016-05-21 11:57:29,571:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Could not find as many test set predictions (0) as ensemble predictions (14)! [INFO] [2016-05-21 11:57:59,866:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 16. configuration. Duration 37.818645; loss 2.000000; status 2; additional run info: [INFO] [2016-05-21 11:57:59,953:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 17. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:57:59,954:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: extra_trees classifier:extra_trees:bootstrap, Value: True classifier:extra_trees:criterion, Value: gini classifier:extra_trees:max_depth, Constant: None classifier:extra_trees:max_features, Value: 2.3779010566365115 classifier:extra_trees:min_samples_leaf, Value: 2 classifier:extra_trees:min_samples_split, Value: 4 classifier:extra_trees:min_weight_fraction_leaf, Constant: 0.0 classifier:extra_trees:n_estimators, Constant: 100 imputation:strategy, Value: most_frequent one_hot_encoding:minimum_fraction, Value: 0.004305770290473698 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: select_rates preprocessor:select_rates:alpha, Value: 0.30934089491790895 preprocessor:select_rates:mode, Value: fdr preprocessor:select_rates:score_func, Value: f_classif rescaling:__choice__, Value: none
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/feature_selection/univariate_selection.py:113: UserWarning: Features [ 0 32 39 56] are constant. UserWarning)
[INFO] [2016-05-21 11:58:06,028:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 17. configuration. Duration 5.939049; loss 0.027630; status 1; additional run info: ;duration: 5.939048528671265;num_run:00017 [INFO] [2016-05-21 11:58:06,030:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 18. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:58:06,031:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: proj_logit classifier:proj_logit:max_epochs, Value: 11 imputation:strategy, Value: median one_hot_encoding:minimum_fraction, Value: 0.002883367159521145 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: gem preprocessor:gem:N, Value: 16 preprocessor:gem:precond, Value: 0.30628439346357783 rescaling:__choice__, Value: standardize [ERROR] [2016-05-21 11:58:06,964:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:58:22,168:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079 Members: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7] Weights: [ 0.02 0.32 0. 0.02 0. 0.08 0.04 0.34 0.16 0. 0. 0.02 0. 0. ] Identifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14) [INFO] [2016-05-21 11:58:22,367:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079 [INFO] [2016-05-21 11:58:22,368:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 15.600882 seconds [INFO] [2016-05-21 11:58:22,370:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change. [INFO] [2016-05-21 11:58:36,171:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 18. configuration. Duration 29.602208; loss 0.015788; status 1; additional run info: ;duration: 29.602208137512207;num_run:00018 [INFO] [2016-05-21 11:58:36,174:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 19. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:58:36,175:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: libsvm_svc classifier:libsvm_svc:C, Value: 295.4185183083245 classifier:libsvm_svc:gamma, Value: 0.05098015552849704 classifier:libsvm_svc:kernel, Value: rbf classifier:libsvm_svc:max_iter, Constant: -1 classifier:libsvm_svc:shrinking, Value: False classifier:libsvm_svc:tol, Value: 0.01616175671031427 imputation:strategy, Value: median one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: feature_agglomeration preprocessor:feature_agglomeration:affinity, Value: cosine preprocessor:feature_agglomeration:linkage, Value: complete preprocessor:feature_agglomeration:n_clusters, Value: 216 preprocessor:feature_agglomeration:pooling_func, Value: mean rescaling:__choice__, Value: none
You are already timing task: index_run8
[ERROR] [2016-05-21 11:58:36,583:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/class_weight.py:62: DeprecationWarning: The class_weight='auto' heuristic is deprecated in 0.17 in favor of a new heuristic class_weight='balanced'. 'auto' will be removed in 0.19 " 0.19", DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning)
[INFO] [2016-05-21 11:58:39,183:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 19. configuration. Duration 2.800076; loss 0.988751; status 1; additional run info: ;duration: 2.8000760078430176;num_run:00019 [INFO] [2016-05-21 11:58:39,185:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 20. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:58:39,186:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: none classifier:__choice__, Value: sgd classifier:sgd:alpha, Value: 0.00039961166337423236 classifier:sgd:average, Value: False classifier:sgd:epsilon, Value: 0.0017527970920483906 classifier:sgd:eta0, Value: 0.060041614581907 classifier:sgd:fit_intercept, Constant: True classifier:sgd:l1_ratio, Value: 0.36339240847373755 classifier:sgd:learning_rate, Value: optimal classifier:sgd:loss, Value: modified_huber classifier:sgd:n_iter, Value: 939 classifier:sgd:penalty, Value: elasticnet imputation:strategy, Value: most_frequent one_hot_encoding:use_minimum_fraction, Value: False preprocessor:__choice__, Value: select_percentile_classification preprocessor:select_percentile_classification:percentile, Value: 76.84823099284878 preprocessor:select_percentile_classification:score_func, Value: chi2 rescaling:__choice__, Value: normalize [INFO] [2016-05-21 11:58:43,769:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079 Members: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7] Weights: [ 0.02 0.32 0. 0.02 0. 0.08 0.04 0.34 0.16 0. 0. 0.02 0. 0. 0. ] Identifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14) [INFO] [2016-05-21 11:58:43,775:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079 [INFO] [2016-05-21 11:58:43,776:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 7.196891 seconds [INFO] [2016-05-21 11:58:43,778:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change.
You are already timing task: index_run8
[ERROR] [2016-05-21 11:58:45,790:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [INFO] [2016-05-21 11:58:53,191:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble Selection: Trajectory: 0: 0.990132 1: 0.992106 2: 0.994079 3: 0.994079 4: 0.994079 5: 0.994079 6: 0.994079 7: 0.994079 8: 0.994079 9: 0.994079 10: 0.994079 11: 0.994079 12: 0.994079 13: 0.994079 14: 0.994079 15: 0.994079 16: 0.994079 17: 0.994079 18: 0.994079 19: 0.994079 20: 0.994079 21: 0.994079 22: 0.994079 23: 0.994079 24: 0.994079 25: 0.994079 26: 0.994079 27: 0.994079 28: 0.994079 29: 0.994079 30: 0.994079 31: 0.994079 32: 0.994079 33: 0.994079 34: 0.994079 35: 0.994079 36: 0.994079 37: 0.994079 38: 0.994079 39: 0.994079 40: 0.994079 41: 0.994079 42: 0.994079 43: 0.994079 44: 0.994079 45: 0.994079 46: 0.994079 47: 0.994079 48: 0.994079 49: 0.994079 Members: [11, 7, 1, 8, 8, 8, 8, 8, 8, 8, 8, 7, 1, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 5, 7, 1, 7, 1, 5, 7, 5, 7, 0, 7, 1, 3, 7, 1, 7, 1, 7, 1, 6, 7, 1, 1, 7, 1, 5, 7] Weights: [ 0.02 0.32 0. 0.02 0. 0.08 0.04 0.34 0.16 0. 0. 0.02 0. 0. 0. 0. ] Identifiers: (1, 2) (1, 3) (1, 5) (1, 7) (1, 8) (1, 9) (1, 10) (1, 14) [INFO] [2016-05-21 11:58:53,196:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Training performance: 0.994079 [INFO] [2016-05-21 11:58:53,197:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Building the ensemble took 7.408631 seconds [INFO] [2016-05-21 11:58:53,198:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Ensemble output did not change. [INFO] [2016-05-21 11:59:02,578:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Finished evaluating 20. configuration. Duration 23.025394; loss 0.041445; status 1; additional run info: ;duration: 23.02539372444153;num_run:00020 [INFO] [2016-05-21 11:59:02,581:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Starting to evaluate 21. configuration (meta-learning configuration) with time limit 36s. [INFO] [2016-05-21 11:59:02,583:AutoMLSMBO(1)::6be0c6791d6c4877e53f8a237238200e] Configuration: balancing:strategy, Value: weighting classifier:__choice__, Value: sgd classifier:sgd:alpha, Value: 0.0005401543374603144 classifier:sgd:average, Value: False classifier:sgd:eta0, Value: 0.04377140004056303 classifier:sgd:fit_intercept, Constant: True classifier:sgd:learning_rate, Value: optimal classifier:sgd:loss, Value: log classifier:sgd:n_iter, Value: 943 classifier:sgd:penalty, Value: l2 imputation:strategy, Value: mean one_hot_encoding:minimum_fraction, Value: 0.0007887810786977907 one_hot_encoding:use_minimum_fraction, Value: True preprocessor:__choice__, Value: select_rates preprocessor:select_rates:alpha, Value: 0.17377268248209546 preprocessor:select_rates:mode, Value: fwe preprocessor:select_rates:score_func, Value: chi2 rescaling:__choice__, Value: min/max
You are already timing task: index_run8
[ERROR] [2016-05-21 11:59:03,389:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Model only predicts at random: /tmp/autosklearn_tmp_4909_5802/.auto-sklearn/predictions_ensemble/predictions_ensemble_1_00001.npy has score: -0.00256562068285 [ERROR] [2016-05-21 11:59:04,772:EnsembleBuilder(1):6be0c6791d6c4877e53f8a237238200e] Caught error!
いっぱいモデルが出てる。
for key in clf.models_.keys():print(clf.models_[key])
MyDummyClassifier(configuration=2, random_states=None) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'random_forest', 'classifier:random_forest:bootstrap': 'True', 'classifier:random_forest:criterion': 'gini', 'classifier:random_forest:max_depth': 'None', 'classifier:random_forest:max_features': 1.0, 'classifier:random_forest:max_leaf_nodes': 'None', 'classifier:random_forest:min_samples_leaf': 1, 'classifier:random_forest:min_samples_split': 2, 'classifier:random_forest:min_weight_fraction_leaf': 0.0, 'classifier:random_forest:n_estimators': 100, 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.01, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'no_preprocessing', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'proj_logit', 'classifier:proj_logit:max_epochs': 2, 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.0011870322904213703, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'no_preprocessing', 'rescaling:__choice__': 'none'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'adaboost', 'classifier:adaboost:algorithm': 'SAMME', 'classifier:adaboost:learning_rate': 1.0081104516473922, 'classifier:adaboost:max_depth': 6, 'classifier:adaboost:n_estimators': 468, 'imputation:strategy': 'mean', 'one_hot_encoding:use_minimum_fraction': 'False', 'preprocessor:__choice__': 'liblinear_svc_preprocessor', 'preprocessor:liblinear_svc_preprocessor:C': 1.1828431725901418, 'preprocessor:liblinear_svc_preprocessor:dual': 'False', 'preprocessor:liblinear_svc_preprocessor:fit_intercept': 'True', 'preprocessor:liblinear_svc_preprocessor:intercept_scaling': 1, 'preprocessor:liblinear_svc_preprocessor:loss': 'squared_hinge', 'preprocessor:liblinear_svc_preprocessor:multi_class': 'ovr', 'preprocessor:liblinear_svc_preprocessor:penalty': 'l1', 'preprocessor:liblinear_svc_preprocessor:tol': 0.0022792606924326923, 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'lda', 'classifier:lda:n_components': 242, 'classifier:lda:shrinkage': 'None', 'classifier:lda:tol': 0.0011339574479631279, 'imputation:strategy': 'most_frequent', 'one_hot_encoding:minimum_fraction': 0.004889144803037605, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'gem', 'preprocessor:gem:N': 8, 'preprocessor:gem:precond': 0.39344596751061517, 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'sgd', 'classifier:sgd:alpha': 0.0001, 'classifier:sgd:average': 'True', 'classifier:sgd:eta0': 0.01, 'classifier:sgd:fit_intercept': 'True', 'classifier:sgd:learning_rate': 'optimal', 'classifier:sgd:loss': 'hinge', 'classifier:sgd:n_iter': 5, 'classifier:sgd:penalty': 'l2', 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.1, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'no_preprocessing', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'libsvm_svc', 'classifier:libsvm_svc:C': 2099.097215335698, 'classifier:libsvm_svc:coef0': 0.32850098053260424, 'classifier:libsvm_svc:degree': 3, 'classifier:libsvm_svc:gamma': 2.809557625370355, 'classifier:libsvm_svc:kernel': 'poly', 'classifier:libsvm_svc:max_iter': -1, 'classifier:libsvm_svc:shrinking': 'False', 'classifier:libsvm_svc:tol': 0.003213729117706896, 'imputation:strategy': 'most_frequent', 'one_hot_encoding:minimum_fraction': 0.0033839910940432396, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'select_rates', 'preprocessor:select_rates:alpha': 0.08594367101091721, 'preprocessor:select_rates:mode': 'fpr', 'preprocessor:select_rates:score_func': 'chi2', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'extra_trees', 'classifier:extra_trees:bootstrap': 'False', 'classifier:extra_trees:criterion': 'gini', 'classifier:extra_trees:max_depth': 'None', 'classifier:extra_trees:max_features': 1.0, 'classifier:extra_trees:min_samples_leaf': 5, 'classifier:extra_trees:min_samples_split': 5, 'classifier:extra_trees:min_weight_fraction_leaf': 0.0, 'classifier:extra_trees:n_estimators': 100, 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.1, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'select_percentile_classification', 'preprocessor:select_percentile_classification:percentile': 2.0, 'preprocessor:select_percentile_classification:score_func': 'chi2', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'libsvm_svc', 'classifier:libsvm_svc:C': 2.820950683561126, 'classifier:libsvm_svc:coef0': 0.7525658406326312, 'classifier:libsvm_svc:degree': 5, 'classifier:libsvm_svc:gamma': 1.924396413436426, 'classifier:libsvm_svc:kernel': 'poly', 'classifier:libsvm_svc:max_iter': -1, 'classifier:libsvm_svc:shrinking': 'True', 'classifier:libsvm_svc:tol': 0.00037682073690554086, 'imputation:strategy': 'mean', 'one_hot_encoding:use_minimum_fraction': 'False', 'preprocessor:__choice__': 'select_percentile_classification', 'preprocessor:select_percentile_classification:percentile': 83.60106850579331, 'preprocessor:select_percentile_classification:score_func': 'f_classif', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'gaussian_nb', 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.1, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'no_preprocessing', 'rescaling:__choice__': 'standardize'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'weighting', 'classifier:__choice__': 'liblinear_svc', 'classifier:liblinear_svc:C': 0.5815868776595028, 'classifier:liblinear_svc:dual': 'False', 'classifier:liblinear_svc:fit_intercept': 'True', 'classifier:liblinear_svc:intercept_scaling': 1, 'classifier:liblinear_svc:loss': 'squared_hinge', 'classifier:liblinear_svc:multi_class': 'ovr', 'classifier:liblinear_svc:penalty': 'l2', 'classifier:liblinear_svc:tol': 0.05261321190581748, 'imputation:strategy': 'most_frequent', 'one_hot_encoding:use_minimum_fraction': 'False', 'preprocessor:__choice__': 'polynomial', 'preprocessor:polynomial:degree': 2, 'preprocessor:polynomial:include_bias': 'True', 'preprocessor:polynomial:interaction_only': 'False', 'rescaling:__choice__': 'none'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'random_forest', 'classifier:random_forest:bootstrap': 'True', 'classifier:random_forest:criterion': 'gini', 'classifier:random_forest:max_depth': 'None', 'classifier:random_forest:max_features': 1.0, 'classifier:random_forest:max_leaf_nodes': 'None', 'classifier:random_forest:min_samples_leaf': 1, 'classifier:random_forest:min_samples_split': 2, 'classifier:random_forest:min_weight_fraction_leaf': 0.0, 'classifier:random_forest:n_estimators': 100, 'imputation:strategy': 'mean', 'one_hot_encoding:minimum_fraction': 0.01, 'one_hot_encoding:use_minimum_fraction': 'True', 'preprocessor:__choice__': 'no_preprocessing', 'rescaling:__choice__': 'min/max'}) SimpleClassificationPipeline(configuration={ 'balancing:strategy': 'none', 'classifier:__choice__': 'adaboost', 'classifier:adaboost:algorithm': 'SAMME', 'classifier:adaboost:learning_rate': 1.2306208006800998, 'classifier:adaboost:max_depth': 6, 'classifier:adaboost:n_estimators': 499, 'imputation:strategy': 'median', 'one_hot_encoding:use_minimum_fraction': 'False', 'preprocessor:__choice__': 'extra_trees_preproc_for_classification', 'preprocessor:extra_trees_preproc_for_classification:bootstrap': 'True', 'preprocessor:extra_trees_preproc_for_classification:criterion': 'entropy', 'preprocessor:extra_trees_preproc_for_classification:max_depth': 'None', 'preprocessor:extra_trees_preproc_for_classification:max_features': 3.5347851525007146, 'preprocessor:extra_trees_preproc_for_classification:min_samples_leaf': 6, 'preprocessor:extra_trees_preproc_for_classification:min_samples_split': 8, 'preprocessor:extra_trees_preproc_for_classification:min_weight_fraction_leaf': 0.0, 'preprocessor:extra_trees_preproc_for_classification:n_estimators': 100, 'rescaling:__choice__': 'standardize'})
多分モデルのアンサンブルをとっている(はず)。
pred=clf.predict(data_test)
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning)
いつもは97.5%くらいだけど今回はたまたまよい?
正答率もまあまあ。
clf.score(data_test,target_test)
/home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/utils/__init__.py:93: DeprecationWarning: Function transform is deprecated; Support to use estimators as feature selectors will be removed in version 0.19. Use SelectFromModel instead. warnings.warn(msg, category=DeprecationWarning) /home/ubuntu/workspace/.pyenv/versions/miniconda3-4.0.5/envs/automl/lib/python3.5/site-packages/sklearn/svm/base.py:547: ChangedBehaviorWarning: The decision_function_shape default value will change from 'ovo' to 'ovr' in 0.18. This will change the shape of the decision function returned by SVC. "SVC.", ChangedBehaviorWarning)
1.0