이 노트북에서 적층 양방향 LSTM을 사용해 감성에 따라 IMDB 영화 리뷰를 분류합니다.
from tensorflow import keras
from tensorflow.keras.datasets import imdb
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Embedding, SpatialDropout1D, LSTM
from tensorflow.keras.layers import Bidirectional
from tensorflow.keras.callbacks import ModelCheckpoint
import os
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt
%matplotlib inline
# 출력 디렉토리
output_dir = 'model_output/stackedLSTM'
# 훈련
epochs = 4
batch_size = 128
# 벡터 공간 임베딩
n_dim = 64
n_unique_words = 10000
max_review_length = 200
pad_type = trunc_type = 'pre'
drop_embed = 0.2
# LSTM 층 구조
n_lstm_1 = 64 # 줄임
n_lstm_2 = 64 # new!
drop_lstm = 0.2
(x_train, y_train), (x_valid, y_valid) = imdb.load_data(num_words=n_unique_words) # n_words_to_skip 삭제
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz 17464789/17464789 [==============================] - 0s 0us/step
x_train = pad_sequences(x_train, maxlen=max_review_length, padding=pad_type, truncating=trunc_type, value=0)
x_valid = pad_sequences(x_valid, maxlen=max_review_length, padding=pad_type, truncating=trunc_type, value=0)
model = Sequential()
model.add(Embedding(n_unique_words, n_dim, input_length=max_review_length))
model.add(SpatialDropout1D(drop_embed))
model.add(Bidirectional(LSTM(n_lstm_1, dropout=drop_lstm,
return_sequences=True)))
model.add(Bidirectional(LSTM(n_lstm_2, dropout=drop_lstm)))
model.add(Dense(1, activation='sigmoid'))
# 양 방향으로 역전파되기 때문에 LSTM 층의 파라미터가 두 배가 됩니다.
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= embedding (Embedding) (None, 200, 64) 640000 spatial_dropout1d (SpatialD (None, 200, 64) 0 ropout1D) bidirectional (Bidirectiona (None, 200, 128) 66048 l) bidirectional_1 (Bidirectio (None, 128) 98816 nal) dense (Dense) (None, 1) 129 ================================================================= Total params: 804,993 Trainable params: 804,993 Non-trainable params: 0 _________________________________________________________________
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
modelcheckpoint = ModelCheckpoint(filepath=output_dir+"/weights.{epoch:02d}.hdf5")
if not os.path.exists(output_dir):
os.makedirs(output_dir)
model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_valid, y_valid), callbacks=[modelcheckpoint])
Epoch 1/4 196/196 [==============================] - 23s 65ms/step - loss: 0.4572 - accuracy: 0.7633 - val_loss: 0.3050 - val_accuracy: 0.8723 Epoch 2/4 196/196 [==============================] - 12s 59ms/step - loss: 0.2476 - accuracy: 0.9032 - val_loss: 0.3475 - val_accuracy: 0.8474 Epoch 3/4 196/196 [==============================] - 12s 60ms/step - loss: 0.1906 - accuracy: 0.9277 - val_loss: 0.3224 - val_accuracy: 0.8672 Epoch 4/4 196/196 [==============================] - 12s 59ms/step - loss: 0.1422 - accuracy: 0.9490 - val_loss: 0.3546 - val_accuracy: 0.8650
<keras.callbacks.History at 0x7f02e94c6f10>
model.load_weights(output_dir+"/weights.02.hdf5")
y_hat = model.predict(x_valid)
782/782 [==============================] - 11s 13ms/step
plt.hist(y_hat)
_ = plt.axvline(x=0.5, color='orange')
"{:0.2f}".format(roc_auc_score(y_valid, y_hat)*100.0)
'94.00'