이 노트북에서 양방향 LSTM을 만들어 감성에 따라 IMDB 영화 리뷰를 분류합니다.
from tensorflow import keras
from tensorflow.keras.datasets import imdb
from tensorflow.keras.preprocessing.sequence import pad_sequences
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Dropout, Embedding, SpatialDropout1D, LSTM
from tensorflow.keras.layers import Bidirectional # new!
from tensorflow.keras.callbacks import ModelCheckpoint
import os
from sklearn.metrics import roc_auc_score
import matplotlib.pyplot as plt
%matplotlib inline
# 출력 디렉토리
output_dir = 'model_output/biLSTM'
# 훈련
epochs = 6
batch_size = 128
# 벡터 공간 임베딩
n_dim = 64
n_unique_words = 10000
max_review_length = 200 # 두베!
pad_type = trunc_type = 'pre'
drop_embed = 0.2
# LSTM 층 구조
n_lstm = 256
drop_lstm = 0.2
(x_train, y_train), (x_valid, y_valid) = imdb.load_data(num_words=n_unique_words) # n_words_to_skip 삭제
Downloading data from https://storage.googleapis.com/tensorflow/tf-keras-datasets/imdb.npz 17464789/17464789 [==============================] - 2s 0us/step
x_train = pad_sequences(x_train, maxlen=max_review_length, padding=pad_type, truncating=trunc_type, value=0)
x_valid = pad_sequences(x_valid, maxlen=max_review_length, padding=pad_type, truncating=trunc_type, value=0)
model = Sequential()
model.add(Embedding(n_unique_words, n_dim, input_length=max_review_length))
model.add(SpatialDropout1D(drop_embed))
model.add(Bidirectional(LSTM(n_lstm, dropout=drop_lstm)))
model.add(Dense(1, activation='sigmoid'))
# 양 방향으로 가중치가 있기 때문에 LSTM 층 파라미터가 두 배가 됩니다.
model.summary()
Model: "sequential" _________________________________________________________________ Layer (type) Output Shape Param # ================================================================= embedding (Embedding) (None, 200, 64) 640000 spatial_dropout1d (SpatialD (None, 200, 64) 0 ropout1D) bidirectional (Bidirectiona (None, 512) 657408 l) dense (Dense) (None, 1) 513 ================================================================= Total params: 1,297,921 Trainable params: 1,297,921 Non-trainable params: 0 _________________________________________________________________
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])
modelcheckpoint = ModelCheckpoint(filepath=output_dir+"/weights.{epoch:02d}.hdf5")
if not os.path.exists(output_dir):
os.makedirs(output_dir)
# 데이터셋이 작기 때문에 긴 문장에 걸쳐 단어의 복잡한 상호작용이 잘 학습되지 않습니다.
# CNN은 리뷰의 감성을 예측하는 위치에 상관없는 2개에서 4개까지 단어 조각을 선택합니다.
# 이 작업이 더 간단하기 때문에 데이터에서 학습하기 쉽습니다.
# 따라서 CNN이 IMDB 데이터셋에서 성능이 더 좋습니다.
model.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose=1, validation_data=(x_valid, y_valid), callbacks=[modelcheckpoint])
Epoch 1/6 196/196 [==============================] - 22s 71ms/step - loss: 0.5663 - accuracy: 0.7075 - val_loss: 0.3506 - val_accuracy: 0.8566 Epoch 2/6 196/196 [==============================] - 13s 68ms/step - loss: 0.2967 - accuracy: 0.8795 - val_loss: 0.3144 - val_accuracy: 0.8679 Epoch 3/6 196/196 [==============================] - 14s 70ms/step - loss: 0.2261 - accuracy: 0.9156 - val_loss: 0.3119 - val_accuracy: 0.8745 Epoch 4/6 196/196 [==============================] - 14s 72ms/step - loss: 0.1781 - accuracy: 0.9325 - val_loss: 0.3227 - val_accuracy: 0.8624 Epoch 5/6 196/196 [==============================] - 14s 72ms/step - loss: 0.1487 - accuracy: 0.9449 - val_loss: 0.3631 - val_accuracy: 0.8681 Epoch 6/6 196/196 [==============================] - 14s 72ms/step - loss: 0.1243 - accuracy: 0.9556 - val_loss: 0.3986 - val_accuracy: 0.8627
<keras.callbacks.History at 0x7fba23b71040>
model.load_weights(output_dir+"/weights.04.hdf5")
y_hat = model.predict(x_valid)
782/782 [==============================] - 8s 9ms/step
plt.hist(y_hat)
_ = plt.axvline(x=0.5, color='orange')
"{:0.2f}".format(roc_auc_score(y_valid, y_hat)*100.0)
'93.74'