Recurrent Neural Network (LSTM)
from keras.models import Sequential
from keras.layers import LSTM, Dense
from keras.datasets import mnist
from keras.utils import np_utils
from keras import initializations
def init_weights(shape, name=None):
return initializations.normal(shape, scale=0.01, name=name)
Using TensorFlow backend.
batch_size = 128
nb_epoch = 50
img_rows, img_cols = 28, 28
nb_classes = 10
nb_lstm_outputs = 30
nb_time_steps = img_rows
dim_input_vector = img_cols
(X_train, y_train), (X_test, y_test) = mnist.load_data()
print('X_train original shape:', X_train.shape)
input_shape = (nb_time_steps, dim_input_vector)
X_train = X_train.astype('float32') / 255.
X_test = X_test.astype('float32') / 255.
Y_train = np_utils.to_categorical(y_train, nb_classes)
Y_test = np_utils.to_categorical(y_test, nb_classes)
print('X_train shape:', X_train.shape)
print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')
X_train original shape: (60000, 28, 28)
X_train shape: (60000, 28, 28)
60000 train samples
10000 test samples
model = Sequential()
model.add(LSTM(nb_lstm_outputs, input_shape=input_shape))
model.add(Dense(nb_classes, activation='softmax', init=init_weights))
model.compile(optimizer='rmsprop', loss='categorical_crossentropy', metrics=['accuracy'])
model.summary()
____________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
====================================================================================================
lstm_1 (LSTM) (None, 30) 7080 lstm_input_1[0][0]
____________________________________________________________________________________________________
dense_1 (Dense) (None, 10) 310 lstm_1[0][0]
====================================================================================================
Total params: 7390
____________________________________________________________________________________________________
history = model.fit(X_train, Y_train, nb_epoch=nb_epoch, batch_size=batch_size, shuffle=True, verbose=1)
Epoch 1/50
60000/60000 [==============================] - 21s - loss: 1.2954 - acc: 0.6153
Epoch 2/50
60000/60000 [==============================] - 21s - loss: 0.5704 - acc: 0.8313
Epoch 3/50
60000/60000 [==============================] - 21s - loss: 0.3784 - acc: 0.8897
Epoch 4/50
60000/60000 [==============================] - 21s - loss: 0.2879 - acc: 0.9173
Epoch 5/50
60000/60000 [==============================] - 21s - loss: 0.2331 - acc: 0.9320
Epoch 6/50
60000/60000 [==============================] - 21s - loss: 0.1977 - acc: 0.9431
Epoch 7/50
60000/60000 [==============================] - 21s - loss: 0.1754 - acc: 0.9494
Epoch 8/50
60000/60000 [==============================] - 21s - loss: 0.1555 - acc: 0.9540
Epoch 9/50
60000/60000 [==============================] - 21s - loss: 0.1416 - acc: 0.9583
Epoch 10/50
60000/60000 [==============================] - 21s - loss: 0.1285 - acc: 0.9623
Epoch 11/50
60000/60000 [==============================] - 21s - loss: 0.1206 - acc: 0.9649
Epoch 12/50
60000/60000 [==============================] - 21s - loss: 0.1096 - acc: 0.9673
Epoch 13/50
60000/60000 [==============================] - 21s - loss: 0.1028 - acc: 0.9697
Epoch 14/50
60000/60000 [==============================] - 21s - loss: 0.0975 - acc: 0.9714
Epoch 15/50
60000/60000 [==============================] - 21s - loss: 0.0911 - acc: 0.9734
Epoch 16/50
60000/60000 [==============================] - 21s - loss: 0.0869 - acc: 0.9745
Epoch 17/50
60000/60000 [==============================] - 21s - loss: 0.0827 - acc: 0.9764
Epoch 18/50
60000/60000 [==============================] - 21s - loss: 0.0767 - acc: 0.9776
Epoch 19/50
60000/60000 [==============================] - 21s - loss: 0.0742 - acc: 0.9781
Epoch 20/50
60000/60000 [==============================] - 21s - loss: 0.0700 - acc: 0.9789
Epoch 21/50
60000/60000 [==============================] - 21s - loss: 0.0679 - acc: 0.9797
Epoch 22/50
60000/60000 [==============================] - 21s - loss: 0.0643 - acc: 0.9808
Epoch 23/50
60000/60000 [==============================] - 21s - loss: 0.0613 - acc: 0.9818
Epoch 24/50
60000/60000 [==============================] - 21s - loss: 0.0599 - acc: 0.9820
Epoch 25/50
60000/60000 [==============================] - 21s - loss: 0.0571 - acc: 0.9831
Epoch 26/50
60000/60000 [==============================] - 21s - loss: 0.0553 - acc: 0.9833
Epoch 27/50
60000/60000 [==============================] - 21s - loss: 0.0535 - acc: 0.9842
Epoch 28/50
60000/60000 [==============================] - 21s - loss: 0.0516 - acc: 0.9842
Epoch 29/50
60000/60000 [==============================] - 21s - loss: 0.0500 - acc: 0.9849
Epoch 30/50
60000/60000 [==============================] - 21s - loss: 0.0485 - acc: 0.9852
Epoch 31/50
60000/60000 [==============================] - 21s - loss: 0.0462 - acc: 0.9861
Epoch 32/50
60000/60000 [==============================] - 21s - loss: 0.0456 - acc: 0.9862
Epoch 33/50
60000/60000 [==============================] - 21s - loss: 0.0439 - acc: 0.9869
Epoch 34/50
60000/60000 [==============================] - 21s - loss: 0.0428 - acc: 0.9873
Epoch 35/50
60000/60000 [==============================] - 21s - loss: 0.0415 - acc: 0.9874
Epoch 36/50
60000/60000 [==============================] - 21s - loss: 0.0405 - acc: 0.9881
Epoch 37/50
60000/60000 [==============================] - 21s - loss: 0.0389 - acc: 0.9884
Epoch 38/50
60000/60000 [==============================] - 21s - loss: 0.0383 - acc: 0.9886
Epoch 39/50
60000/60000 [==============================] - 21s - loss: 0.0371 - acc: 0.9894
Epoch 40/50
60000/60000 [==============================] - 21s - loss: 0.0358 - acc: 0.9892
Epoch 41/50
60000/60000 [==============================] - 21s - loss: 0.0350 - acc: 0.9898
Epoch 42/50
60000/60000 [==============================] - 21s - loss: 0.0347 - acc: 0.9894
Epoch 43/50
60000/60000 [==============================] - 21s - loss: 0.0340 - acc: 0.9897
Epoch 44/50
60000/60000 [==============================] - 21s - loss: 0.0323 - acc: 0.9901
Epoch 45/50
60000/60000 [==============================] - 21s - loss: 0.0321 - acc: 0.9901
Epoch 46/50
60000/60000 [==============================] - 21s - loss: 0.0307 - acc: 0.9908
Epoch 47/50
60000/60000 [==============================] - 21s - loss: 0.0307 - acc: 0.9906
Epoch 48/50
60000/60000 [==============================] - 21s - loss: 0.0288 - acc: 0.9912
Epoch 49/50
60000/60000 [==============================] - 21s - loss: 0.0282 - acc: 0.9915
Epoch 50/50
60000/60000 [==============================] - 21s - loss: 0.0285 - acc: 0.9910
evaluation = model.evaluate(X_test, Y_test, batch_size=batch_size, verbose=1)
print('Summary: Loss over the test dataset: %.2f, Accuracy: %.2f' % (evaluation[0], evaluation[1]))
10000/10000 [==============================] - 0s
Summary: Loss over the test dataset: 0.07, Accuracy: 0.98