Linear Regression

from keras.models import Sequential
from keras.layers import Dense
import numpy as np
import matplotlib.pyplot as plt
%matplotlib inline

Using TensorFlow backend.

# Generate dataset
trX = np.linspace(-1, 1, 101)
trY = 2 * trX + np.random.randn(*trX.shape) * 0.33 # create a y value which is approximately linear but with some random noise

# Linear regression model
model = Sequential()
model.add(Dense(input_dim=1, output_dim=1, init='uniform', activation='linear'))
model.compile(optimizer='sgd', loss='mse')

# Print initial weights
weights = model.layers[0].get_weights()
w_init = weights[0][0][0]
b_init = weights[1][0]
print('Linear regression model is initialized with weight w: %.2f, b: %.2f' % (w_init, b_init))

Linear regression model is initialized with weight w: -0.02, b: 0.00

# Train
model.fit(trX, trY, nb_epoch=100, verbose=1)

Epoch 1/100
101/101 [==============================] - 0s - loss: 1.6034     
Epoch 2/100
101/101 [==============================] - 0s - loss: 1.5246     
Epoch 3/100
101/101 [==============================] - 0s - loss: 1.4446     
Epoch 4/100
101/101 [==============================] - 0s - loss: 1.3690     
Epoch 5/100
101/101 [==============================] - 0s - loss: 1.2975     
Epoch 6/100
101/101 [==============================] - 0s - loss: 1.2362     
Epoch 7/100
101/101 [==============================] - 0s - loss: 1.1691     
Epoch 8/100
101/101 [==============================] - 0s - loss: 1.1179     
Epoch 9/100
101/101 [==============================] - 0s - loss: 1.0717     
Epoch 10/100
101/101 [==============================] - 0s - loss: 1.0224     
Epoch 11/100
101/101 [==============================] - 0s - loss: 0.9743     
Epoch 12/100
101/101 [==============================] - 0s - loss: 0.9290     
Epoch 13/100
101/101 [==============================] - 0s - loss: 0.8886     
Epoch 14/100
101/101 [==============================] - 0s - loss: 0.8452     
Epoch 15/100
101/101 [==============================] - 0s - loss: 0.8061     
Epoch 16/100
101/101 [==============================] - 0s - loss: 0.7721     
Epoch 17/100
101/101 [==============================] - 0s - loss: 0.7335     
Epoch 18/100
101/101 [==============================] - 0s - loss: 0.6992     
Epoch 19/100
101/101 [==============================] - 0s - loss: 0.6663     
Epoch 20/100
101/101 [==============================] - 0s - loss: 0.6388     
Epoch 21/100
101/101 [==============================] - 0s - loss: 0.6106     
Epoch 22/100
101/101 [==============================] - 0s - loss: 0.5842     
Epoch 23/100
101/101 [==============================] - 0s - loss: 0.5584     
Epoch 24/100
101/101 [==============================] - 0s - loss: 0.5357     
Epoch 25/100
101/101 [==============================] - 0s - loss: 0.5126     
Epoch 26/100
101/101 [==============================] - 0s - loss: 0.4931     
Epoch 27/100
101/101 [==============================] - 0s - loss: 0.4715     
Epoch 28/100
101/101 [==============================] - 0s - loss: 0.4544     
Epoch 29/100
101/101 [==============================] - 0s - loss: 0.4364     
Epoch 30/100
101/101 [==============================] - 0s - loss: 0.4194     
Epoch 31/100
101/101 [==============================] - 0s - loss: 0.4032     
Epoch 32/100
101/101 [==============================] - 0s - loss: 0.3883     
Epoch 33/100
101/101 [==============================] - 0s - loss: 0.3703     
Epoch 34/100
101/101 [==============================] - 0s - loss: 0.3565     
Epoch 35/100
101/101 [==============================] - 0s - loss: 0.3433     
Epoch 36/100
101/101 [==============================] - 0s - loss: 0.3312     
Epoch 37/100
101/101 [==============================] - 0s - loss: 0.3179     
Epoch 38/100
101/101 [==============================] - 0s - loss: 0.3067     
Epoch 39/100
101/101 [==============================] - 0s - loss: 0.2956     
Epoch 40/100
101/101 [==============================] - 0s - loss: 0.2856     
Epoch 41/100
101/101 [==============================] - 0s - loss: 0.2767     
Epoch 42/100
101/101 [==============================] - 0s - loss: 0.2660     
Epoch 43/100
101/101 [==============================] - 0s - loss: 0.2579     
Epoch 44/100
101/101 [==============================] - 0s - loss: 0.2498     
Epoch 45/100
101/101 [==============================] - 0s - loss: 0.2428     
Epoch 46/100
101/101 [==============================] - 0s - loss: 0.2355     
Epoch 47/100
101/101 [==============================] - 0s - loss: 0.2281     
Epoch 48/100
101/101 [==============================] - 0s - loss: 0.2222     
Epoch 49/100
101/101 [==============================] - 0s - loss: 0.2158     
Epoch 50/100
101/101 [==============================] - 0s - loss: 0.2101     
Epoch 51/100
101/101 [==============================] - 0s - loss: 0.2054     
Epoch 52/100
101/101 [==============================] - 0s - loss: 0.2013     
Epoch 53/100
101/101 [==============================] - 0s - loss: 0.1963     
Epoch 54/100
101/101 [==============================] - 0s - loss: 0.1918     
Epoch 55/100
101/101 [==============================] - 0s - loss: 0.1870     
Epoch 56/100
101/101 [==============================] - 0s - loss: 0.1836     
Epoch 57/100
101/101 [==============================] - 0s - loss: 0.1800     
Epoch 58/100
101/101 [==============================] - 0s - loss: 0.1760     
Epoch 59/100
101/101 [==============================] - 0s - loss: 0.1721     
Epoch 60/100
101/101 [==============================] - 0s - loss: 0.1691     
Epoch 61/100
101/101 [==============================] - 0s - loss: 0.1664     
Epoch 62/100
101/101 [==============================] - 0s - loss: 0.1631     
Epoch 63/100
101/101 [==============================] - 0s - loss: 0.1602     
Epoch 64/100
101/101 [==============================] - 0s - loss: 0.1570     
Epoch 65/100
101/101 [==============================] - 0s - loss: 0.1542     
Epoch 66/100
101/101 [==============================] - 0s - loss: 0.1519     
Epoch 67/100
101/101 [==============================] - 0s - loss: 0.1496     
Epoch 68/100
101/101 [==============================] - 0s - loss: 0.1470     
Epoch 69/100
101/101 [==============================] - 0s - loss: 0.1446     
Epoch 70/100
101/101 [==============================] - 0s - loss: 0.1430     
Epoch 71/100
101/101 [==============================] - 0s - loss: 0.1414     
Epoch 72/100
101/101 [==============================] - 0s - loss: 0.1397     
Epoch 73/100
101/101 [==============================] - 0s - loss: 0.1381     
Epoch 74/100
101/101 [==============================] - 0s - loss: 0.1364     
Epoch 75/100
101/101 [==============================] - 0s - loss: 0.1352     
Epoch 76/100
101/101 [==============================] - 0s - loss: 0.1338     
Epoch 77/100
101/101 [==============================] - 0s - loss: 0.1321     
Epoch 78/100
101/101 [==============================] - 0s - loss: 0.1312     
Epoch 79/100
101/101 [==============================] - 0s - loss: 0.1299     
Epoch 80/100
101/101 [==============================] - 0s - loss: 0.1286     
Epoch 81/100
101/101 [==============================] - 0s - loss: 0.1270     
Epoch 82/100
101/101 [==============================] - 0s - loss: 0.1260     
Epoch 83/100
101/101 [==============================] - 0s - loss: 0.1252     
Epoch 84/100
101/101 [==============================] - 0s - loss: 0.1247     
Epoch 85/100
101/101 [==============================] - 0s - loss: 0.1239     
Epoch 86/100
101/101 [==============================] - 0s - loss: 0.1228     
Epoch 87/100
101/101 [==============================] - 0s - loss: 0.1222     
Epoch 88/100
101/101 [==============================] - 0s - loss: 0.1214     
Epoch 89/100
101/101 [==============================] - 0s - loss: 0.1210     
Epoch 90/100
101/101 [==============================] - 0s - loss: 0.1201     
Epoch 91/100
101/101 [==============================] - 0s - loss: 0.1194     
Epoch 92/100
101/101 [==============================] - 0s - loss: 0.1189     
Epoch 93/100
101/101 [==============================] - 0s - loss: 0.1184     
Epoch 94/100
101/101 [==============================] - 0s - loss: 0.1176     
Epoch 95/100
101/101 [==============================] - 0s - loss: 0.1168     
Epoch 96/100
101/101 [==============================] - 0s - loss: 0.1162     
Epoch 97/100
101/101 [==============================] - 0s - loss: 0.1157     
Epoch 98/100
101/101 [==============================] - 0s - loss: 0.1150     
Epoch 99/100
101/101 [==============================] - 0s - loss: 0.1144     
Epoch 100/100
101/101 [==============================] - 0s - loss: 0.1140     





<keras.callbacks.History at 0x7f9644059750>

# Print trained weights
weights = model.layers[0].get_weights()
w = weights[0][0][0]
b = weights[1][0]
print('Linear regression model is trained with weight w: %.2f, b: %.2f' % (w, b))

Linear regression model is trained with weight w: 1.95, b: -0.01

plt.plot(trX, trY, label='data')
plt.plot(trX, w_init*trX + b_init, label='init')
plt.plot(trX, w*trX + b, label='prediction')
plt.legend()
plt.show()