Non Linear Regression Example with Keras and Tensorflow Backend

New! Google Colab version of this code is available in this link. No need to install any software to run code. Download these files (training.csv and validation.csv) and upload when Colab requests for it.

Github Repository of Colab Code is available in this link.

For post on Keras Nonlinear Regression – Guass3 function click on this link

_____________________________________________________

This post is about using Keras to do non linear fitting. The simplicity of Keras made it possible to quickly try out some neural network model without deep knowledge of Tensorflow.

The data for fitting was generated using a non linear continuous function. It has five inputs and one output. Both the training set and validation set have around 500 data points.

Y = SIN(A) x EXP(B) + COS(C x C) + POWER(D,5) – TANH(E)

I realized that adding too many hidden layers worsened the fit. Looks like for continuous functions, one hidden layer with sufficient number of nodes and good choice of activation function is sufficient. I chose hyperbolic tangent (tanh) for activation function and adam for optimizer. The results were pretty good but required some good number of iterations.

I plan to compare this with other regression algorithms available in Azure Machine Learning.

Complete code available on Github – https://github.com/shankarananth/Keras-Nonlinear-Regression

[pastacode lang=”python” manual=”from%20keras.models%20import%20Sequential%0Afrom%20keras.layers%20import%20Dense%0Afrom%20sklearn.metrics%20import%20r2_score%0Aimport%20matplotlib.pyplot%20as%20plt%0Aimport%20numpy%0A%25matplotlib%20inline%0A%0A%23Red%20data%20from%20csv%20file%20for%20training%20and%20validation%20data%0ATrainingSet%20%3D%20numpy.genfromtxt(%22training.csv%22%2C%20delimiter%3D%22%2C%22%2C%20skip_header%3DTrue)%0AValidationSet%20%3D%20numpy.genfromtxt(%22validation.csv%22%2C%20delimiter%3D%22%2C%22%2C%20skip_header%3DTrue)%0A%0A%23%20split%20into%20input%20(X)%20and%20output%20(Y)%20variables%0AX1%20%3D%20TrainingSet%5B%3A%2C0%3A5%5D%0AY1%20%3D%20TrainingSet%5B%3A%2C5%5D%0A%0AX2%20%3D%20ValidationSet%5B%3A%2C0%3A5%5D%0AY2%20%3D%20ValidationSet%5B%3A%2C5%5D%0A%0A%23%20create%20model%0Amodel%20%3D%20Sequential()%0Amodel.add(Dense(20%2C%20activation%3D%22tanh%22%2C%20input_dim%3D5%2C%20kernel_initializer%3D%22uniform%22))%0Amodel.add(Dense(1%2C%20activation%3D%22linear%22%2C%20kernel_initializer%3D%22uniform%22))%0A%0A%23%20Compile%20model%0Amodel.compile(loss%3D’mse’%2C%20optimizer%3D’adam’%2C%20metrics%3D%5B’accuracy’%5D)%0A%0A%23%20Fit%20the%20model%0Amodel.fit(X1%2C%20Y1%2C%20epochs%3D100%2C%20batch_size%3D10%2C%20%20verbose%3D2)%0A%0A%23%20Calculate%20predictions%0APredTestSet%20%3D%20model.predict(X1)%0APredValSet%20%3D%20model.predict(X2)%0A%0A%23%20Save%20predictions%0Anumpy.savetxt(%22trainresults.csv%22%2C%20PredTestSet%2C%20delimiter%3D%22%2C%22)%0Anumpy.savetxt(%22valresults.csv%22%2C%20PredValSet%2C%20delimiter%3D%22%2C%22)%0A%0A%23Plot%20actual%20vs%20predition%20for%20training%20set%0ATestResults%20%3D%20numpy.genfromtxt(%22trainresults.csv%22%2C%20delimiter%3D%22%2C%22)%0Aplt.plot(Y1%2CTestResults%2C’ro’)%0Aplt.title(‘Training%20Set’)%0Aplt.xlabel(‘Actual’)%0Aplt.ylabel(‘Predicted’)%0A%0A%23Compute%20R-Square%20value%20for%20training%20set%0ATestR2Value%20%3D%20r2_score(Y1%2CTestResults)%0Aprint(%22Training%20Set%20R-Square%3D%22%2C%20TestR2Value)%0A%0A%23Plot%20actual%20vs%20predition%20for%20validation%20set%0AValResults%20%3D%20numpy.genfromtxt(%22valresults.csv%22%2C%20delimiter%3D%22%2C%22)%0Aplt.plot(Y2%2CValResults%2C’ro’)%0Aplt.title(‘Validation%20Set’)%0Aplt.xlabel(‘Actual’)%0Aplt.ylabel(‘Predicted’)%0A%0A%23Compute%20R-Square%20value%20for%20validation%20set%0AValR2Value%20%3D%20r2_score(Y2%2CValResults)%0Aprint(%22Validation%20Set%20R-Square%3D%22%2CValR2Value)” message=”Python Code” highlight=”” provider=”manual”/]

 

 

The results were pretty good

Keras

Keras


Comments

7 responses to “Non Linear Regression Example with Keras and Tensorflow Backend”

  1. Jaydeep Maddipati

    Can you use R^2 incase of non linear regression?Isn’t that not advised?

    1. Hi Jaydeep,
      The R^2 value is for a fit between Predicted value and actual. It is only an indication for the quality of fit. It is not the R^2 for the Non Linear fit itself as R^2 is impossible for Non linear regression.

      http://blog.minitab.com/blog/adventures-in-statistics-2/why-is-there-no-r-squared-for-nonlinear-regression

      – Shankar

  2. niklas wehkamp

    this looks pretty linear… why do you call it Non Linear Regression?
    (Not sure what you are plotting… labels would be nice)

    1. Hi Niklas,
      The function is itself Non Linear. The fit is indeed confusing (Predicted Vs Actual). I have labelled them now. Thanks for your feedback.

      — Shankar

    2. I’ll also include some standard non linear functions (Guass3, Fourier Transform..) as the level is non linear is low in the above example.

      – Shankar

  3. I think there’s a bug in your code. Your inputs are actually:
    X1 = TrainingSet[:,0:4]
    not
    X1 = TrainingSet[:,0:5]
    Since you’re including the outputs in the fit of the outputs, I think it’s just setting the weights on the inputs to zero, causing artificially high r^2 values.

    1. Hi Adam,
      Thanks for your comments. I checked the data, fit parameters and it is correct. The r^2 value looks perfect as too many iterations are executed. If the iterations are limited to 100 for example the r^2 value is 0.87.

      The r^2 is not for the fit itself. It is for the fit between the actual data and predicted, and indirect indication of accuracy of fit.

      – Shankar

Leave a Reply

Your email address will not be published. Required fields are marked *


The reCAPTCHA verification period has expired. Please reload the page.

This site uses Akismet to reduce spam. Learn how your comment data is processed.