Forked from zhaleh-rahimi/2.1Prediction1Dregression_v3.ipynb
Created
August 17, 2022 16:02
-
-
Save mohayl/3e94dc9f0dcf0109eeedeecdf2c8d9fc to your computer and use it in GitHub Desktop.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| {"cells":[{"cell_type":"markdown","metadata":{},"source":["<center>\n"," <img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/IDSNlogo.png\" width=\"300\" alt=\"cognitiveclass.ai logo\" />\n","</center>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h1>Linear regression 1D: Training Two Parameter</h1>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2>Objective</h2><ul><li> How to train the model and visualize the loss results.</li></ul> \n"]},{"cell_type":"markdown","metadata":{"slideshow":{"slide_type":"slide"}},"source":["<h2>Table of Contents</h2>\n","<p>In this lab, you will train a model with PyTorch by using the data that we created. The model will have the slope and bias. And we will review how to make a prediction in several different ways by using PyTorch.</p>\n","\n","<ul>\n"," <li><a href=\"#Makeup_Data\">Make Some Data</a></li>\n"," <li><a href=\"#Model_Cost\">Create the Model and Cost Function (Total Loss) </a></li>\n"," <li><a href=\"#Train\">Train the Model </a></li>\n","</ul>\n","<p>Estimated Time Needed: <strong>20 min</strong></ul>\n","\n","<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2>Preparation</h2>\n"]},{"cell_type":"markdown","metadata":{"slideshow":{"slide_type":"slide"}},"source":["We'll need the following libraries: \n"]},{"cell_type":"code","execution_count":null,"metadata":{"slideshow":{"slide_type":"slide"}},"outputs":[],"source":["# These are the libraries we are going to use in the lab.\n","\n","import numpy as np\n","import matplotlib.pyplot as plt\n","from mpl_toolkits import mplot3d"]},{"cell_type":"markdown","metadata":{},"source":["The class <code>plot_error_surfaces</code> is just to help you visualize the data space and the parameter space during training and has nothing to do with PyTorch. \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# The class for plot the diagram\n","\n","class plot_error_surfaces(object):\n"," \n"," # Constructor\n"," def __init__(self, w_range, b_range, X, Y, n_samples = 30, go = True):\n"," W = np.linspace(-w_range, w_range, n_samples)\n"," B = np.linspace(-b_range, b_range, n_samples)\n"," w, b = np.meshgrid(W, B) \n"," Z = np.zeros((30,30))\n"," count1 = 0\n"," self.y = Y.numpy()\n"," self.x = X.numpy()\n"," for w1, b1 in zip(w, b):\n"," count2 = 0\n"," for w2, b2 in zip(w1, b1):\n"," Z[count1, count2] = np.mean((self.y - w2 * self.x + b2) ** 2)\n"," count2 += 1\n"," count1 += 1\n"," self.Z = Z\n"," self.w = w\n"," self.b = b\n"," self.W = []\n"," self.B = []\n"," self.LOSS = []\n"," self.n = 0\n"," if go == True:\n"," plt.figure()\n"," plt.figure(figsize = (7.5, 5))\n"," plt.axes(projection='3d').plot_surface(self.w, self.b, self.Z, rstride = 1, cstride = 1,cmap = 'viridis', edgecolor = 'none')\n"," plt.title('Cost/Total Loss Surface')\n"," plt.xlabel('w')\n"," plt.ylabel('b')\n"," plt.show()\n"," plt.figure()\n"," plt.title('Cost/Total Loss Surface Contour')\n"," plt.xlabel('w')\n"," plt.ylabel('b')\n"," plt.contour(self.w, self.b, self.Z)\n"," plt.show()\n"," \n"," # Setter\n"," def set_para_loss(self, W, B, loss):\n"," self.n = self.n + 1\n"," self.W.append(W)\n"," self.B.append(B)\n"," self.LOSS.append(loss)\n"," \n"," # Plot diagram\n"," def final_plot(self): \n"," ax = plt.axes(projection = '3d')\n"," ax.plot_wireframe(self.w, self.b, self.Z)\n"," ax.scatter(self.W,self.B, self.LOSS, c = 'r', marker = 'x', s = 200, alpha = 1)\n"," plt.figure()\n"," plt.contour(self.w,self.b, self.Z)\n"," plt.scatter(self.W, self.B, c = 'r', marker = 'x')\n"," plt.xlabel('w')\n"," plt.ylabel('b')\n"," plt.show()\n"," \n"," # Plot diagram\n"," def plot_ps(self):\n"," plt.subplot(121)\n"," plt.ylim\n"," plt.plot(self.x, self.y, 'ro', label=\"training points\")\n"," plt.plot(self.x, self.W[-1] * self.x + self.B[-1], label = \"estimated line\")\n"," plt.xlabel('x')\n"," plt.ylabel('y')\n"," plt.ylim((-10, 15))\n"," plt.title('Data Space Iteration: ' + str(self.n))\n","\n"," plt.subplot(122)\n"," plt.contour(self.w, self.b, self.Z)\n"," plt.scatter(self.W, self.B, c = 'r', marker = 'x')\n"," plt.title('Total Loss Surface Contour Iteration' + str(self.n))\n"," plt.xlabel('w')\n"," plt.ylabel('b')\n"," plt.show()"]},{"cell_type":"markdown","metadata":{},"source":["<!--Empty Space for separating topics-->\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"Makeup_Data\">Make Some Data</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["Import PyTorch: \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Import PyTorch library\n","\n","import torch"]},{"cell_type":"markdown","metadata":{},"source":["Start with generating values from -3 to 3 that create a line with a slope of 1 and a bias of -1. This is the line that you need to estimate.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Create f(X) with a slope of 1 and a bias of -1\n","\n","X = torch.arange(-3, 3, 0.1).view(-1, 1)\n","f = 1 * X - 1"]},{"cell_type":"markdown","metadata":{},"source":["Now, add some noise to the data:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Add noise\n","\n","Y = f + 0.1 * torch.randn(X.size())"]},{"cell_type":"markdown","metadata":{},"source":["Plot the line and <code>Y</code> with noise:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Plot out the line and the points with noise\n","\n","plt.plot(X.numpy(), Y.numpy(), 'rx', label = 'y')\n","plt.plot(X.numpy(), f.numpy(), label = 'f')\n","plt.xlabel('x')\n","plt.ylabel('y')\n","plt.legend()"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"Model_Cost\">Create the Model and Cost Function (Total Loss)</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["Define the <code>forward</code> function: \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Define the forward function\n","\n","def forward(x):\n"," return w * x + b"]},{"cell_type":"markdown","metadata":{},"source":["Define the cost or criterion function (MSE): \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Define the MSE Loss function\n","\n","def criterion(yhat,y):\n"," return torch.mean((yhat-y)**2)"]},{"cell_type":"markdown","metadata":{},"source":["Create a <code> plot_error_surfaces</code> object to visualize the data space and the parameter space during training:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Create plot_error_surfaces for viewing the data\n","\n","get_surface = plot_error_surfaces(15, 15, X, Y, 30)"]},{"cell_type":"markdown","metadata":{},"source":["<!--Empty Space for separating topics-->\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2 id=\"Train\">Train the Model</h2>\n"]},{"cell_type":"markdown","metadata":{},"source":["Create model parameters <code>w</code>, <code>b</code> by setting the argument <code>requires_grad</code> to True because we must learn it using the data.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Define the parameters w, b for y = wx + b\n","\n","w = torch.tensor(-15.0, requires_grad = True)\n","b = torch.tensor(-10.0, requires_grad = True)"]},{"cell_type":"markdown","metadata":{},"source":["Set the learning rate to 0.1 and create an empty list <code>LOSS</code> for storing the loss for each iteration.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Define learning rate and create an empty list for containing the loss for each iteration.\n","\n","lr = 0.1\n","LOSS = []"]},{"cell_type":"markdown","metadata":{},"source":["Define <code>train_model</code> function for train the model.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# The function for training the model\n","\n","def train_model(iter):\n"," \n"," # Loop\n"," for epoch in range(iter):\n"," \n"," # make a prediction\n"," Yhat = forward(X)\n"," \n"," # calculate the loss \n"," loss = criterion(Yhat, Y)\n","\n"," # Section for plotting\n"," get_surface.set_para_loss(w.data.tolist(), b.data.tolist(), loss.tolist())\n"," if epoch % 3 == 0:\n"," get_surface.plot_ps()\n"," \n"," # store the loss in the list LOSS\n"," LOSS.append(loss)\n"," \n"," # backward pass: compute gradient of the loss with respect to all the learnable parameters\n"," loss.backward()\n"," \n"," # update parameters slope and bias\n"," w.data = w.data - lr * w.grad.data\n"," b.data = b.data - lr * b.grad.data\n"," \n"," # zero the gradients before running the backward pass\n"," w.grad.data.zero_()\n"," b.grad.data.zero_()"]},{"cell_type":"markdown","metadata":{},"source":["Run 15 iterations of gradient descent: <b>bug</b> data space is 1 iteration ahead of parameter space \n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Train the model with 15 iterations\n","\n","train_model(15)"]},{"cell_type":"markdown","metadata":{},"source":["Plot total loss/cost surface with loss values for different parameters in red:\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Plot out the Loss Result\n","\n","get_surface.final_plot()\n","plt.plot(LOSS)\n","plt.tight_layout()\n","plt.xlabel(\"Epoch/Iterations\")\n","plt.ylabel(\"Cost\")"]},{"cell_type":"markdown","metadata":{},"source":["<!--Empty Space for separating topics-->\n"]},{"cell_type":"markdown","metadata":{},"source":["<h3>Practice</h3>\n"]},{"cell_type":"markdown","metadata":{},"source":["Experiment using s learning rates 0.2 and width the following parameters. Run 15 iterations.\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Practice: train and plot the result with lr = 0.2 and the following parameters\n","\n","w = torch.tensor(-15.0, requires_grad = True)\n","b = torch.tensor(-10.0, requires_grad = True)\n","lr = 0.2\n","LOSS2 = []"]},{"cell_type":"markdown","metadata":{},"source":["Double-click <b>here</b> for the solution.\n","\n","<!-- \n","def my_train_model(iter):\n"," for epoch in range(iter):\n"," Yhat = forward(X)\n"," loss = criterion(Yhat, Y)\n"," get_surface.set_para_loss(w.data.tolist(), b.data.tolist(), loss.tolist())\n"," if epoch % 3 == 0:\n"," get_surface.plot_ps()\n"," LOSS2.append(loss)\n"," loss.backward()\n"," w.data = w.data - lr * w.grad.data\n"," b.data = b.data - lr * b.grad.data\n"," w.grad.data.zero_()\n"," b.grad.data.zero_()\n","my_train_model(15)\n","-->\n"]},{"cell_type":"markdown","metadata":{},"source":["Plot the <code>LOSS</code> and <code>LOSS2</code>\n"]},{"cell_type":"code","execution_count":null,"metadata":{},"outputs":[],"source":["# Practice: Plot the LOSS and LOSS2 in order to compare the Total Loss\n","\n","# Type your code here"]},{"cell_type":"markdown","metadata":{},"source":["Double-click <b>here</b> for the solution.\n","\n","<!--\n","plt.plot(LOSS, label = \"LOSS\")\n","plt.plot(LOSS2, label = \"LOSS2\")\n","plt.tight_layout()\n","plt.xlabel(\"Epoch/Iterations\")\n","plt.ylabel(\"Cost\")\n","plt.legend()\n","-->\n"]},{"cell_type":"markdown","metadata":{},"source":["<a href=\"https://dataplatform.cloud.ibm.com/registration/stepone?context=cpdaas&apps=data_science_experience,watson_machine_learning\"><img src=\"https://cf-courses-data.s3.us.cloud-object-storage.appdomain.cloud/IBMDeveloperSkillsNetwork-DL0110EN-SkillsNetwork/Template/module%201/images/Watson_Studio.png\"/></a>\n"]},{"cell_type":"markdown","metadata":{},"source":["<!--Empty Space for separating topics-->\n"]},{"cell_type":"markdown","metadata":{},"source":["<h2>About the Authors:</h2> \n","\n","<a href=\"https://www.linkedin.com/in/joseph-s-50398b136/\">Joseph Santarcangelo</a> has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n"]},{"cell_type":"markdown","metadata":{},"source":["Other contributors: <a href=\"https://www.linkedin.com/in/michelleccarey/\">Michelle Carey</a>, <a href=\"www.linkedin.com/in/jiahui-mavis-zhou-a4537814a\">Mavis Zhou</a> \n"]},{"cell_type":"markdown","metadata":{},"source":["## Change Log\n","\n","| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n","| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n","| 2020-09-21 | 2.0 | Shubham | Migrated Lab to Markdown and added to course repo in GitLab |\n"]},{"cell_type":"markdown","metadata":{},"source":["<hr>\n"]},{"cell_type":"markdown","metadata":{},"source":["## <h3 align=\"center\"> © IBM Corporation 2020. All rights reserved. <h3/>\n"]}],"metadata":{"kernelspec":{"display_name":"Python 3","language":"python","name":"python3"},"language_info":{"codemirror_mode":{"name":"ipython","version":3},"file_extension":".py","mimetype":"text/x-python","name":"python","nbconvert_exporter":"python","pygments_lexer":"ipython3","version":"3.7.6"}},"nbformat":4,"nbformat_minor":2} |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment