{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", " \"cognitiveclass.ai\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Linear Regression 1D: Prediction

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Objective

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Table of Contents

\n", "

In this lab, we will review how to make a prediction in several different ways by using PyTorch.\n", "

\n", "

Estimated Time Needed: 15 min

\n", "\n", "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Preparation

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The following are the libraries we are going to use for this lab.\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Collecting torch\n", " Downloading torch-1.8.1-cp38-cp38-win_amd64.whl (190.5 MB)\n", "Requirement already satisfied: numpy in c:\\users\\zhaleh\\anaconda3\\lib\\site-packages (from torch) (1.19.2)\n", "Requirement already satisfied: typing-extensions in c:\\users\\zhaleh\\anaconda3\\lib\\site-packages (from torch) (3.7.4.3)\n", "Installing collected packages: torch\n", "Successfully installed torch-1.8.1\n" ] } ], "source": [ "# These are the libraries will be used for this lab.\n", "!pip install torch\n", "import torch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Prediction

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us create the following expressions:\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$b=-1,w=2$\n", "\n", "$\\hat{y}=-1+2x$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, define the parameters:\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "# Define w = 2 and b = -1 for y = wx + b\n", "\n", "w = torch.tensor(2.0, requires_grad = True)\n", "b = torch.tensor(-1.0, requires_grad = True)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Then, define the function forward(x, w, b) makes the prediction: \n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [], "source": [ "# Function forward(x) for prediction\n", "\n", "def forward(x):\n", " yhat = w * x + b\n", " return yhat" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's make the following prediction at x = 1\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$\\hat{y}=-1+2x$\n", "\n", "$\\hat{y}=-1+2(1)$\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[1.]], grad_fn=)\n" ] } ], "source": [ "# Predict y = 2x - 1 at x = 1\n", "\n", "x = torch.tensor([[1.0]])\n", "yhat = forward(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let us try to make the prediction for multiple inputs:\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"Linear\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us construct the x tensor first. Check the shape of x.\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The shape of x: torch.Size([2, 1])\n" ] } ], "source": [ "# Create x Tensor and check the shape of x tensor\n", "\n", "x = torch.tensor([[1.0], [2.0]])\n", "print(\"The shape of x: \", x.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now make the prediction: \n" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[1.],\n", " [3.]], grad_fn=)\n" ] } ], "source": [ "# Make the prediction of y = 2x - 1 at x = [1, 2]\n", "\n", "yhat = forward(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The result is the same as what it is in the image above.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Practice

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make a prediction of the following x tensor using the w and b from above.\n" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[1.],\n", " [3.],\n", " [5.]], grad_fn=)" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Practice: Make a prediction of y = 2x - 1 at x = [[1.0], [2.0], [3.0]]\n", "\n", "x = torch.tensor([[1.0], [2.0], [3.0]])\n", "forward(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Double-click here for the solution.\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Class Linear

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The linear class can be used to make a prediction. We can also use the linear class to build more complex models. Let's import the module:\n" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "# Import Class Linear\n", "\n", "from torch.nn import Linear" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Set the random seed because the parameters are randomly initialized:\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Set random seed\n", "\n", "torch.manual_seed(1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us create the linear object by using the constructor. The parameters are randomly created. Let us print out to see what w and b. The parameters of an torch.nn.Module model are contained in the model’s parameters accessed with lr.parameters():\n" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parameters w and b: [Parameter containing:\n", "tensor([[0.5153]], requires_grad=True), Parameter containing:\n", "tensor([-0.4414], requires_grad=True)]\n" ] } ], "source": [ "# Create Linear Regression Model, and print out the parameters\n", "\n", "lr = Linear(in_features=1, out_features=1, bias=True)\n", "print(\"Parameters w and b: \", list(lr.parameters()))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "This is equivalent to the following expression: \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "$b=-0.44, w=0.5153$\n", "\n", "$\\hat{y}=-0.44+0.5153x$\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A method state_dict() Returns a Python dictionary object corresponding to the layers of each parameter tensor. \n" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Python dictionary: OrderedDict([('weight', tensor([[0.5153]])), ('bias', tensor([-0.4414]))])\n", "keys: odict_keys(['weight', 'bias'])\n", "values: odict_values([tensor([[0.5153]]), tensor([-0.4414])])\n" ] } ], "source": [ "print(\"Python dictionary: \",lr.state_dict())\n", "print(\"keys: \",lr.state_dict().keys())\n", "print(\"values: \",lr.state_dict().values())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The keys correspond to the name of the attributes and the values correspond to the parameter value.\n" ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "weight: Parameter containing:\n", "tensor([[0.5153]], requires_grad=True)\n", "bias: Parameter containing:\n", "tensor([-0.4414], requires_grad=True)\n" ] } ], "source": [ "print(\"weight:\",lr.weight)\n", "print(\"bias:\",lr.bias)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now let us make a single prediction at x = [[1.0]].\n" ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[0.0739]], grad_fn=)\n" ] } ], "source": [ "# Make the prediction at x = [[1.0]]\n", "\n", "x = torch.tensor([[1.0]])\n", "yhat = lr(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Similarly, you can make multiple predictions:\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\"Linear\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Use model lr(x) to predict the result.\n" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[0.0739],\n", " [0.5891]], grad_fn=)\n" ] } ], "source": [ "# Create the prediction using linear model\n", "\n", "x = torch.tensor([[1.0], [2.0]])\n", "yhat = lr(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Practice

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Make a prediction of the following x tensor using the linear regression model lr.\n" ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[0.0739],\n", " [0.5891],\n", " [1.1044]], grad_fn=)" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Practice: Use the linear regression model object lr to make the prediction.\n", "\n", "x = torch.tensor([[1.0],[2.0],[3.0]])\n", "lr(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Double-click here for the solution.\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Build Custom Modules

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let's build a custom module. We can make more complex models by using this method later on. \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, import the following library.\n" ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [], "source": [ "# Library for this section\n", "\n", "from torch import nn" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let us define the class: \n" ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "# Customize Linear Regression Class\n", "\n", "class LR(nn.Module):\n", " \n", " # Constructor\n", " def __init__(self, input_size, output_size):\n", " \n", " # Inherit from parent\n", " super(LR, self).__init__()\n", " self.linear = nn.Linear(input_size, output_size)\n", " \n", " # Prediction function\n", " def forward(self, x):\n", " out = self.linear(x)\n", " return out" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an object by using the constructor. Print out the parameters we get and the model.\n" ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The parameters: [Parameter containing:\n", "tensor([[-0.1939]], requires_grad=True), Parameter containing:\n", "tensor([0.4694], requires_grad=True)]\n", "Linear model: Linear(in_features=1, out_features=1, bias=True)\n" ] } ], "source": [ "# Create the linear regression model. Print out the parameters.\n", "\n", "lr = LR(1, 1)\n", "print(\"The parameters: \", list(lr.parameters()))\n", "print(\"Linear model: \", lr.linear)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us try to make a prediction of a single input sample.\n" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[0.2755]], grad_fn=)\n" ] } ], "source": [ "# Try our customize linear regression model with single input\n", "\n", "x = torch.tensor([[1.0]])\n", "yhat = lr(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, let us try another example with multiple samples.\n" ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The prediction: tensor([[0.2755],\n", " [0.0816]], grad_fn=)\n" ] } ], "source": [ "# Try our customize linear regression model with multiple input\n", "\n", "x = torch.tensor([[1.0], [2.0]])\n", "yhat = lr(x)\n", "print(\"The prediction: \", yhat)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "the parameters are also stored in an ordered dictionary :\n" ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Python dictionary: OrderedDict([('linear.weight', tensor([[-0.1939]])), ('linear.bias', tensor([0.4694]))])\n", "keys: odict_keys(['linear.weight', 'linear.bias'])\n", "values: odict_values([tensor([[-0.1939]]), tensor([0.4694])])\n" ] } ], "source": [ "print(\"Python dictionary: \", lr.state_dict())\n", "print(\"keys: \",lr.state_dict().keys())\n", "print(\"values: \",lr.state_dict().values())\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Practice

\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create an object lr1 from the class we created before and make a prediction by using the following tensor: \n" ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor([[ 0.2755],\n", " [ 0.0816],\n", " [-0.1122]], grad_fn=)" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Practice: Use the LR class to create a model and make a prediction of the following tensor.\n", "\n", "x = torch.tensor([[1.0], [2.0], [3.0]])\n", "lr(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Double-click here for the solution.\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ " \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

About the Authors:

\n", "\n", "Joseph Santarcangelo has a PhD in Electrical Engineering, his research focused on using machine learning, signal processing, and computer vision to determine how videos impact human cognition. Joseph has been working for IBM since he completed his PhD.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Other contributors: Michelle Carey, Mavis Zhou \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Change Log\n", "\n", "| Date (YYYY-MM-DD) | Version | Changed By | Change Description |\n", "| ----------------- | ------- | ---------- | ----------------------------------------------------------- |\n", "| 2020-09-21 | 2.0 | Shubham | Migrated Lab to Markdown and added to course repo in GitLab |\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "##

© IBM Corporation 2020. All rights reserved.

\n" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 2 }