{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o20p1vjIfmaR",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8a165ac093a29dee",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "# HW 2: Implementing a Neural Network\n",
    "\n",
    "In this homework assignment, you will implement a neural network from scratch using [`numpy`](https://numpy.org/).\n",
    "\n",
    "There are three parts to this assignment:\n",
    "- In part 1, you will implement a neural network that processes a single training sample at a time.  \n",
    "- In part 2, you will vectorize your implementation to process all training samples in parallel, and you will verify your backpropagation with gradient checking.\n",
    "- In part 3, you will use your neural network implementation to train a handwritten digit classifier on MNIST.\n",
    "\n",
    "**Environment:** this notebook needs `scikit-learn`, which was added to the course\n",
    "environment for this lab. If `import sklearn` fails below, follow the updated\n",
    "instructions under **Setup** on the Course Materials page and re-run\n",
    "`pip install -r requirements.txt`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zkY53S_2fmaV"
   },
   "source": [
    "A few important notes:\n",
    "\n",
    "* For parts 1 and 2, you should not modify any code except where you see `### YOUR CODE HERE`.  Please remove `raise NotImplementedError()` and write your code in its place.\n",
    "\n",
    "* This assignment also has two **written** exercises, in markdown cells rather than code cells: the table of gradients in Part 1, and the table of vectorized gradients in Part 2.  Fill in these tables directly in the markdown cell provided, and do not delete the cell.  If you are not comfortable writing the equations in LaTeX, plain text is acceptable as long as it is unambiguous.  These tables are not checked by the test cells, but they are graded, and filling them in before you write the corresponding function will save you a great deal of debugging.\n",
    "\n",
    "* We provide code cells that check your code as you progress through the assignment. These tests will help catch obvious errors, but passing these tests is not a guarantee that your implementation is correct.\n",
    "\n",
    "* You may add new cells anywhere in the notebook while working, but make sure to delete them before submitting your assignment.  Please do not change the sequence of the cells.\n",
    "\n",
    "* If you run cells out of order, it may change variable values which may cause tests to fail.  The easiest way to avoid this problem is to use the ***Restart & Run all*** option available in the **Kernel** tab.\n",
    "\n",
    "* **AI policy:** For this assignment, the only AI chatbot you may interact with is the Lab2 chatbot provided in glassroomai.  Do not use general-purpose AI tools such as ChatGPT or Claude.  You may use Google to look up general programming questions \u2014 for example, searching for the numpy function that computes the maximum of each column is perfectly fine.\n",
    "\n",
    "* **How to submit:** before submitting, do a ***Restart & Run all*** and verify that the notebook runs from beginning to end without errors.  Then save the notebook so that all of the outputs (printed values, plots, and test results) are visible in the saved file, and upload that `.ipynb` file to Canvas.  Do not clear the outputs before submitting \u2014 we grade the notebook as submitted, including its outputs."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WtnuiVUMfmaW",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d269cca8e2af3e60",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Part 1\n",
    "\n",
    "In this part you will implement a simple neural network with one hidden layer.  This implementation will process one data point at a time.  You will have to implement functions to do the following:\n",
    "\n",
    "* initialize network weights\n",
    "* calculate ReLU function\n",
    "* calculate softmax function\n",
    "* forward propagation\n",
    "* calculate cross-entropy loss\n",
    "* backward propagation\n",
    "* update network weights\n",
    "* train the network on a set of data points\n",
    "* use the trained network to make a prediction"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5a49w3l3fmaX"
   },
   "source": [
    "Let's first define our notation:\n",
    "\n",
    "* The input $x$ and output $y$ are vectors of shape $(1, n_0)$ and $(1, n_2)$ respectively. \n",
    "\n",
    "* The number of nodes in the input, hidden, and output layers are $n_0$, $n_1$, and $n_2$ respectively.\n",
    "\n",
    "* The weights connecting the input and hidden layer are given by the matrix $W_1$ of shape $(n_0, n_1)$.\n",
    "\n",
    "* The weights connecting the hidden and output layer are given by the matrix $W_2$ of shape $(n_1, n_2)$.\n",
    "\n",
    "* The bias terms feeding into the hidden layer are given by the vector $b_1$ of shape $(1, n_1)$.\n",
    "\n",
    "* The bias terms feeding into the output layer are given by the vector $b_2$ of shape $(1, n_2)$.\n",
    "\n",
    "* The ReLU activations for the hidden layer are given by the vector $a_1$ of shape $(1, n_1)$.\n",
    "\n",
    "* The softmax outputs at the output layer are given by the vector $a_2$ of shape $(1, n_2)$.\n",
    "\n",
    "Make sure to double check your dimensions when debugging.  Caution: An array of shape (1, n) is *not* the same as an array of shape (n, )."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "bFO-VNCifmaY"
   },
   "source": [
    "Let's define our layer sizes:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 214,
     "status": "ok",
     "timestamp": 1632503071416,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "QeFlr6zCfmaY",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-16443fd8b77928f3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "n0 = 4 # size of input layer\n",
    "n1 = 7 # size of hidden layer\n",
    "n2 = 3 # size of output layer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071416,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "l1Jvydwlfmaa",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-3398bc2c1228257c",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import time"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RQhfbKl0fmaa",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8b9e781573ae18c1",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Initialize weight matrices\n",
    "\n",
    "Initialize the weight matrices $W_1$ and $W_2$ with small random values.  You can use [`np.random.randn()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.random.randn.html) and multiply the matrices by $0.01$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071416,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "rSQe8pfefmab",
    "nbgrader": {
     "grade": false,
     "grade_id": "weights",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def initialize_weights(n0, n1, n2):\n",
    "    '''\n",
    "    Returns a tuple (W1, W2) containing the initialized weight matrices.\n",
    "    '''\n",
    "    np.random.seed(0)\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (W1, W2)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071416,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "QVnuJ6W0fmac",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-83f22147650c23d9",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "W1, W2 = initialize_weights(n0, n1, n2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071417,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "hLLcgC32fmac",
    "nbgrader": {
     "grade": true,
     "grade_id": "weights_test",
     "locked": true,
     "points": 0.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(W1, np.ndarray), \"W1 should be a numpy array.\"\n",
    "assert isinstance(W2, np.ndarray), \"W2 should be a numpy array.\"\n",
    "\n",
    "assert W1.shape[0] == n0, \"The number of rows in W1 is incorrect.\"\n",
    "assert W1.shape[1] == n1, \"The number of columns in W1 is incorrect.\"\n",
    "\n",
    "assert W2.shape[0] == n1, \"The number of rows in W2 is incorrect.\"\n",
    "assert W2.shape[1] == n2, \"The number of columns in W2 is incorrect.\"\n",
    "\n",
    "assert np.sum(np.abs(W1)) != 0, \"All the elements in W1 should not be zero.\"\n",
    "assert np.sum(np.abs(W2)) != 0, \"All the elements in W2 should not be zero.\"\n",
    "\n",
    "assert np.round(W1[0,0], decimals=8) == 0.01764052, \"Your matrices are not normalized correctly.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7xJODQWFfmac",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-da0bc716eed96fc2",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Initialize bias terms\n",
    "\n",
    "Initialize the bias terms $b_1$ and $b_2$ with zeros.  You can use [`np.zeros()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.zeros.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071417,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "bu1UP03wfmad",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-efc325bb7130d00b",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def initialize_bias(n0, n1, n2):\n",
    "    '''\n",
    "    Returns a tuple of vectors (b1, b2) with the initialized bias terms.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (b1, b2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071417,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "qENJ4N74fmad",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-c0ba658379bd6651",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "b1, b2 = initialize_bias(n0, n1, n2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071417,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "HIzs-m1lfmae",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-713f6dea75488458",
     "locked": true,
     "points": 0.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(b1, np.ndarray), \"b1 should be a numpy array.\"\n",
    "assert isinstance(b2, np.ndarray), \"b2 should be a numpy array.\"\n",
    "\n",
    "assert b1.shape[0] == 1, \"The number of rows in b1 is incorrect.\"\n",
    "assert b1.shape[1] == n1, \"The number of columns in b1 is incorrect.\"\n",
    "\n",
    "assert b2.shape[0] == 1, \"The number of rows in b2 is incorrect.\"\n",
    "assert b2.shape[1] == n2, \"The number of columns in b2 is incorrect.\"\n",
    "\n",
    "assert np.all(b1 == 0), \"The values in b1 should be all zeros, but they are not.\"\n",
    "assert np.all(b2 == 0), \"The values in b2 should be all zeros, but they are not.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "MxFxzjvpfmae",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-cb9f5e5331dc3cac",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### ReLU activation function\n",
    "\n",
    "$$\\text{relu}(z) = \\max(0, z)$$\n",
    "\n",
    "Define the ReLU activation function.  If the input is a numpy array, the function should apply the ReLU activation function to each element."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071418,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "2CV9lM1ffmae",
    "nbgrader": {
     "grade": false,
     "grade_id": "sigmoid",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def relu(z):\n",
    "    '''\n",
    "    Returns the ReLU function value for each element in z.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071418,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "cQY6v2Prfmae",
    "nbgrader": {
     "grade": true,
     "grade_id": "sigmoid_test",
     "locked": true,
     "points": 1,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert relu(0) == 0, \"Returns incorrect value for z = 0.\"\n",
    "assert relu(1) == 1, \"Returns incorrect value for z > 0.\"\n",
    "assert relu(-1) == 0, \"Returns incorrect value for z < 0.\"\n",
    "assert relu(np.inf) == np.inf, \"Returns incorrect value for z = infinity.\"\n",
    "assert relu(-np.inf) == 0, \"Returns incorrect value for z = -infinity.\"\n",
    "assert np.isnan(relu(np.nan)), \"Returns incorrect value when z is nan.\"\n",
    "\n",
    "assert np.allclose(relu(np.array([1., -1., 0., 3.])), np.array([1., 0., 0., 3.])), \"Returns incorrect values for 1D input.\"\n",
    "assert np.allclose(relu(np.array([[1., -2.], [-3., 4.]])), np.array([[1., 0.], [0., 4.]])), \"Returns incorrect values for 2D input.\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "pi7_xIoPfmaf"
   },
   "source": [
    "##### Softmax function\n",
    "\n",
    "$$ softmax([z_1, \\dots, z_k]) = \\left[\\frac{e^{z_1}}{e^{z_1}+ \\dots + e^{z_k}}, \\dots, \\frac{e^{z_k}}{e^{z_1}+ \\dots + e^{z_k}}\\right]$$\n",
    "\n",
    "The softmax function takes an input vector of length $k$ and returns a valid probability distribution.  In other words, it converts a set of $k$ numerical scores into a set of $k$ probabilities.  This is useful for multi-class classification, where we want to know the probability that the input is from a certain class.  Note that adding a constant scalar to all terms does not change the output:\n",
    "\n",
    "$$ softmax([z_1, z_2, \\dots, z_k]) = softmax([z_1 + c, z_2 + c, \\dots, z_k + c])$$\n",
    "\n",
    "In practice, it is useful to first subtract $c = max\\{z_1, z_2, \\dots, z_k\\}$ from the input terms in order to avoid numerical overflow or underflow.\n",
    "\n",
    "Define the softmax function.  Since we will use this function in both part 1 and part 2, we will implement a vectorized version.  Your implementation should use vectorized operations and broadcasting, so it should not have any for loops.  You may need to use [`np.max()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.ndarray.max.html), [`np.exp()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.exp.html), and [`np.sum()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.sum.html)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 143,
     "status": "ok",
     "timestamp": 1632503071556,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "b8vzOqvtfmaf"
   },
   "outputs": [],
   "source": [
    "def softmax(Z):\n",
    "    '''\n",
    "    Applies the softmax function to each row of a matrix Z.\n",
    "    This implementation should be vectorized, so it should not contain any for loops.\n",
    "    '''\n",
    "    assert isinstance(Z, np.ndarray) and Z.ndim == 2, \"Input should be a 2D numpy array.\"\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071557,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "29E2zLjSfmaf"
   },
   "outputs": [],
   "source": [
    "assert np.allclose(np.round(softmax(np.array([[0, 0, 0, 0]])), decimals=8), np.array([[0.25, 0.25, 0.25, 0.25]])), \"Returns incorrect result for all zero matrix.\"\n",
    "assert np.allclose(np.round(softmax(np.array([[1e6, 1e6, 1e6, 1e6]])), decimals=8), np.array([[0.25, 0.25, 0.25, 0.25]])), \"Your implementation does not handle numerical overflow.\"\n",
    "assert np.allclose(np.round(softmax(np.array([[1, 2, 3]])), decimals=8), np.array([[0.09003057, 0.24472847, 0.66524096]])), \"Returns incorrect result for non-zero single row matrix.\"\n",
    "assert np.allclose(np.round(softmax(np.array([[-2, 0, 10]])), decimals=8), np.array([[6.1400000e-06, 4.5400000e-05, 9.9994846e-01]])), \"Returns incorrect result for single row matrix with .\"\n",
    "assert np.allclose(np.round(softmax(np.array([[0, 0], [-3.2, 1.1], [4, 3.4]])), decimals=8),  np.array([[0.5, 0.5],[0.01338692, 0.98661308],[0.64565631, 0.35434369]])), \"Returns incorrect result for non-zero multiple row matrix.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "EDd2HM0Ofmaf",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-7ae9305e3ea32a70",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Forward propagation\n",
    "\n",
    "The process of propagating the input through the layers of the neural network to get the output is called forward propagation.  In our case, forward propagation consists of the following steps:\n",
    "\n",
    "* The input vector $x$ is fed into the network, multiplied by the weight matrix $W_1$, and the bias terms $b_1$ are added to the result.  The resulting vector $z_1$ is passed through the ReLU activation function to produce the hidden layer activations $a_1$.\n",
    "* The activations $a_1$ are fed as input to the second layer, multiplied by the weight matrix $W_2$, and the bias terms $b_2$ are added to the result.  The resulting vector $z_2$ is passed through the softmax function to produce the output activations $a_2$.\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "qDJMiJnGfmaf",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1bceffed8cdda3aa",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Implement the forward propagation function.  You can use [`np.matmul()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.matmul.html) or `@` for matrix multiplication.  Be sure to check the dimensions of your variables in order to determine the correct order of matrix multiplication. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071557,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "ULvCKxypfmag",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-3df38176993b76ee",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def forward_propagation(W1, W2, b1, b2, x):\n",
    "    '''\n",
    "    Propagates the input vector x through the neural network.  Returns a tuple of \n",
    "    vectors (a1, a2) containing the activations for the hidden and output layers.\n",
    "    '''\n",
    "    assert x.shape == (1, n0), \"Input does not have correct dimensions.\"\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (a1, a2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071557,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "_2kWpPk0fmag",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-b65765e64f2915d6",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "x = np.array([3, 4, 1, 2]).reshape((1, n0)) # reshape from size (n0, ) to size (1, n0)\n",
    "(a1, a2) = forward_propagation(W1, W2, b1, b2, x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071557,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "nupWUNRxfmag",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-8d260be3d1c66bca",
     "locked": true,
     "points": 1.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert a1.shape[0] == 1, \"The number of rows in a1 is incorrect.\"\n",
    "assert a1.shape[1] == n1, \"The number of columns in a1 is incorrect.\"\n",
    "\n",
    "assert a2.shape[0] == 1, \"The number of rows in a2 is incorrect.\"\n",
    "assert a2.shape[1] == n2, \"The number of columns in a2 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(a1, decimals=8), np.array([[0.06437829, 0.02850143, 0.04588357, 0.11633205, 0.08824104, 0.        , 0.00409608]])), \"Incorrect values in a1.\"\n",
    "assert np.allclose(np.round(a2, decimals=8), np.array([[0.33393859, 0.33318333, 0.33287808]])), \"Incorrect values in a2.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "gpHZcJlJfmag"
   },
   "source": [
    "##### Cross-entropy loss\n",
    "\n",
    "The cross-entropy loss for the $i^{th}$ sample is given by\n",
    "\n",
    "$$ J^{(i)} = - \\sum_{j=1}^k y^{(i)}_j \\log(p^{(i)}_j)$$\n",
    "\n",
    "The average cross-entropy loss across all samples is given by\n",
    "\n",
    "$$ J = - \\frac{1}{m} \\sum_{i=1}^m \\sum_{j=1}^k y^{(i)}_j \\log(p^{(i)}_j)$$\n",
    "\n",
    "where $y^{(i)}$ is the one-hot encoded label for the $i^{th}$ data point and $p^{(i)}_j$ is the probability of the $j^{th}$ class for the $i^{th}$ data point.  Note that the inner summation will only contain a single non-zero term.\n",
    "\n",
    "Implement the cross-entropy loss function.  Since we will use this function in both part 1 and part 2, this implementation should be vectorized."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071557,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "daDcWIS1fmah"
   },
   "outputs": [],
   "source": [
    "def cross_entropy_loss(A, Y):\n",
    "    '''\n",
    "    Calculates the cross-entropy loss for a set of output probabilities and target labels.\n",
    "    \n",
    "    A is a matrix where each row specifies the output layer activations for a single input.\n",
    "    Y is a matrix where each row specifies the one-hot encoded target label for a single input.\n",
    "    The computed cross entropy loss should be a scalar.    \n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071558,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "_hjPCEaTfmah"
   },
   "outputs": [],
   "source": [
    "A = np.array([[.25, .20, .05, .5], [.7, .1, .1, .1]])\n",
    "Y = np.array([[0, 1, 0, 0], [1, 0, 0, 0]])\n",
    "J = cross_entropy_loss(A, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071558,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "jGy3gDwYfmah"
   },
   "outputs": [],
   "source": [
    "assert np.isscalar(J), \"Output is not a scalar.\"\n",
    "assert np.round(J, decimals=8) == 0.98305643, \"Returns incorrect loss value.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "jvfx9X34fmah",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-337219485a399076",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Initialize gradients for weights and bias terms\n",
    "\n",
    "Initialize the gradients with zeros.  The shape of the gradients should be the same as the shape of the corresponding weight and bias terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 143,
     "status": "ok",
     "timestamp": 1632503071697,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "_2jYt2Cgfmah",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f4d3109f25579939",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def initialize_gradients(n0, n1, n2):\n",
    "    '''\n",
    "    Initializes the gradients with zeroes and returns a tuple of matrices (dW1, dW2, db1, db2)\n",
    "    corresponding to the gradients of W1, W2, b1, and b2, respectively.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (dW1, dW2, db1, db2)\n",
    "    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071698,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "Sm9VO6fWfmai",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-75af41da31d6fcb5",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "(dW1, dW2, db1, db2) = initialize_gradients(n0, n1, n2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1632503071698,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "WwKWYvjYfmai",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-b24cebb300db0bcc",
     "locked": true,
     "points": 0.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(dW1, np.ndarray), \"dW1 should be a numpy array.\"\n",
    "assert isinstance(dW2, np.ndarray), \"dW2 should be a numpy array.\"\n",
    "\n",
    "assert dW1.shape[0] == n0, \"The number of rows in dW1 is incorrect.\"\n",
    "assert dW1.shape[1] == n1, \"The number of columns in dW1 is incorrect.\"\n",
    "\n",
    "assert dW2.shape[0] == n1, \"The number of rows in dW2 is incorrect.\"\n",
    "assert dW2.shape[1] == n2, \"The number of columns in dW2 is incorrect.\"\n",
    "\n",
    "assert np.sum(np.abs(dW1)) == 0, \"All the elements in dW1 should be initialized to zero.\"\n",
    "assert np.sum(np.abs(dW2)) == 0, \"All the elements in dW2 should be initialized to zero.\"\n",
    "\n",
    "assert isinstance(db1, np.ndarray), \"db1 should be a numpy array.\"\n",
    "assert isinstance(db2, np.ndarray), \"db2 should be a numpy array.\"\n",
    "\n",
    "assert db1.shape[0] == 1, \"The number of rows in db1 is incorrect.\"\n",
    "assert db1.shape[1] == n1, \"The number of columns in db1 is incorrect.\"\n",
    "\n",
    "assert db2.shape[0] == 1, \"The number of rows in db2 is incorrect.\"\n",
    "assert db2.shape[1] == n2, \"The number of columns in db2 is incorrect.\"\n",
    "\n",
    "assert np.sum(np.abs(db1)) == 0, \"All the elements in db1 should be initialized to zero.\"\n",
    "assert np.sum(np.abs(db2)) == 0, \"All the elements in db2 should be initialized to zero.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "4EKtjo-ffmai",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e693c5b5445a71bc",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Backward propagation\n",
    "\n",
    "Backward propagation refers to the process of taking the error of the output prediction and propagating it backwards through the neural network in order to calculate the gradients of the network weights."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YqB2t81-fmai",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e544397f7c3728a5",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "***Exercise: from the lecture results to code***\n",
    "\n",
    "In lecture we derived all of the gradients you need for backpropagation.  You do **not** need to re-derive them here.  What you do need is to (a) translate them into the notation used in this notebook, and (b) know the shape of every quantity \u2014 that is what determines the order of matrix multiplication when you write the code.\n",
    "\n",
    "The lecture slides and this notebook use different notation for the same things:\n",
    "\n",
    "| Lecture | This notebook | Meaning |\n",
    "|---|---|---|\n",
    "| $\\bar{x}$ | $x$ | the input, shape $(1, n_0)$ |\n",
    "| $\\bar{z}^{(1)}$, $\\bar{a}^{(1)}$ | $z_1$, $a_1$ | hidden layer pre-activation and activation |\n",
    "| $\\bar{z}^{(2)}$, $\\bar{a}^{(2)} = \\hat{y}$ | $z_2$, $a_2$ | output layer pre-activation and activation |\n",
    "| $W^{(1)}, \\bar{b}^{(1)}, W^{(2)}, \\bar{b}^{(2)}$ | $W_1$, $b_1$, $W_2$, $b_2$ | weights and biases |\n",
    "| $s_0$, $s_1$, $s_2$ | $n_0$, $n_1$, $n_2$ | layer sizes |\n",
    "| $\\odot$ | $.*$ | elementwise multiplication |\n",
    "\n",
    "Note that the layer index appears as a **superscript** in the lecture slides and as a **subscript** here.  These are the same quantities.\n",
    "\n",
    "***Your task:*** For a single training sample, fill in the table in the next cell with the eight gradients, taking the results directly from lecture and rewriting them in this notebook's notation.  Denote matrix multiplication with $A * B$ and elementwise multiplication with $A .* B$.  Also give the shape of each gradient.\n",
    "\n",
    "Two checks to use as you go:\n",
    "\n",
    "* The gradient of $J$ with respect to any variable has the **same shape as that variable**.  For example, $\\frac{dJ}{dW_1}$ must have shape $(n_0, n_1)$, and $\\frac{dJ}{db_2}$ must have shape $(1, n_2)$.\n",
    "\n",
    "* Given the shapes, usually only one order of multiplication \u2014 and only one placement of a transpose \u2014 is even legal.  If the shapes in your expression don't line up, the expression is wrong.  This is a fast and surprisingly reliable way to catch mistakes."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "V_h-9GEwfmai",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-3a740c37e6a74cf1",
     "locked": false,
     "points": 0.5,
     "schema_version": 1,
     "solution": true
    }
   },
   "source": [
    "***Fill in the table below.***\n",
    "\n",
    "***Double-click this cell to edit it,*** and replace each *(fill in)* below with your answer.  Then run the cell (Shift+Enter) to render the table.\n",
    "\n",
    "Express each gradient in terms of $x$, $y$, $a_1$, $a_2$, $z_1$, $W_1$, and $W_2$.  For the ReLU derivative, you may write $\\mathbb{1}[z_1 > 0]$ for the indicator matrix that is 1 where $z_1 > 0$ and 0 elsewhere.\n",
    "\n",
    "| Gradient | Expression | Shape |\n",
    "|---|---|---|\n",
    "| $\\frac{dJ}{da_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dz_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dW_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{db_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{da_1}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dz_1}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dW_1}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{db_1}$ | *(fill in)* | *(fill in)* |"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "7lFqcdXyfmai",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-92d2558c67c5b193",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "We will use the following notation in our backpropagation function:\n",
    "\n",
    "`dW1`: $\\frac{dJ}{dW_1}$\n",
    "\n",
    "`dW2 `: $\\frac{dJ}{dW_2}$\n",
    "\n",
    "`db1` : $\\frac{dJ}{db_1}$\n",
    "\n",
    "`db2` : $\\frac{dJ}{db_2}$\n",
    "\n",
    "`dz1` : $\\frac{dJ}{dz_1}$\n",
    "\n",
    "`dz2` : $\\frac{dJ}{dz_2}$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "SxzqdgOGfmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-91886a9522c98896",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "Hints: \n",
    "- Be careful to distinguish between matrix multiplication and elementwise multiplication of matrices.  For matrix multiplication, you can use [`np.matmul()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.matmul.html) or `@`.  For elementwise multiplication, you can use [`np.multiply()`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.multiply.html) or `*`.\n",
    "- Pay close attention to the shape of matrices and vectors to determine the correct order of matrix multiplication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 2,
     "status": "ok",
     "timestamp": 1632503071698,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "nLZlzjeSfmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-eb4f12428289fdc5",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def back_propagation(W1, W2, b1, b2, a1, a2, x, y):\n",
    "    '''\n",
    "    Calculates the gradients of the network weights for a single sample.\n",
    "    Returns a tuple of matrices (dW1, dW2, db1, db2).\n",
    "    '''\n",
    "    assert x.shape == (1, n0)\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (dW1, dW2, db1, db2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 2,
     "status": "ok",
     "timestamp": 1632503071698,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "Ki0pUF2Yfmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-eef6ef875cee6d3b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.array([[3, 4, 1, -1]])\n",
    "y = np.array([1, 0, 0])\n",
    "dW1, dW2, db1, db2 = back_propagation(W1, W2, b1, b2, a1, a2, x, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 157,
     "status": "ok",
     "timestamp": 1632503071853,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "HVHxsPOVfmaj",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-669871e75c09463",
     "locked": true,
     "points": 2.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(db1, np.ndarray), \"db1 should be a numpy array.\"\n",
    "assert isinstance(db2, np.ndarray), \"db2 should be a numpy array.\"\n",
    "\n",
    "assert db2.shape[0] == 1, \"The number of rows in db2 is incorrect.\"\n",
    "assert db2.shape[1] == n2, \"The number of columns in db2 is incorrect.\"\n",
    "\n",
    "assert db1.shape[0] == 1, \"The number of rows in db1 is incorrect.\"\n",
    "assert db1.shape[1] == n1, \"The number of columns in db1 is incorrect.\"\n",
    "\n",
    "assert isinstance(dW1, np.ndarray), \"dW1 should be a numpy array.\"\n",
    "assert isinstance(dW2, np.ndarray), \"dW2 should be a numpy array.\"\n",
    "\n",
    "assert dW2.shape[0] == n1, \"The number of rows in dW2 is incorrect.\"\n",
    "assert dW2.shape[1] == n2, \"The number of columns in dW2 is incorrect.\"\n",
    "\n",
    "assert dW1.shape[0] == n0, \"The number of rows in dW1 is incorrect.\"\n",
    "assert dW1.shape[1] == n1, \"The number of columns in dW1 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(dW1[:,0], decimals=8), np.array([-0.01439342, -0.01919123, -0.00479781,  0.00479781])), \"Incorrect values for dW1.\"\n",
    "assert np.allclose(np.round(db1[0,0:4], decimals=8), np.array([-0.00479781, -0.01207039,  0.00693361, -0.01030540])), \"Incorrect values for db1.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "_wkk2NX8fmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-065a748cac173fdc",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Updating network weights\n",
    "\n",
    "The network weights are updated as follows:\n",
    "\n",
    "$W_1 = W_1 - \\alpha * \\frac{1}{m} \\sum_{i=1}^{m} dW^{(i)}_1$  \n",
    "\n",
    "$W_2 = W_2 - \\alpha * \\frac{1}{m} \\sum_{i=1}^{m} dW^{(i)}_2$  \n",
    "\n",
    "$b_1 = b_1 - \\alpha * \\frac{1}{m} \\sum_{i=1}^{m} db^{(i)}_1$  \n",
    "\n",
    "$b_2 = b_2 - \\alpha * \\frac{1}{m} \\sum_{i=1}^{m} db^{(i)}_2$\n",
    "\n",
    "where $\\alpha$ is the learning rate, $m$ is the number of training examples, and $dW^{(i)}_1$, $dW^{(i)}_2$, $db^{(i)}_1$, and $db^{(i)}_2$ are the gradients for the $i^{th}$ training sample."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071853,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "HgLCL9G9fmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-f5cf67f3f7bcd0fc",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def update_parameters(W1, W2, b1, b2, dW1, dW2, db1, db2, alpha):\n",
    "    '''\n",
    "    Returns the weights after a single gradient descent update.\n",
    "    dW1, dW2, db1, and db2 are the gradients averaged across all training samples.\n",
    "    W1, W2, b1, and b2 are the network weights, which should be updated and returned.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (W1, W2, b1, b2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503071854,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "O0L2Nj0ufmaj",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-0134d8ebccc1ed19",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "alpha = .1\n",
    "W1_prev, W2_prev, b1_prev, b2_prev = W1.copy(), W2.copy(), b1.copy(), b2.copy()\n",
    "W1, W2, b1, b2 = update_parameters(W1, W2, b1, b2, dW1, dW2, db1, db2, alpha)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503071854,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "Hce0YV2jfmak",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-669871e75c094763",
     "locked": true,
     "points": 1,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(b1, np.ndarray), \"b1 should be a numpy array.\"\n",
    "assert isinstance(b2, np.ndarray), \"b2 should be a numpy array.\"\n",
    "\n",
    "assert b1.shape[0] == 1, \"The number of rows in b1 is incorrect.\"\n",
    "assert b1.shape[1] == n1, \"The number of columns in b1 is incorrect.\"\n",
    "\n",
    "assert b2.shape[0] == 1, \"The number of rows in b2 is incorrect.\"\n",
    "assert b2.shape[1] == n2, \"The number of columns in b2 is incorrect.\"\n",
    "\n",
    "assert isinstance(W1, np.ndarray), \"W1 should be a numpy array.\"\n",
    "assert isinstance(W2, np.ndarray), \"W2 should be a numpy array.\"\n",
    "\n",
    "assert W1.shape[0] == n0, \"The number of rows in W1 is incorrect.\"\n",
    "assert W1.shape[1] == n1, \"The number of columns in W1 is incorrect.\"\n",
    "\n",
    "assert W2.shape[0] == n1, \"The number of rows in W2 is incorrect.\"\n",
    "assert W2.shape[1] == n2, \"The number of columns in W2 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round((W1 - W1_prev)[:,0], decimals=8), np.array([ 0.00143934,  0.00191912,  0.00047978, -0.00047978])), \"Incorrect update to W1.\"\n",
    "assert np.allclose(np.round((b1 - b1_prev)[0,0:4], decimals=8), np.array([ 0.00047978,  0.00120704, -0.00069336,  0.00103054])), \"Incorrect updatae to b1.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "LJ66AgGefmak",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-aa3bc08c52b19d23",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "#### Training\n",
    "\n",
    "In this function we will bring all the functions together to iteratively update the weights `W1`, `b1`, `W2`, and `b2` on a set of training examples.\n",
    "\n",
    "* The outer for loop iterates over the dataset `num_epochs` times, updating the weights once per epoch.  Each update should be based on the gradients averaged across all training samples in a single epoch.\n",
    "* The inner for loop iterates over the training samples in the dataset, accumulating gradients from each data point.  To calculate the gradient on each data point, you will need to do forward and backward propagation.\n",
    "* We will also want to calculate the average cross entropy loss across the training samples, so that we can monitor the training progress."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 149,
     "status": "ok",
     "timestamp": 1632503072000,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "DDnoK4-ufmak",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-1002d9b35d78cdce",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def training(X, Y, num_hidden, alpha, num_epochs):\n",
    "    '''\n",
    "    Learn the network weights on a set of training examples.  This implementation is not vectorized.\n",
    "    \n",
    "    Inputs\n",
    "    ------\n",
    "    X: a matrix containing the training data, where each row corresponds to a single training example\n",
    "    Y: a matrix containing the target labels, where each row contains a one hot encoded target label.\n",
    "    num_hidden: specifies the number of hidden units in the neural network.\n",
    "    alpha: the learning rate for gradient descent\n",
    "    num_epochs: the number of times to iterate over the dataset\n",
    "    \n",
    "    Returns a tuple of matrices (W1, W2, b1, b2) containing the learned network weights.\n",
    "    '''\n",
    "    \n",
    "    n0 = X.shape[1]\n",
    "    n1 = num_hidden\n",
    "    n2 = Y.shape[1]\n",
    "    m = X.shape[0]\n",
    "    \n",
    "    W1, W2 = initialize_weights(n0, n1, n2)\n",
    "    b1, b2 = initialize_bias(n0, n1, n2)\n",
    "    \n",
    "    for epoch in range(num_epochs):\n",
    "\n",
    "        loss = 0\n",
    "        dW1, dW2, db1, db2 = initialize_gradients(n0, n1, n2)\n",
    "        \n",
    "        for i in range(m):\n",
    "            \n",
    "            x = X[i,:].reshape((1,-1))\n",
    "            y = Y[i,:].reshape((1,-1))\n",
    "\n",
    "            ### YOUR CODE HERE\n",
    "            raise NotImplementedError()\n",
    "        \n",
    "        dW1 = dW1 / m\n",
    "        dW2 = dW2 / m\n",
    "        db1 = db1 / m\n",
    "        db2 = db2 / m\n",
    "        loss = loss / m\n",
    "        \n",
    "        ### YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "        \n",
    "        if epoch % 100 == 0:\n",
    "            print('Epoch {}: Training Loss = {:.8f}'.format(epoch, loss))\n",
    "        \n",
    "    return W1, W2, b1, b2\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PGN22ROzfmak",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-e4a3aa07781d2935",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "To test our training function, we can generate a small dataset suitable for a classification task using [`make_classification`](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_classification.html) in [scikit-learn](https://scikit-learn.org/stable/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 0
    },
    "executionInfo": {
     "elapsed": 1550,
     "status": "ok",
     "timestamp": 1632503073548,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "y6gvFfN8fmal",
    "outputId": "3df36553-2ba0-4685-deb5-7ae99d54deae"
   },
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "\n",
    "m = 100\n",
    "X, Y = make_classification(n_samples=m, n_features=n0, n_redundant=0, n_informative=n0, n_classes=n2, random_state=1)\n",
    "plt.scatter(X[:, 0], X[:, 1], marker='o', c=Y, s=25, edgecolor='k');\n",
    "plt.show()\n",
    "Y = np.eye(n2)[Y]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 29367,
     "status": "ok",
     "timestamp": 1632503102913,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "jUE19uULfmal",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-01cb6166d1d9cf72",
     "locked": true,
     "schema_version": 1,
     "solution": false
    },
    "outputId": "c2dad751-d06d-414f-8a83-794ccc6ed0d0"
   },
   "outputs": [],
   "source": [
    "W1, W2, b1, b2 = training(X, Y, n1, 1, 3000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 6,
     "status": "ok",
     "timestamp": 1632503102915,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "wSzUAjRSfmam",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-e73ff98fe76d271b",
     "locked": true,
     "points": 1,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(b1, np.ndarray), \"b1 should be a numpy array.\"\n",
    "assert isinstance(b2, np.ndarray), \"b2 should be a numpy array.\"\n",
    "\n",
    "assert b1.shape[0] == 1, \"The number of rows in b1 is incorrect.\"\n",
    "assert b1.shape[1] == n1, \"The number of columns in b1 is incorrect.\"\n",
    "\n",
    "assert b2.shape[0] == 1, \"The number of rows in b2 is incorrect.\"\n",
    "assert b2.shape[1] == n2, \"The number of columns in b2 is incorrect.\"\n",
    "\n",
    "assert isinstance(W1, np.ndarray), \"W1 should be a numpy array.\"\n",
    "assert isinstance(W2, np.ndarray), \"W2 should be a numpy array.\"\n",
    "\n",
    "assert W1.shape[0] == n0, \"The number of rows in W1 is incorrect.\"\n",
    "assert W1.shape[1] == n1, \"The number of columns in W1 is incorrect.\"\n",
    "\n",
    "assert W2.shape[0] == n1, \"The number of rows in W2 is incorrect.\"\n",
    "assert W2.shape[1] == n2, \"The number of columns in W2 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(W1[0,:], decimals=8), np.array([ 3.70240917, -1.49833461,  2.6521066 ,  0.57524552,  0.56873664, -1.83484853,  1.36708234])), \"Trained weights W1 are incorrect.\"\n",
    "assert np.allclose(np.round(b1, decimals = 8), np.array([-3.53744454, -4.85498228, -1.04975656, -2.78135901,  0.42618399,  2.62152429, -1.46579243])), \"Trained weights b1 are incorrect.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "-xxyWqVEfmam",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-ca398026a3c2ec38",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "##### Prediction\n",
    "\n",
    "Write a function to predict the label and class probabilities for a single input using the trained neural network.  You can use the function [`np.argmax`](https://docs.scipy.org/doc/numpy/reference/generated/numpy.argmax.html) to find the largest element in an array."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503102915,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "I3s2_diJfmam",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-20971a390d8ed68a",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def predict(W1, W2, b1, b2, x):\n",
    "    '''\n",
    "    Returns the predicted label y and class probabilities p for a single input.\n",
    "    The predicted label should be an integer, and the class probabilities should be a vector.\n",
    "    '''\n",
    "    assert isinstance(x, np.ndarray) and x.shape == (1, n0), \"Input should be a numpy array of shape (1, n0).\"\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return y, p"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503102915,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "iyFqBUZmfmam",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-beecf6d027507509",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "x = np.array([[.9, 2.1, 0.1, -.1]])\n",
    "y, p = predict(W1, W2, b1, b2, x)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503102916,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "qNkxzBhjfmam",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-61b6afcf6bd5d13e",
     "locked": true,
     "points": 0.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert not isinstance(y, np.ndarray), \"y should be an integer, not a numpy array.\"\n",
    "assert isinstance(p, np.ndarray), \"p should be a numpy array.\"\n",
    "\n",
    "assert y == 1, \"Predicted label is incorrect.\"\n",
    "assert np.allclose(np.round(p, decimals=8), np.array([0., 1., 0.])), \"Class probabilities are incorrect.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CYBEL06yfmam"
   },
   "source": [
    "Congratulations, you just implemented your first neural network!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "C_Fn7fstfman",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-964b64a3161c7b1b",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "## Part 2\n",
    " \n",
    "In the second part of this assignment, you will re-implement several functions using vectorization.  The main reason for vectorizing is to speed up the training process by performing operations in parallel rather than sequentially.  Rather than processing each training sample individually, you will stack all of the training examples into a matrix and rewrite the following functions to process all training samples simultaneously:\n",
    "\n",
    "* forward propagation    \n",
    "* backward propagation      \n",
    "* training the network\n",
    "* making predictions on new inputs\n",
    "\n",
    "You will also write a function for gradient checking to verify that your backpropagation implementation is correct."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "M2h55hy7fman"
   },
   "source": [
    "Let's define our notation: \n",
    "\n",
    "* The number of nodes in the input, hidden, and output layers are $n_0$, $n_1$, and $n_2$ respectively.\n",
    "\n",
    "* The number of training samples in the data set is $m$.\n",
    "\n",
    "* The input $X$ is a matrix of shape $(m, n_0)$ where each row corresponds to a training example.\n",
    "\n",
    "* The output $Y$ is a matrix of shape $(m, n_2)$ where each row corresponds to a one hot encoded target label.\n",
    "\n",
    "* The weights connecting the input and hidden layer are given by the matrix $W_1$ of shape $(n_0, n_1)$.\n",
    "\n",
    "* The weights connecting the hidden and output layer are given by the matrix $W_2$ of shape $(n_1, n_2)$.\n",
    "\n",
    "* The bias terms feeding into the hidden layer are given by the vector $b_1$ of shape $(1, n_1)$.\n",
    "\n",
    "* The bias terms feeding into the output layer are given by the vector $b_2$ of shape $(1, n_2)$.\n",
    "\n",
    "* The ReLU activations for the hidden layer are given by the matrix $A_1$ of shape $(m, n_1)$.\n",
    "\n",
    "* The softmax outputs at the output layer are given by the matrix $A_2$ of shape $(m, n_2)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PkfrxK7Rfman",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-d38530a1b9120d37",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "##### Forward propagation with vectorization\n",
    "\n",
    "Re-implement the forward propagation function to handle all training samples simultaneously.  Make sure to take advantage of broadcasting!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503102916,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "FFLL-RVxfman",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-5d2b143c49cfe1f8",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def forward_propagation_using_vectorization(W1, W2, b1, b2, X):\n",
    "    '''\n",
    "    Performs forward propagation on a set of data samples.\n",
    "    X is a matrix of input data, where each row corresponds to a single data sample.\n",
    "    \n",
    "    Returns a tuple of matrices (A1, A2) containing the activations at the hidden and output layers\n",
    "    for all training examples in X.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (A1, A2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503102916,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "ksdgziVbfmao",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-286bc4d6d399d25d",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "(A1, A2) = forward_propagation_using_vectorization(W1, W2, b1, b2, X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503102916,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "xSL_47wBfmao",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-207d184f712491b0",
     "locked": true,
     "points": 0.5,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "m = X.shape[0]\n",
    "\n",
    "assert A1.shape[0] == m, \"The number of rows in A1 is incorrect.\"\n",
    "assert A1.shape[1] == n1, \"The number of columns in A1 is incorrect.\"\n",
    "\n",
    "assert A2.shape[0] == m, \"The number of rows in A2 is incorrect.\"\n",
    "assert A2.shape[1] == n2, \"The number of columns in A2 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(A1[50,0:4], decimals=8), np.array([ 0.        ,  0.        , 11.03984275,  0.        ])), \"Incorrect values in A1.\"\n",
    "assert np.allclose(np.round(A2[80,:], decimals=8), np.array([9.9999927e-01, 0.0000000e+00, 7.3000000e-07])), \"Incorrect values in A2.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "evPJMO6Cfmao",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8100d0944daf08bd",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "##### Backward propagation using vectorization.\n",
    "\n",
    "Re-implement the backward propagation function to handle all training samples simultaneously. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "***Exercise: the vectorized gradients***\n",
    "\n",
    "Both the lecture derivation and your Part 1 code handle a **single** sample.  Here you will process all $m$ samples at once, and the weight update uses the gradients averaged over the samples:\n",
    "\n",
    "$$\\frac{dJ}{dW_1} = \\frac{1}{m} \\sum_{i=1}^{m} \\left. \\frac{dJ}{dW_1} \\right|^{(i)}$$\n",
    "\n",
    "and similarly for $W_2$, $b_1$, and $b_2$.\n",
    "\n",
    "Again, there is nothing new to derive.  You just need to state what the single-sample results become once the samples are stacked into matrices.  ***Fill in the table in the next cell.***\n",
    "\n",
    "Two things to think about:\n",
    "\n",
    "* Stacking the samples turns $x$ into $X$ of shape $(m, n_0)$ and $y$ into $Y$ of shape $(m, n_2)$, and the intermediate quantities $z_1, a_1, z_2, a_2$ likewise become matrices $Z_1, A_1, Z_2, A_2$ with $m$ rows.  But the gradients with respect to the **weights** keep the shapes they had before: $\\frac{dJ}{dW_1}$ is still $(n_0, n_1)$, because it is the gradient with respect to $W_1$, whose shape has not changed.\n",
    "\n",
    "* For each gradient, ask yourself *where the sum over the $m$ samples happens*.  In some of these expressions a matrix multiplication already performs that sum for you; in others there is no matrix multiplication to do it, and you will have to sum over the sample axis explicitly.  Your answer should make clear which is which.  Getting this wrong \u2014 either double-counting the $\\frac{1}{m}$ or leaving it out \u2014 is the single most common bug in Part 2."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-vectorized-grad-table",
     "locked": false,
     "points": 0.5,
     "schema_version": 1,
     "solution": true
    }
   },
   "source": [
    "***Fill in the table below.***\n",
    "\n",
    "***Double-click this cell to edit it,*** and replace each *(fill in)* below with your answer.  Then run the cell (Shift+Enter) to render the table.\n",
    "\n",
    "Express each gradient in terms of $X$, $Y$, $A_1$, $A_2$, $Z_1$, $W_2$, and $m$.\n",
    "\n",
    "| Gradient | Vectorized expression | Shape |\n",
    "|---|---|---|\n",
    "| $\\frac{dJ}{dZ_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dW_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{db_2}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dZ_1}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{dW_1}$ | *(fill in)* | *(fill in)* |\n",
    "| $\\frac{dJ}{db_1}$ | *(fill in)* | *(fill in)* |"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503102917,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "qytKrc5qfmao",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-6e63a3081ade3038",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def back_propagation_using_vectorization(W1, W2, b1, b2, A1, A2, X, Y):\n",
    "    '''\n",
    "    Performs backpropagation on a set of training samples using vectorization.\n",
    "    X is a matrix of input data, where each row corresponds to a single data sample.\n",
    "    Y is a matrix of target labels, where each row corresponds to a one hot encoded label.\n",
    "    \n",
    "    Returns a tuple of matrices (dW1, dW2, db1, db2) containing the average gradients across all training examples.\n",
    "    '''\n",
    "    m = X.shape[0]\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return (dW1, dW2, db1, db2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503102917,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "9fnXFBc8fmao",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-11a32da3b4ff94ca",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "dW1, dW2, db1, db2 = back_propagation_using_vectorization(W1, W2, b1, b2, A1, A2, X, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 184,
     "status": "ok",
     "timestamp": 1632503103097,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "3kjRTrYQfmap",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-800d0fae4f1c4077",
     "locked": true,
     "points": 1,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(dW1, np.ndarray), \"dW1 should be a numpy array.\"\n",
    "assert isinstance(dW2, np.ndarray), \"dW2 should be a numpy array.\"\n",
    "\n",
    "assert dW2.shape[0] == n1, \"The number of rows in dW2 is incorrect.\"\n",
    "assert dW2.shape[1] == n2, \"The number of columns in dW2 is incorrect.\"\n",
    "\n",
    "assert dW1.shape[0] == n0, \"The number of rows in dW1 is incorrect.\"\n",
    "assert dW1.shape[1] == n1, \"The number of columns in dW1 is incorrect.\"\n",
    "\n",
    "assert isinstance(db1, np.ndarray), \"db1 should be a numpy array.\"\n",
    "assert isinstance(db2, np.ndarray), \"db2 should be a numpy array.\"\n",
    "\n",
    "assert db2.shape[0] == 1, \"The number of rows in db2 is incorrect.\"\n",
    "assert db2.shape[1] == n2, \"The number of columns in db2 is incorrect.\"\n",
    "\n",
    "assert db1.shape[0] == 1, \"The number of rows in db1 is incorrect.\"\n",
    "assert db1.shape[1] == n1, \"The number of columns in db1 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(dW1[:,5], decimals=8), np.array([ 0.00886289,  0.00601732,  0.00180937, -0.00448098])), \"Incorrect values in dW1.\"\n",
    "assert np.allclose(np.round(db1[0,0:4], decimals=8), np.array([-0.0042399 ,  0.00357332,  0.00108065,  0.00101495])), \"Incorrect values in db1.\"\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n33ToutRfmap"
   },
   "source": [
    "#### Gradient Checking\n",
    "\n",
    "Backpropagation is easy to get subtly wrong: a transposed matrix or a missing $\\frac{1}{m}$ produces gradients that are wrong but still look plausible, and the network will often still train \u2014 just badly.  Gradient checking, which we covered in the videos, is how you catch this.  The idea is to compare your analytical gradients against a numerical estimate of the same quantity.\n",
    "\n",
    "You will implement the gradient checking function below.  It consists of the following steps:\n",
    "- Set your network weights to arbitrary default values.\n",
    "- Use your forward and backward propagation functions to calculate the gradients of the network weights.  These are the values that we would like to verify.\n",
    "- For each network weight, compute the approximate gradient by doing the following:\n",
    "    - Add a small epsilon to the weight/bias element of interest to create a modified set of network weights.\n",
    "    - Perform forward propagation and calculate the loss for the modified network weights ($J_+$).\n",
    "    - Subtract a small epsilon from the weight/bias element of interest to create another modified set of network weights.\n",
    "    - Perform forward propagation and calculate the loss of these modified network weights ($J_-$).\n",
    "    - Compute the approximate gradient for the element of interest as $\\frac{J_+ - J_-}{2 * \\epsilon}$.\n",
    "- Once you have computed an approximate gradient for each network weight, compare your approximate gradient to your backpropagated gradient to verify that they approximately match.\n",
    "\n",
    "To simplify your implementation, we provide utility functions for flattening all network weights into a single vector and for extracting the network weights back out of a flattened vector.  Working with one flat vector means you can loop over every weight and bias in the network with a single `for` loop, rather than writing separate nested loops for $W_1$, $W_2$, $b_1$, and $b_2$.\n",
    "\n",
    "Most of the function below is provided for you, including the bookkeeping that perturbs one element of the weight vector at a time.  **You only need to fill in the middle section**: given the two perturbed sets of weights, compute the two losses $J_+$ and $J_-$ and use them to fill in one element of `grad_approx`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503103097,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "oCavvzkYfmap",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-03f94f0db38a2648",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def flattened(W1, W2, b1, b2):\n",
    "    V = np.concatenate((W1.flatten(), W2.flatten(), b1.flatten(), b2.flatten()))\n",
    "    return V\n",
    "\n",
    "def reshaped(V, n0, n1, n2):\n",
    "    t1 = n0*n1\n",
    "    t2 = t1 + n1*n2\n",
    "    t3 = t2 + n1\n",
    "    W1 = V[:t1].reshape(n0, n1)\n",
    "    W2 = V[t1:t2].reshape(n1, n2)\n",
    "    b1 = V[t2:t3].reshape(1, n1)\n",
    "    b2 = V[t3:].reshape(1, n2)\n",
    "    return W1, W2, b1, b2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1632503103097,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "z-zpRJT4fmap",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-8eed26ec413655ba",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def gradient_checking(W1, W2, b1, b2, X, Y, epsilon = 1e-8):\n",
    "    '''\n",
    "    Verifies the backpropagation gradients against numerical gradients computed with\n",
    "    finite differences.  Returns the maximum absolute difference between the two.\n",
    "    '''\n",
    "    \n",
    "    n0, n1, n2 = W1.shape[0], W2.shape[0], W2.shape[1]\n",
    "    \n",
    "    # The gradients from backpropagation.  These are the values we want to verify.\n",
    "    (A1, A2) = forward_propagation_using_vectorization(W1, W2, b1, b2, X)\n",
    "    dW1, dW2, db1, db2 = back_propagation_using_vectorization(W1, W2, b1, b2, A1, A2, X, Y)\n",
    "    \n",
    "    V = flattened(W1, W2, b1, b2)          # all network parameters as a single 1D vector\n",
    "    grad = flattened(dW1, dW2, db1, db2)   # the backprop gradients, in that same order\n",
    "    grad_approx = np.zeros_like(grad)\n",
    "    \n",
    "    for i in range(len(V)):\n",
    "        \n",
    "        # Provided: make two copies of V, one with epsilon added to element i and one with\n",
    "        # epsilon subtracted from element i, then unpack each back into network weights.\n",
    "        V_plus = V.copy()\n",
    "        V_plus[i] += epsilon\n",
    "        V_minus = V.copy()\n",
    "        V_minus[i] -= epsilon\n",
    "        \n",
    "        W1_plus, W2_plus, b1_plus, b2_plus = reshaped(V_plus, n0, n1, n2)\n",
    "        W1_minus, W2_minus, b1_minus, b2_minus = reshaped(V_minus, n0, n1, n2)\n",
    "        \n",
    "        # Run forward propagation with each of the two perturbed sets of weights above,\n",
    "        # compute the resulting losses J_plus and J_minus, and store the central-difference\n",
    "        # approximation of the gradient with respect to V[i] in grad_approx[i].\n",
    "        ### YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "        \n",
    "    # Provided: the largest absolute difference between the backprop gradients and the\n",
    "    # numerical approximations.\n",
    "    maxerror = np.max(np.abs(grad - grad_approx))\n",
    "    \n",
    "    print(\"Max error in gradient approximation: {}\".format(maxerror))\n",
    "    if maxerror < epsilon:\n",
    "        print('Passes gradient checking')\n",
    "    else:\n",
    "        print('Fails gradient checking')\n",
    "    return maxerror"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1632503103097,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "nD_re-04fmap",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-82fa87c2c64e21d3",
     "locked": true,
     "schema_version": 1,
     "solution": false
    },
    "outputId": "821393d2-7c78-4279-948a-ad18d5589847"
   },
   "outputs": [],
   "source": [
    "maxerror = gradient_checking(W1, W2, b1, b2, X, Y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 2,
     "status": "ok",
     "timestamp": 1632503103097,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "iGTQGEf6fmaq",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-9ac25807fda5c343",
     "locked": true,
     "points": 0,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert maxerror < 1e-8, \"Gradient checking fails.\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "1kIYgYZUfmaq",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-3a2b9dc065ea4fb5",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "##### Training using vectorization\n",
    "\n",
    "Re-implement the training function with vectorization.  Note that the inner for loop will be eliminated, since the forward and backward propagation functions now process the entire training set simultaneously.  You will still iterate multiple times thru the entire training dataset, so the outer for loop will remain.  Your vectorized implementation should be much, much faster than your earlier implementation!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1632503103098,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "NwWmWjb0fmaq",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-236b66d1780cbab5",
     "locked": false,
     "schema_version": 1,
     "solution": true
    }
   },
   "outputs": [],
   "source": [
    "def training_using_vectorization(X, Y, num_hidden, alpha, num_epochs):\n",
    "    '''\n",
    "    Learn the network weights on a set of training examples.  This implementation is vectorized.\n",
    "    \n",
    "    Inputs\n",
    "    ------\n",
    "    X: a matrix containing the training data, where each row corresponds to a single training example\n",
    "    Y: a matrix containing the target labels, where each row contains a one hot encoded target label.\n",
    "    num_hidden: specifies the number of hidden units in the neural network.\n",
    "    alpha: the learning rate for gradient descent\n",
    "    num_epochs: the number of times to iterate over the dataset\n",
    "    \n",
    "    Returns a tuple of matrices (W1, W2, b1, b2) containing the learned network weights.\n",
    "    '''\n",
    "    \n",
    "    n0 = X.shape[1] \n",
    "    n1 = num_hidden\n",
    "    n2 = Y.shape[1]\n",
    "    m = X.shape[0]\n",
    "    \n",
    "    W1, W2 = initialize_weights(n0, n1, n2)\n",
    "    b1, b2 = initialize_bias(n0, n1, n2)\n",
    "    \n",
    "    for epoch in range(num_epochs):\n",
    "        \n",
    "        ### YOUR CODE HERE\n",
    "        raise NotImplementedError()\n",
    "        \n",
    "        if epoch % 100 == 0:\n",
    "            print('Epoch {}: Training Loss = {:.8f}'.format(epoch, loss))    \n",
    "    \n",
    "    return W1, W2, b1, b2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 705,
     "status": "ok",
     "timestamp": 1632503103800,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "FaIoG14afmaq",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-27d66c80e96932e6",
     "locked": true,
     "schema_version": 1,
     "solution": false
    },
    "outputId": "11d28229-1bc4-426e-ae13-10d6173e8be0"
   },
   "outputs": [],
   "source": [
    "W1, W2, b1, b2 = training_using_vectorization(X, Y, n1, 1, 3000)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1632503103801,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "n50u2i-tfmaq",
    "nbgrader": {
     "grade": true,
     "grade_id": "cell-efbd6e34ff91e003",
     "locked": true,
     "points": 0,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "assert isinstance(b1, np.ndarray), \"b1 should be a numpy array.\"\n",
    "assert isinstance(b2, np.ndarray), \"b2 should be a numpy array.\"\n",
    "\n",
    "assert b1.shape[0] == 1, \"The number of rows in b1 is incorrect.\"\n",
    "assert b1.shape[1] == n1, \"The number of columns in b1 is incorrect.\"\n",
    "\n",
    "assert b2.shape[0] == 1, \"The number of rows in b2 is incorrect.\"\n",
    "assert b2.shape[1] == n2, \"The number of columns in b2 is incorrect.\"\n",
    "\n",
    "assert isinstance(W1, np.ndarray), \"W1 should be a numpy array.\"\n",
    "assert isinstance(W2, np.ndarray), \"W2 should be a numpy array.\"\n",
    "\n",
    "assert W1.shape[0] == n0, \"The number of rows in W1 is incorrect.\"\n",
    "assert W1.shape[1] == n1, \"The number of columns in W1 is incorrect.\"\n",
    "\n",
    "assert W2.shape[0] == n1, \"The number of rows in W2 is incorrect.\"\n",
    "assert W2.shape[1] == n2, \"The number of columns in W2 is incorrect.\"\n",
    "\n",
    "assert np.allclose(np.round(W1[0,:], decimals=8), np.array([ 3.70240917, -1.49833461,  2.6521066 ,  0.57524552,  0.56873664, -1.83484853,  1.36708234]))\n",
    "assert np.allclose(np.round(b1, decimals = 8), np.array([-3.53744454, -4.85498228, -1.04975656, -2.78135901,  0.42618399,  2.62152429, -1.46579243]))\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "RnlrjdxEfmaq",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-14c926d99aa1cf95",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "source": [
    "##### Prediction using vectorization\n",
    "\n",
    "Re-implement the prediction function using vectorization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "ok",
     "timestamp": 1632503103801,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "eEpHaKzSfmaq",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-07ee2343ff3cedbe",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "def predict_using_vectorization(W1, W2, b1, b2, X):\n",
    "    '''\n",
    "    Returns the predicted labels 'Y' and class probabilities 'P' for a set of data samples.\n",
    "    X is a matrix specifying the data, where each row corresponds to a single data sample.\n",
    "    Y is a vector of integers specifying the predicted class labels.\n",
    "    P is matrix of probabilities, where each row specifies the class probabilities for a single data sample.\n",
    "    '''\n",
    "    \n",
    "    ### YOUR CODE HERE\n",
    "    raise NotImplementedError()\n",
    "    \n",
    "    return Y, P"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503103994,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "m5Nw859cfmar",
    "nbgrader": {
     "grade": false,
     "grade_id": "cell-aff2ffbdcac1001a",
     "locked": true,
     "schema_version": 1,
     "solution": false
    }
   },
   "outputs": [],
   "source": [
    "Y_pred, P = predict_using_vectorization(W1, W2, b1, b2, X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503103995,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "jxYgmMPOfmar"
   },
   "outputs": [],
   "source": [
    "assert isinstance(Y_pred, np.ndarray) and Y_pred.shape == (X.shape[0],), \"Y should be a numpy array of shape (m,)\"\n",
    "assert isinstance(P, np.ndarray) and P.shape == (X.shape[0], n2), \"P should be a numpy array of shape (m, n2).\"\n",
    "assert np.array_equal(Y_pred[0:5], np.array([2, 0, 2, 1, 1])), \"Incorrect predicted labels.\"\n",
    "assert np.allclose(np.round(P[15,:], decimals = 8), np.array([0., 1., 0.])), \"Incorrect class probabilities (test 1).\"\n",
    "assert np.allclose(np.round(P[25,:], decimals=8), np.array([5.4000000e-07, 6.6609600e-03, 9.9333851e-01])), \"Incorrect class probabilities (test 2).\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "n-Jrfvozfmar"
   },
   "source": [
    "## Part 3\n",
    "\n",
    "In this part you will use the functions you implemented in Parts 1 and 2 to train a classifier on the [MNIST handwritten digit dataset](http://yann.lecun.com/exdb/mnist/).\n",
    "\n",
    "MNIST contains 70,000 grayscale images of handwritten digits (0\u20139), each 28\u00d728 pixels.  Since our NumPy implementation is not GPU-accelerated, we will work with a small subset of **2,000 training images** and **500 test images**.\n",
    "\n",
    "Your goal is to do the following:\n",
    "- Choose a suitable number of hidden units and train a neural network using `training_using_vectorization`.\n",
    "- Evaluate the trained network on the held-out test set using `predict_using_vectorization` and report training and test accuracy.\n",
    "- Visualize a sample of test predictions.\n",
    "\n",
    "We have provided the code below for loading the data.  It has already been preprocessed for you: each image has been flattened into a vector of 784 pixel values scaled to the range [0, 1], and the code below one-hot encodes the labels.\n",
    "\n",
    "You may use as many cells and code blocks as necessary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Reads mnist.npz from next to this notebook if you have it, and otherwise\n",
    "# downloads it from the course site.\n",
    "from pathlib import Path\n",
    "import io, urllib.request\n",
    "\n",
    "MNIST = \"mnist.npz\"\n",
    "if Path(MNIST).exists():\n",
    "    data = np.load(MNIST)\n",
    "else:\n",
    "    url = \"https://course.aiprototypes.org/resources/files/eng190bo-fa26-lab-2-mnist.npz\"\n",
    "    data = np.load(io.BytesIO(urllib.request.urlopen(url).read()))\n",
    "\n",
    "X_train, y_train = data[\"X_train\"], data[\"y_train\"]\n",
    "X_test, y_test   = data[\"X_test\"],  data[\"y_test\"]\n",
    "\n",
    "# one-hot encode\n",
    "Y_train = np.eye(10)[y_train]\n",
    "Y_test = np.eye(10)[y_test]\n",
    "\n",
    "\n",
    "print(f'Training set: {X_train.shape}, {Y_train.shape}')\n",
    "print(f'Test set:     {X_test.shape}, {Y_test.shape}')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plt.imshow(X_train[0].reshape((28,28)), cmap=\"gray\")\n",
    "plt.title(f\"Label: {y_train[0]}\")\n",
    "plt.axis(\"off\")\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 5,
     "status": "ok",
     "timestamp": 1632503103995,
     "user": {
      "displayName": "Alec Vercruysse",
      "photoUrl": "https://lh3.googleusercontent.com/a/default-user=s64",
      "userId": "06702618696541638460"
     },
     "user_tz": 420
    },
    "id": "hoyfA-Tsfmar"
   },
   "outputs": [],
   "source": [
    "### INSERT CODE AND MARKDOWN BELOW ###\n"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "hw4_solns.ipynb",
   "provenance": []
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.5"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}