{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "l4-00",
   "metadata": {},
   "source": [
    "# L4 In-Class Activity: The Quiz, in Code\n",
    "\n",
    "### E190BO — Fall 2026\n",
    "\n",
    "You just did forward prop and backprop **by hand**. Now write the same computation in\n",
    "NumPy:\n",
    "\n",
    "$$\\bar{x} \\;\\longrightarrow\\; \\boxed{\\text{linear}} \\;\\xrightarrow{\\;\\bar{z}^{(1)}\\;}\\;\n",
    "\\boxed{\\text{ReLU}} \\;\\xrightarrow{\\;\\bar{a}^{(1)}\\;}\\; \\boxed{\\text{linear}}\n",
    "\\;\\xrightarrow{\\;\\hat{y}\\;}\\; \\boxed{\\text{SE loss}} \\;\\longrightarrow\\; J$$\n",
    "\n",
    "Work with a partner. Your quiz is the answer key: every number your code prints should\n",
    "match something you wrote on paper."
   ]
  },
  {
   "cell_type": "code",
   "id": "l4-01",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# --- PROVIDED --- the quiz setup. Run this cell first; you do not need to modify it.\n",
    "import numpy as np\n",
    "np.set_printoptions(precision=4, suppress=True)\n",
    "\n",
    "x  = np.array([[1.0, 1.0, -1.0]])          # (1, 3)  input sample, a row vector\n",
    "W1 = np.array([[1.0, 0.0, 0.0],\n",
    "               [0.0, 1.0, 0.0],\n",
    "               [0.0, 0.0, 1.0]])           # (3, 3)\n",
    "b1 = np.array([[0.0, 0.0, 0.0]])           # (1, 3)\n",
    "W2 = np.array([[1.0], [1.0], [-1.0]])      # (3, 1)\n",
    "b2 = 0.0                                   # scalar\n",
    "y  = 3.0                                   # scalar target\n",
    "\n",
    "params = {\"W1\": W1, \"b1\": b1, \"W2\": W2, \"b2\": b2}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l4-02",
   "metadata": {},
   "source": [
    "## TODO 1 — the forward pass (quiz part a)\n",
    "\n",
    "One line per block in the graph. `forward` also returns a **cache** of intermediate values,\n",
    "because `backward` needs them.\n",
    "\n",
    "*Hints:* `@` is matrix multiply; `np.maximum(0, z)` is an elementwise ReLU; `a1 @ W2` has\n",
    "shape $(1, 1)$, so use `.item()` to get a plain number."
   ]
  },
  {
   "cell_type": "code",
   "id": "l4-03",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "def forward(x, params, y):\n",
    "    \"\"\"Forward pass:  x -> linear -> ReLU -> linear -> SE loss -> J.\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    x      : (1, 3) array, the input sample\n",
    "    params : dict with \"W1\" (3, 3), \"b1\" (1, 3), \"W2\" (3, 1), \"b2\" float\n",
    "    y      : float, the target\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    J     : float, the loss (y - y_hat)**2\n",
    "    cache : dict with \"z1\" (1, 3), \"a1\" (1, 3), \"y_hat\" float\n",
    "    \"\"\"\n",
    "    W1, b1, W2, b2 = params[\"W1\"], params[\"b1\"], params[\"W2\"], params[\"b2\"]\n",
    "\n",
    "    # ============ TODO 1 ============\n",
    "    z1    = x @ W1 + b1                 # (1, 3)\n",
    "    a1    = np.maximum(0, z1)           # (1, 3)\n",
    "    y_hat = (a1 @ W2).item() + b2       # (1, 1) -> float\n",
    "    J     = (y - y_hat) ** 2\n",
    "    # ================================\n",
    "\n",
    "    return J, {\"z1\": z1, \"a1\": a1, \"y_hat\": y_hat}\n",
    "\n",
    "\n",
    "J, cache = forward(x, params, y)\n",
    "print(\"z1    =\", cache[\"z1\"])\n",
    "print(\"a1    =\", cache[\"a1\"])\n",
    "print(\"y_hat =\", cache[\"y_hat\"])\n",
    "print(\"J     =\", J)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l4-04",
   "metadata": {},
   "source": [
    "## TODO 2 — the backward pass (quiz part b)\n",
    "\n",
    "Work from $J$ back toward $\\bar{x}$, reusing each gradient in the next line. Two rules:\n",
    "\n",
    "- **A gradient has the same shape as the thing it is taken with respect to.** If you're\n",
    "  unsure whether it's `A.T @ B` or `B.T @ A`, check the shapes.\n",
    "- The ReLU derivative is 1 where $\\bar{z}^{(1)} > 0$ and 0 otherwise."
   ]
  },
  {
   "cell_type": "code",
   "id": "l4-05",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "def backward(x, params, y, cache):\n",
    "    \"\"\"Backward pass: the gradients from quiz part (b).\n",
    "\n",
    "    Parameters\n",
    "    ----------\n",
    "    x, params, y : same as `forward`\n",
    "    cache        : dict returned by `forward`\n",
    "\n",
    "    Returns\n",
    "    -------\n",
    "    grads : dict with\n",
    "        \"y_hat\" : float,        dJ/dy_hat\n",
    "        \"a1\"    : (1, 3) array, dJ/da1\n",
    "        \"z1\"    : (1, 3) array, dJ/dz1\n",
    "        \"W1\"    : (3, 3) array, dJ/dW1\n",
    "        \"b1\"    : (1, 3) array, dJ/db1\n",
    "    \"\"\"\n",
    "    W2 = params[\"W2\"]\n",
    "    z1, y_hat = cache[\"z1\"], cache[\"y_hat\"]\n",
    "\n",
    "    # ============ TODO 2 ============\n",
    "    dJ_dyhat = -2.0 * (y - y_hat)       # float\n",
    "    dJ_da1   = dJ_dyhat * W2.T          # (1, 3)\n",
    "    dJ_dz1   = dJ_da1 * (z1 > 0)        # (1, 3), ReLU mask\n",
    "    dJ_dW1   = x.T @ dJ_dz1             # (3, 1) @ (1, 3) -> (3, 3)\n",
    "    dJ_db1   = dJ_dz1                   # (1, 3)\n",
    "    # ================================\n",
    "\n",
    "    return {\"y_hat\": dJ_dyhat, \"a1\": dJ_da1, \"z1\": dJ_dz1, \"W1\": dJ_dW1, \"b1\": dJ_db1}\n",
    "\n",
    "\n",
    "grads = backward(x, params, y, cache)\n",
    "for name in [\"y_hat\", \"a1\", \"z1\", \"W1\", \"b1\"]:\n",
    "    print(f\"dJ/d{name} =\\n{grads[name]}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l4-06",
   "metadata": {},
   "source": [
    "## Check your work\n",
    "\n",
    "Run the cell below. It checks `forward` and spot-checks your gradients on a **random**\n",
    "network, since the quiz numbers are too tidy to catch every bug."
   ]
  },
  {
   "cell_type": "code",
   "id": "l4-07",
   "metadata": {},
   "execution_count": null,
   "outputs": [],
   "source": [
    "# --- PROVIDED --- run this to check your work. You do not need to modify it.\n",
    "def check_work(eps=1e-6):\n",
    "    rng = np.random.default_rng(2)\n",
    "    xr = rng.normal(size=(1, 3))\n",
    "    pr = {\"W1\": rng.normal(size=(3, 3)), \"b1\": rng.normal(size=(1, 3)),\n",
    "          \"W2\": rng.normal(size=(3, 1)), \"b2\": float(rng.normal())}\n",
    "    yr = float(rng.normal())\n",
    "\n",
    "    Jr, c = forward(xr, pr, yr)\n",
    "    g = backward(xr, pr, yr, c)\n",
    "    if any(v is Ellipsis for v in [Jr, *c.values(), *g.values()]):\n",
    "        print(\"Finish TODO 1 and TODO 2 first.\")\n",
    "        return\n",
    "\n",
    "    ok = True\n",
    "    if not np.isclose(Jr, 0.387988):\n",
    "        print(f\"forward: J = {Jr:.5f} on the random network, expected 0.38799 -- recheck TODO 1.\")\n",
    "        ok = False\n",
    "\n",
    "    for name, idx in [(\"W1\", (0, 2)), (\"W1\", (1, 0)), (\"W1\", (2, 2)), (\"b1\", (0, 1)), (\"b1\", (0, 2))]:\n",
    "        if np.shape(g[name]) != pr[name].shape:\n",
    "            print(f\"dJ/d{name} has shape {np.shape(g[name])}, expected {pr[name].shape}\")\n",
    "            ok = False\n",
    "            continue\n",
    "        plus, minus = pr[name].copy(), pr[name].copy()\n",
    "        plus[idx] += eps\n",
    "        minus[idx] -= eps\n",
    "        numeric = (forward(xr, {**pr, name: plus}, yr)[0]\n",
    "                   - forward(xr, {**pr, name: minus}, yr)[0]) / (2 * eps)\n",
    "        if not np.isclose(g[name][idx], numeric, atol=1e-5):\n",
    "            print(f\"dJ/d{name}{list(idx)}: yours = {g[name][idx]:.5f}, expected {numeric:.5f}\")\n",
    "            ok = False\n",
    "\n",
    "    print(\"Your forward and backward passes are correct!\" if ok else \"\\nNot quite -- see the lines above.\")\n",
    "\n",
    "check_work()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "l4-08",
   "metadata": {},
   "source": [
    "## Carry into Lab 2\n",
    "\n",
    "- Each line of `forward` is one block of the graph; each line of `backward` is one chain-rule step.\n",
    "- A gradient has the shape of its parameter — let the shapes tell you where the `.T` goes.\n",
    "- In Lab 2 you do the same thing for a whole batch at once."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}