{ "cells": [ { "cell_type": "markdown", "id": "6131203a", "metadata": {}, "source": [ "# E190BO Homework 1 — Gradient Descent — SOLUTIONS\n", "\n", "### Fall 2026\n", "\n", "Submit **this notebook**, run top to bottom with all output visible, to Canvas.\n", "If you did any derivations on paper, scan them and attach them as a separate PDF.\n", "\n", "In your one-on-one check-in meeting, we will ask you questions about your notebook to\n", "assess your level of understanding. Your goal is to understand this notebook deeply.\n", "\n", "### How this notebook works\n", "\n", "Almost all of the code is written for you. Everywhere you see\n", "\n", "```\n", "# YOUR CODE HERE (1 line): ...\n", "raise NotImplementedError(\"Replace this line with your code.\")\n", "```\n", "\n", "you delete that `raise` line and write your own line (or two, or three — it always says how\n", "many) in its place. Every one of those lines is a piece of *math* you derived, translated\n", "into numpy. The plotting, the printing, the bookkeeping, and the loops are all provided,\n", "because none of that is what this assignment is about.\n", "\n", "**Everything must be implemented from scratch in numpy.** Do not call\n", "`np.polyfit`, `np.linalg.lstsq`, `sklearn`, or any other library that solves the\n", "regression for you.\n", "\n", "### AI policy for this assignment\n", "\n", "The **course tutor bot** is the permitted AI resource. It will ask you questions\n", "and give you hints, but it will not hand you derivations or working code — that is\n", "deliberate, and it is the point. Other AI assistants are not permitted on this\n", "assignment.\n", "\n", "Fill in the disclosure cell below. An honest \"I was stuck on the Problem 3 gradient\n", "for two hours and the tutor bot walked me through the product rule\" costs you\n", "nothing, and it is exactly the sort of thing worth talking about at our check-in." ] }, { "cell_type": "markdown", "id": "870a3238", "metadata": {}, "source": [ "### Disclosure\n", "\n", "*Roughly how long did this assignment take you?*\n", "\n", "*Where did you get stuck, and what did you use to get unstuck (tutor bot, classmates,\n", "office hours, notes)?*" ] }, { "cell_type": "markdown", "id": "3cb798f0", "metadata": {}, "source": [ "## Setup\n", "\n", "Run this cell first. It loads the exam-score data and standardizes every column\n", "(subtract the mean, divide by the standard deviation) so that each column has mean 0\n", "and standard deviation 1." ] }, { "cell_type": "code", "execution_count": 1, "id": "79624770", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(25, 4) -> ['EXAM1', 'EXAM2', 'EXAM3', 'FINAL']\n" ] }, { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
EXAM1EXAM2EXAM3FINAL
0-0.5020130.042445-0.562121-0.426175
11.1602820.6954470.9993260.974599
20.8278230.9403230.7390850.762361
31.4096261.5117001.6065551.441524
4-0.502013-1.100309-0.995856-0.850652
\n", "
" ], "text/plain": [ " EXAM1 EXAM2 EXAM3 FINAL\n", "0 -0.502013 0.042445 -0.562121 -0.426175\n", "1 1.160282 0.695447 0.999326 0.974599\n", "2 0.827823 0.940323 0.739085 0.762361\n", "3 1.409626 1.511700 1.606555 1.441524\n", "4 -0.502013 -1.100309 -0.995856 -0.850652" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import matplotlib.pyplot as plt\n", "\n", "df = pd.read_csv(\"testScores.csv\").dropna(axis=1, how=\"all\")\n", "z = (df - df.mean()) / df.std(ddof=0) # standardize every column\n", "print(df.shape, \"->\", list(z.columns))\n", "z.head()" ] }, { "cell_type": "markdown", "id": "9f1c398e", "metadata": {}, "source": [ "### Provided: the gradient checker\n", "\n", "You will use this tool in all three problems, so it is worth reading the docstring once now.\n", "\n", "The idea: you derive a gradient by hand and code it up. `grad_check` then estimates the *same*\n", "gradient a completely different way — by nudging one parameter at a time and watching how much\n", "the loss changes. If your derivation and your code are both correct, the two answers agree to\n", "about ten decimal places. If they disagree, something is wrong, and you want to know that\n", "*before* you spend an hour debugging a fit that will not converge.\n", "\n", "Notice the convention used everywhere in this notebook: **every loss function and every\n", "gradient function takes the data first and the parameters last.** So `grad_check` can call\n", "your functions as `loss_fn(X, y, w)` and `grad_fn(X, y, w)` without knowing anything else\n", "about them." ] }, { "cell_type": "code", "execution_count": 2, "id": "95b48a5b", "metadata": {}, "outputs": [], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "def grad_check(loss_fn, grad_fn, w, X, y, eps=1e-5):\n", " \"\"\"Compare your analytical gradient against a numerical estimate of the same thing.\n", "\n", " The definition of a partial derivative says that, for a small step eps,\n", "\n", " de/dw_j is approximately ( e(w + eps in slot j) - e(w - eps in slot j) ) / (2*eps)\n", "\n", " This function computes that estimate for every slot j and compares it to whatever your\n", " gradient function returns.\n", "\n", " Inputs\n", " ------\n", " loss_fn : your loss function. It gets called as loss_fn(X, y, w).\n", " grad_fn : your gradient function. It gets called as grad_fn(X, y, w).\n", " w : the parameter values to check at. Use RANDOM values, not all zeros -- a\n", " wrong gradient can accidentally look correct at a special point like zero.\n", " X, y : the data. These are passed straight through to your two functions.\n", " eps : the size of the small nudge. The default is fine; leave it alone.\n", "\n", " Output\n", " ------\n", " rel_err : one number. Below 1e-7 counts as a pass.\n", " \"\"\"\n", " w = np.array(w, dtype=float)\n", " analytic = np.array(grad_fn(X, y, w), dtype=float)\n", " numeric = np.zeros(w.size)\n", "\n", " for j in range(w.size):\n", " w_plus = w.copy()\n", " w_minus = w.copy()\n", " w_plus[j] = w_plus[j] + eps\n", " w_minus[j] = w_minus[j] - eps\n", " numeric[j] = (loss_fn(X, y, w_plus) - loss_fn(X, y, w_minus)) / (2 * eps)\n", "\n", " size = np.linalg.norm(analytic) + np.linalg.norm(numeric)\n", " if size == 0:\n", " rel_err = 0.0\n", " else:\n", " rel_err = np.linalg.norm(analytic - numeric) / size\n", "\n", " print(\" your gradient:\", np.array2string(analytic, precision=6))\n", " print(\" numerical :\", np.array2string(numeric, precision=6))\n", " if rel_err < 1e-7:\n", " print(f\" relative error = {rel_err:.3e} -> PASS\")\n", " else:\n", " print(f\" relative error = {rel_err:.3e} -> FAIL\")\n", " return rel_err" ] }, { "cell_type": "markdown", "id": "e8ab93b5", "metadata": {}, "source": [ "---\n", "# Problem 1 — Vectorizing the in-class activity\n", "\n", "In class you fit $\\hat{y}_i = a x_i + b$ with an explicit `agrad` and `bgrad`. That works\n", "for two parameters. It does not scale.\n", "\n", "Here you will rewrite the same fit in **matrix form**. Collect the data into a design matrix\n", "$X$ of shape $(N, m)$ where each **row** is one data sample, and let the **last column be all\n", "ones** so that the last weight is the intercept:\n", "\n", "$$\\hat{\\bar y} = X \\bar w, \\qquad e = \\frac{1}{N}(\\bar y - X\\bar w)^T(\\bar y - X\\bar w)$$\n", "\n", "For this 1-D problem, $m = 2$ and $\\bar w = [a, b]$.\n", "\n", "Write `compute_loss` and `compute_grad` for a **general** $m$ — you will reuse them\n", "unchanged in Problem 2, so do not hard-code $m = 2$.\n", "\n", "**(a)** Build the design matrix `X1`.\n", "**(b)** Implement `compute_loss` and `compute_grad`.\n", "**(c)** Verify `compute_grad` with `grad_check`.\n", "**(d)** Implement the gradient descent step, and confirm it reproduces your in-class answer." ] }, { "cell_type": "markdown", "id": "fe801f72", "metadata": {}, "source": [ "### 1(a) Build the design matrix" ] }, { "cell_type": "code", "execution_count": 3, "id": "b0f9b497", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X1 shape: (25, 2) (should be (25, 2))\n", "[[-0.5020131 1. ]\n", " [ 1.16028194 1. ]\n", " [ 0.82782293 1. ]]\n" ] } ], "source": [ "# The data for Problem 1: use EXAM1 to predict FINAL.\n", "x = z[\"EXAM1\"].values # shape (25,) -- the input\n", "y = z[\"FINAL\"].values # shape (25,) -- the target\n", "N = len(y) # 25 data samples\n", "\n", "# YOUR CODE HERE (1 line): build the design matrix X1, which should have shape (N, 2).\n", "# Column 0 is x. Column 1 is all ones -- that is the intercept column.\n", "# Two functions that help: np.column_stack([a, b]) glues 1-D arrays together as columns,\n", "# and np.ones(N) makes an array of N ones.\n", "X1 = np.column_stack([x, np.ones(N)])\n", "\n", "print(\"X1 shape:\", X1.shape, \" (should be (25, 2))\")\n", "print(X1[:3])" ] }, { "cell_type": "markdown", "id": "cd40e97d", "metadata": {}, "source": [ "### 1(b) The loss and its gradient\n", "\n", "Derive $\\partial e / \\partial \\bar w$ in matrix form first (it is on the L2 slides), then\n", "implement it below. In both functions the residual vector\n", "\n", "$$\\bar r = \\bar y - X\\bar w$$\n", "\n", "is computed for you — it is the one quantity both formulas are built out of." ] }, { "cell_type": "code", "execution_count": 4, "id": "9c4e6bc0", "metadata": {}, "outputs": [], "source": [ "def compute_loss(X, y, w):\n", " \"\"\"Mean squared error of the linear model yhat = X @ w.\n", "\n", " Inputs\n", " ------\n", " X : the design matrix, shape (N, m). Each ROW is one data sample, and the LAST\n", " column is the column of ones.\n", " y : the targets, shape (N,).\n", " w : the weights, shape (m,). w[-1] is the intercept.\n", "\n", " Output\n", " ------\n", " A single number: the mean squared error.\n", " \"\"\"\n", " residuals = y - X @ w # shape (N,). X @ w is the prediction yhat.\n", "\n", " # YOUR CODE HERE (1 line): return the MEAN of the squared residuals.\n", " # np.mean(a) averages an array, and a ** 2 squares every entry of it.\n", " return np.mean(residuals ** 2)\n", "\n", "\n", "def compute_grad(X, y, w):\n", " \"\"\"Gradient of compute_loss with respect to w.\n", "\n", " Inputs\n", " ------\n", " X, y, w : exactly the same as in compute_loss above.\n", "\n", " Output\n", " ------\n", " An array of shape (m,) -- one partial derivative per weight. It must have the same\n", " shape as w, NOT a single number.\n", " \"\"\"\n", " N = len(y)\n", " residuals = y - X @ w # shape (N,), same as above\n", "\n", " # YOUR CODE HERE (1 line): return the gradient you derived.\n", " # In numpy, X.T is the transpose of X, and A @ b is matrix-vector multiplication.\n", " # No for loop over the N data samples -- the matrix multiply IS the sum over samples.\n", " return -2.0 / N * (X.T @ residuals)" ] }, { "cell_type": "markdown", "id": "2ee26ec4", "metadata": {}, "source": [ "Some checks. These should all pass before you go any further." ] }, { "cell_type": "code", "execution_count": 5, "id": "d7c99243", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "assertions passed\n" ] } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "assert np.isclose(compute_loss(X1, y, np.array([0.0, 0.0])), np.mean(y ** 2))\n", "assert np.isclose(compute_loss(X1, y, np.array([1.0, 0.0])), np.mean((y - x) ** 2))\n", "assert compute_grad(X1, y, np.array([0.5, 0.1])).shape == (2,)\n", "np.testing.assert_allclose(compute_grad(X1, y, np.array([0.0, 0.0])),\n", " [-2 * np.mean(x * y), -2 * np.mean(y)], atol=1e-12)\n", "print(\"assertions passed\")" ] }, { "cell_type": "markdown", "id": "33ae0c3d", "metadata": {}, "source": [ "### 1(c) Gradient checking\n", "\n", "Nothing to write here — just run it. Check at a **random** $\\bar w$, not at $\\bar w = 0$." ] }, { "cell_type": "code", "execution_count": 6, "id": "cb967a83", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " your gradient: [-1.640681 -0.26421 ]\n", " numerical : [-1.640681 -0.26421 ]\n", " relative error = 8.283e-13 -> PASS\n" ] } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "rng = np.random.default_rng(0)\n", "w_test = rng.normal(size=2)\n", "\n", "err = grad_check(compute_loss, compute_grad, w_test, X1, y)\n", "assert err < 1e-7, \"gradient check failed -- fix compute_grad before continuing\"" ] }, { "cell_type": "markdown", "id": "30b8b084", "metadata": {}, "source": [ "### 1(d) Gradient descent\n", "\n", "One line to write: the update step itself. Everything around it — starting the weights,\n", "recording the loss at every iteration, returning the history for plotting — is done for you.\n", "\n", "You will call this same function in Problem 2 and again in Problem 3. It never needs to\n", "change, because it does not know or care *which* loss and gradient it is being handed." ] }, { "cell_type": "code", "execution_count": 7, "id": "bbf0e911", "metadata": {}, "outputs": [], "source": [ "def gradient_descent(loss_fn, grad_fn, X, y, w0, gamma, numIterations):\n", " \"\"\"Batch gradient descent.\n", "\n", " Inputs\n", " ------\n", " loss_fn : a loss function, called as loss_fn(X, y, w)\n", " grad_fn : a gradient function, called as grad_fn(X, y, w)\n", " X, y : the data, passed straight through to loss_fn and grad_fn\n", " w0 : starting values for the parameters\n", " gamma : the learning rate (the step size)\n", " numIterations : how many steps to take\n", "\n", " Outputs\n", " -------\n", " w : the parameters after the final step\n", " loss_hist : the loss after every step, length numIterations + 1\n", " \"\"\"\n", " w = np.array(w0, dtype=float)\n", " loss_hist = [loss_fn(X, y, w)]\n", "\n", " for i in range(numIterations):\n", " # YOUR CODE HERE (1 line): take one step downhill.\n", " # The new w is the old w, minus gamma times the gradient at the current w.\n", " # The gradient at the current w is grad_fn(X, y, w).\n", " w = w - gamma * grad_fn(X, y, w)\n", "\n", " loss_hist.append(loss_fn(X, y, w))\n", "\n", " return w, np.array(loss_hist)" ] }, { "cell_type": "code", "execution_count": 8, "id": "cf659ec3", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "a = 0.946071, b = 0.000000\n", "matches the in-class scalar result to 1e-8\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAGJCAYAAAAKZg7vAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAALoxJREFUeJzt3Qd0VGX6x/EnPfQOoYUiRYOKCoFlXf6IoIiKoriyHlcRPWKJFcvCUUF0d7GiolnwyCroUUFcxbWLiCCIEkBAjSCwURApIlJCSTLJ/Z/nTWaYJJMpMcl9J/l+zhln5s7NvXfmzYSfb41xHMcRAAAARI1Yty8AAAAAkSHAAQAARBkCHAAAQJQhwAEAAEQZAhwAAECUIcABAABEGQIcAABAlCHAAQAARBkCHAAAQJQhwAGoMUVFRXLiiSfKP/7xj5D73n///RITE+N7XlBQIB07dpR//etf1XyVAGA/AhyASsvNzZXJkyfLOeecI82bNzeBa/bs2RXu/+qrr8q2bdvkpptuivhcCQkJMn78eBP+jh49SqkBqNMIcAAqbc+ePfLAAw/Id999J7179w65/6OPPip/+ctfpEmTJpU639ixY805X3nllUr9PADUFgQ4AJXWtm1b2bFjh/z4448mnAXz1Vdfybp16+TSSy+t9PmaNm0qZ599dtBavrrg0KFDbl8CAJcR4ABUWlJSkqSkpIS174IFCyQxMVH+7//+r9xry5Ytk/T0dElOTpbjjjtOnn322QqPc9ZZZ5n99+7dG/Kc2tSqfel69Ohhjq2B8+KLL5YtW7aUCkN33HGH6V+n76dnz57y2GOPieM4pY6lzcPa9KvvQ/vx6b69evWSDz74wLfP66+/bvZbsmRJuWvR96SvffPNN75tGzZskEsuucQ0P+v19e3bV/773/+W+jkNq95j3njjjdK6dWvp0KGD7/XMzEzp2rWr1KtXT/r16yefffaZnHHGGebmLy8vzzR3d+vWzVy7vt+7777bbI/0fXpt375drrnmGmnXrp3Zr0uXLnLDDTdIfn6+b599+/bJbbfd5vt89fwPP/yw6Q8JoPLif8fPAkDYPv/8cxMItC+bv6+//trUqrVq1cqELY/HY4JGmzZtAh6nT58+Jlzp8c4///wKz1dYWGheX7RokWm2vfXWW+XgwYOycOFCE6I0KOpxLrjgAlm8eLEJIqeccop8+OGHctddd5lw8sQTT5Q6pgbHN954wwSpRo0ayfTp02XUqFGydetWadGihZx33nnSsGFDee2112TQoEGlfnbevHkmCOlnoL799ls5/fTTpX379jJhwgRp0KCB+bmRI0fKf/7zH7noootK/byeUz+jSZMm+WrgZsyYYcLWwIED5fbbb5cffvjB/HyzZs1KhTwNS/o+9frHjRsnJ5xwgvnc9f19//33JqxF8j7Vzz//bAKjBjQ95vHHH28+Mw2xhw8fNmFd7/Vz0O3XXXedpKammnKbOHGiqbl98skng/7OAAjCAYAqkJWVpVVWzgsvvBDw9Q4dOjijRo0qt33kyJFOcnKy8+OPP/q2ZWdnO3FxceZ4Zf38889m+8MPPxz0ep5//nmz37Rp08q9VlRUZO4XLFhg9vn73/9e6vVLLrnEiYmJcTZv3uzbpvslJiaW2rZu3Tqz/emnn/Ztu+yyy5zWrVs7Ho/Ht23Hjh1ObGys88ADD/i2DRkyxDnppJOco0ePlrquP/7xj0737t192/Tz1HP86U9/KnXMvLw8p0WLFk56erpTUFDg2z579myz/6BBg3zbXnrpJXP+zz77rNT7nDlzptl3+fLlEb/PK6+80hxTy72iz/fBBx90GjRo4Hz//felXp8wYYIp361bt5b7WQDhoQkVQI349ddfTc1Q2VoyrfHSWiOtnfHSGqJhw4YFPI73GDqYIRitxWrZsqXcfPPN5V7zTk/y3nvvSVxcnNxyyy2lXtcmVc0y77//fqntQ4cONTV3XieffLI0btxY/ve///m2jR49Wnbv3i2ffvqpb5vWSmktmL6mtPn3k08+Mf0BtVZQ34ve9DPS971p0yZTa+Xv2muvNdfqtWrVKrO/bo+PP9aYcvnll5f7nOfPn28+U60l855Lb2eeeaZ5XWsgI3mf+l601m7EiBGm2beiz1fPq7WDej3+59Xja9kvXbq03M8CCA9NqABqTNl+Zb/88oscOXJEunfvXm5f7YumAauiY/jPEReI9nPTY/iHm7J08IX239JmQn8adryv+/MPmV4aTn777Tffc51SRUfZapPpkCFDzDZ9rM2z2hdPbd682byP++67z9wC0RCozate2r+s7LUr7VPmT99v586dS23TQKgjhbUJtqJzRfI+tdwOHDjgaw6uiJ53/fr1YZ8XQPgIcABqhPad8g86leU9htau1TT/GrCKgql21NcaxTfffNNMOrxr1y5Zvny5/POf//Tt4+3Af+edd1ZY01g2mOkghcrS85100kkybdq0gK/rAINI32e459VBJzpYIhBvoAUQOQIcgBqhzXc5OTmltmnNjAYTrakpa+PGjQGP4z2Gt5asItoE+OWXX5oVHMoOnPDq1KmTfPzxx6YZ078WTkeHel+vDG0qnTNnjhlAoTVfGny8zadKR40qvS5tTqwM77Vpbd7gwYN923UQiA5m0GZP/89Cp3DRGsFQNZfh0HLTJlX/EbWB6Hl1sufKvkcAFaMPHIAaMWDAAPMPvv+0FVrTozVQ2p9KRzh6aejRvnGBrF692oQQPV4wOmpS+1s988wzFdYknXvuuaYvVtl9dHSmnmP48OFSGRpYdGoQbTrVm47W9G8C1alAdJoPnVpER2OWpU2UoWjfM63VfO6550xo83r55ZfL1XRqXzvtU6f7lqVN2JHOKxcbG2tqGd9++23TF6+iz1fPu2LFioBlqaNX/a8bQGSogQPwu2j40X+MdVoJpf+o//TTT+axDiDwrrpw4YUXyoMPPmjmM9NpQ7ymTJli5hjTzu46bYX+o/7000+bKTe0/1RZOg2ITr/hnc6iIldeeaW8+OKLZvmtlStXmuNrUNEaNz2PXo92wtfaq3vuucfUWulqEh999JG89dZbZu4y/478kdCaNZ1vbu7cueacOq9cWTp/25/+9CfTtKkDEbRWTptbNfDo56c1ZsHoNB067Yp+xjoYQcOSvgedN06v27+m7YorrjBTlFx//fVmwIJ+fhpctaZRt2vACjQYIRhtEtbPSqcJ8U5NomFUBy7oNCQ66bJOx6Lz2ul0LldddZWZAkY/D53CRAd26PW60RQO1AphjlYFgIA6depkppgIdMvJySm178knn+xcc8015Y6xZMkSp0+fPmb6iq5du5rpLSZPnlxuGpF9+/aZfWbNmhVWaRw+fNi55557nC5dujgJCQlOSkqKmSJky5Ytvn0OHjzo3H777U67du3MPjqFx6OPPuqbCsNLryUjIyPg+x8zZky57QsXLjQ/o9ORbNu2LeD16XXodBx6XXru9u3bO+eff77z+uuvl5tGJNB0HWr69OnmGpKSkpx+/fqZKUH0szznnHNK7Zefn2+mXunVq5fZt1mzZma/KVOmOPv376/U+9SpX/T6W7VqZY6pZac/q1Oc+H++EydOdLp162bKrmXLlmaqlMcee8xcE4DKidH/uB0iAdQNL730kmRkZJjmUq2hiZRO/PrII4+YEaa/p1N/baYDB7SPmtYABmoyBVA70AcOQI3ROcp0igptPoyUDkbQUZT33nsv4c1vqbCy/w+uzcY6z1zZpbQA1C7UwAFAlNLJgnUJrT//+c+mT+CaNWvk3//+t+mPpoM9tJ8cgNqJQQwAEKV0wl6dw03XKtVaNx35qoM3HnroIcIbUMtRAwcAABBl6AMHAAAQZQhwAAAAUYY+cCGG4+vkpLrETlUsPwMAABCMjizX5f3atWtnVj2pCAEuCA1vZRd5BgAAqG7btm2TDh06VPg6AS4I7+LW+iHqws3VVcun6x7qxJvBkjaqH2VhB8rBHpSFPSiLulMOBw4cMJVH3gxSEQJcEN5mUw1v1RngdDJOPT4Bzl2UhR0oB3tQFvagLOpeOcSE6LpFlQ8AAECUIcABAABEGQIcAABAlCHAAQAARBkCHAAAQJSp9QHunXfekZ49e0r37t1l1qxZbl8OAADA71arpxHxeDwyfvx4Wbx4sTRp0kT69OkjF110kbRo0cLtSwMAAKi0Wl0Dt3LlSunVq5e0b99eGjZsKMOHD5ePPvrI7csCAACovQFu6dKlMmLECLMemE5ot2DBgnL7ZGZmSufOnSU5OVn69+9vQpv/Ulga3rz08fbt22vs+gEAAOpcgDt06JD07t3bhLRA5s2bZ5pIJ0+eLGvWrDH7Dhs2THbv3i3RYvHG3fLJpt8kN8/j9qUAAIAoYXUfOG3y1FtFpk2bJtdee62MHTvWPJ85c6a8++678vzzz8uECRNMzZ1/jZs+7tevX4XHy8vLMzf/9ci8S2forTrc8dp62XekQPp0aysNk6wujlpPy9hxnGora1AO0YbvhD0oi7pTDkVhHjtqE0N+fr6sXr1aJk6c6Num65INHTpUVqxYYZ5rWPvmm29McNNBDO+//77cd999FR5z6tSpMmXKlHLbdeFaXfusOsTGOOZ+1y+/SpOY6jkHwv/S7N+/33w5WZfWPZSDPSgLe1AWdaccDh48WLsD3J49e6SwsFDatGlTars+37Bhg3kcHx8vjz/+uAwePNh86HfffXfQEagaBrVJ1r8GrmPHjtKqVatqW8w+MV6LwCONmzSV1q2bVcs5EB79HdG+llreBDj3UA72oCzsQVnUnXJITk6u3QEuXBdccIG5hSMpKcncytJCqq6CSoyPMfcep/g8cJd+MauzvEE5RBu+E/agLOpGOcSGedyo/VeqZcuWEhcXJ7t27Sq1XZ+npKRItIgvKShPYXFTKgAAQK0NcImJiWZi3kWLFpWq2tTnAwYMkGiREF9cBAWFdJwHAAAS/U2oubm5snnzZt/znJwcWbt2rTRv3lxSU1NNf7UxY8ZI3759zYCFJ5980kw94h2VGg0SYoubUAuogQMAALUhwK1atcoMQPDyDjDQ0DZ79mwZPXq0GSE6adIk2blzp5xyyinywQcflBvYYLP4uJI+cExdAQAAakOAO+OMM8xQ3WBuuukmc4tWCXHFTaj5HppQAQBALe8DV1v4BjEUMYgBAACEhwDnMt80IvSBAwAAYSLAWVIDxyhUAAAQLgKcJYMYCHAAACBcBDhLBjEwjQgAAAgXAS6AzMxMSUtLk/T0dKluCUwjAgAAIkSACyAjI0Oys7MlKytLqhs1cAAAIFIEOJfF+1ZiYB44AAAQHgKcy7w1cEwjAgAAwkWAc9mxJlRq4AAAQHgIcLZMI8JKDAAAIEwEOGuaUKmBAwAA4SHAuSzBN4iBtVABAEB4CHAuYyUGAAAQKQKcyxjEAAAAIkWAcxnTiAAAgEgR4Fx2bCkt+sABAIDwEOBcFl8yCjWfUagAACBMBDhLRqEyjQgAAAgXAS6AzMxMSUtLk/T0dKluLGYPAAAiRYALICMjQ7KzsyUrK0tqahoRauAAAEC4CHAu89XAMYgBAACEiQBnyShUFrMHAADhIsC5LD7WuxYq04gAAIDwEOBclhDPNCIAACAyBDhrphGhBg4AAISHAGfLUlpFRW5fCgAAiBIEOJd5pxEp8BDgAABAeAhwliylxTQiAAAgXAQ4lyUyjQgAAIgQAc5lTCMCAAAiRYCzZCJfT5EjjsNIVAAAEBoBzpJRqKqAqUQAAEAYCHCWjEJVTCUCAADCQYCzqQbOQxMqAAAIjQDnsviSlRhUAZP5AgCAMBDgAsjMzJS0tDRJT0+X6hYTE+MLcSynBQAAwkGACyAjI0Oys7MlKytLaoI3wBUUshoDAAAIjQBnAQIcAACIBAHOorngmEYEAACEgwBnAWrgAABAJAhwFs0Fp6sxAAAAhEKAswA1cAAAIBIEOAsQ4AAAQCQIcBZgEAMAAIgEAc4CxybyZR44AAAQGgHOokEMTCMCAADCQYCzAH3gAABAJAhwNjWhspg9AAAIAwHOAglxxcVQ4GEeOAAAEBoBzqYmVGrgAABAGAhwVo1CpQYOAACERoCzahQq04gAAIDQCHBWjUKlBg4AAIRGgLNAgi/AUQMHAABCI8AFkJmZKWlpaZKeni412YTKSgwAACAcBLgAMjIyJDs7W7KysqRmR6HShAoAAEIjwFnAF+A8NKECAIDQCHAW8DWhUgMHAADCQICzaBBDPoMYAABAGAhwVk3kSxMqAAAIjQBngfiStVBZiQEAAISDAGdRDRxNqAAAIBwEOAuwFioAAIgEAc4CCayFCgAAIkCAswAT+QIAgEgQ4CyqgWMUKgAACAcBzgJxLGYPAAAiQICzqQm1kLVQAQBAaAQ4CzCIAQAARIIAZwGmEQEAAJEgwFmAGjgAABAJApxV04iwFioAAAiNAGcBmlABAEAkCHAWLWZfUEgNHAAACI0AZwGmEQEAAJEgwFmAQQwAACASBDgL0AcOAABEggAXQGZmpqSlpUl6errU9ChUx2E1BgAAEBwBLoCMjAzJzs6WrKwsqQnxJYvZa3YrLCLAAQCA4AhwFvDWwCkPAQ4AAIRAgLNAgl+Ay2cqEQAAEAIBzgLeJlTlKaQJFQAABEeAs0BsTIx4K+E81MABAIAQCHCWSChZjYEmVAAAEAoBzrIAV0ATKgAACIEAZ4nEkn5wNKECAIBQCHCWSIgvLoo8DwvaAwCA4Ahw1jWhEuAAAEBwBDhLJNIHDgAAhIkAZ1kTKjVwAAAgFAKcJRJKBjHk0wcOAACEQICzrAmVeeAAAEAoBDhLMIgBAACEiwBniUT6wAEAgDAR4CzrA1fgYTF7AAAQHAHOsibUPOaBAwAAIRDgbOsDxyhUAAAQAgHOuol8WYkBAAAER4CzBIMYAABAuAhwtk3kW8ggBgAAEBwBzrI+cKzEAAAAQiHAWYKJfAEAQLgIcJZI9M4DxyAGAAAQAgHOEgxiAAAA4SLAWdcHjkEMAAAgOAJcAJmZmZKWlibp6elS4wGOJlQAABACAS6AjIwMyc7OlqysLKn5tVCZyBcAAARHgLNEQjwrMQAAgPAQ4CyRRBMqAAAIEwHOEswDBwAAwkWAs6wJlZUYAABAKAQ4S/gGMbAWKgAACIEAZwmaUAEAQLgIcJZIZBADAAAIEwHOEiylBQAAwkWAs6wPHIMYAABAKAQ46/rAsRYqAAAIjgBnWxMqS2kBAIAQCHCWYDF7AAAQLgKcZaNQCwpZzB4AAFRhgHvkkUfkyJEjvufLly+XvLw83/ODBw/KjTfeGMkhUWYQQ5Ej4iHEAQCAqgpwEydONCHNa/jw4bJ9+3bf88OHD8uzzz4bySFRpglVMZABAABUWYBzHCfoc/z+QQwqnxo4AAAQBH3gLBEfW9yEqugHBwAAgiHAWSImJubYclpMJQIAAIKIlwjNmjVLGjZsaB57PB6ZPXu2tGzZ0jz37x+Hyg1kyC8kwAEAgCoMcKmpqfLcc8/5nqekpMhLL71Ubh9Uvh/cofxC+sABAICqC3A//PBDJLujkgMZaEIFAADB0AfOIknxceY+jz5wAACgqgLcihUr5J133im17cUXX5QuXbpI69atZdy4caUm9kVkqIEDAABVHuAeeOAB+fbbb33Pv/76a7nmmmtk6NChMmHCBHn77bdl6tSpkRwSfnyjUJkHDgAAVFWAW7t2rQwZMsT3fO7cudK/f38zsGH8+PEyffp0ee211yI5JPxQAwcAAKo8wP3222/Spk0b3/MlS5aY5bS80tPTZdu2bZEcEn4IcAAAoMoDnIa3nJwc8zg/P1/WrFkjf/jDH3yv6zxwCQkJkRwSfpK8o1ALC/lcAABA1QS4c8891/R1++yzz8zC9vXr15eBAwf6Xl+/fr0cd9xxkRwSfliJAQAAVPk8cA8++KBcfPHFMmjQILMag67CkJiY6Hv9+eefl7PPPjuSQyJAEyrTiAAAgCoLcLpk1tKlS2X//v0mwMXFFc9b5jV//nxp1KhRJIeEH/rAAQCAKg9wV199dVj7aU0cKt+ESg0cAACosgCnTaadOnWSU089VRzHieRHEYakBJbSAgAAVRzgbrjhBnn11VfNSNSxY8fKX//6V2nevHkkh0AQiSVN0kzkCwAAqmwUamZmpuzYsUPuvvtus+pCx44d5dJLL5UPP/ywVtXI6ftMS0sz89rVJPrAAQCAalnMPikpSS677DJZuHChZGdnS69eveTGG2+Uzp07S25urtQGGRkZ5r1lZWXV6HkJcAAAoFoCXKkfjo2VmJgYU/tWyOSzVTeRr6fo9x8MAADUWhEHuLy8PNMP7qyzzpIePXqYBe2feeYZ2bp1q5laBJXHYvYAAKDKBzFoU6kuYK9933RKEQ1yOjccqgZNqAAAoMoD3MyZMyU1NVW6du1qFrLXWyBvvPFGJIdFuZUYWAsVAABUUYC78sorTZ83VA8m8gUAANUykS+qD02oAACg2kehopoCXCGjUAEAQMUIcBZhGhEAABAOApxFaEIFAADhIMDZWANHEyoAAAiCAGfjYvasxAAAAIIgwFmEJlQAABAOApxFCHAAACAcBDgbV2KgDxwAAAiCAGfjYvaeInEcx+3LAQAAliLAWVgDpxiJCgAAKkKAs3AaEcVIVAAAUBECnIVNqIoABwAAKkKAs0hsbIwkxMWYx3nMBQcAACpAgLNMUnzxZL4EOAAAUBECnGWSE0qmEvEUun0pAADAUgQ4S2vgjhYUuX0pAADAUgQ4yyR5a+AKqIEDAACBEeAsk+ytgWMQAwAAqAABzjLUwAEAgFAIcJahBg4AAIRCgLMMNXAAACAUApxlqIEDAAChEOAsQw0cAAAIhQBnaQ0cKzEAAICKEOAsrYE7yjxwAACgAgQ4yyQnUAMHAACCI8BZJjmeGjgAABAcAc4ySd4aONZCBQAAFSDAWSbJWwPnYS1UAAAQGAHOMtTAAQCAUAhwtvaBowYOAABUgABnaQ0c04gAAICKEOAsrYFjIl8AAFARApxljtXAFbl9KQAAwFIEOGtr4BiFCgAAAiPA2boSAzVwAACgAgQ4S9dCpQYOAABUhABnmeR4+sABAIDgCHCWoQYOAACEQoCztAauoNCRwiLH7csBAAAWIsBZWgOnmMwXAAAEQoCzTFJJDZwiwAEAgEAIcJaJi42RxDjveqhM5gsAAMojwAWQmZkpaWlpkp6eLm6ol1hcC3ck3+PK+QEAgN0IcAFkZGRIdna2ZGVl1XyJiEh9X4CjBg4AAJRHgLOQtwbuMDVwAAAgAAKcheqVLKd1uID1UAEAQHkEOAsda0IlwAEAgPIIcBaqlxhv7glwAAAgEAKcherThAoAAIIgwFmIaUQAAEAwBDirR6HSBw4AAJRHgLO4CfUIo1ABAEAABDirm1CpgQMAAOUR4CxEEyoAAAiGAGdzEyo1cAAAIAACnIXqe+eBow8cAAAIgABnoWTWQgUAAEEQ4CxEEyoAAAiGAGfxWqjMAwcAAAIhwNk8jQh94AAAQAAEOAsxDxwAAAiGAGeh+gnFo1BpQgUAAIEQ4CxvQi0qcty+HAAAYBkCnMWDGFSep8jVawEAAPYhwFmoXslKDOpQvsfVawEAAPYhwFkoNjZGGpTUwuUeJcABAIDSCHCWaphcPJAhN48ABwAASiPAWaphUnGAO0gNHAAAKIMAZ6lGyQnmnho4AABQFgHOUo18TagFbl8KAACwDAHOUjShAgCAihDgLEWAAwAAFSHAWYo+cAAAoCIEONunEWEUKgAAKIMAZ6lGvmlEGMQAAABKI8BZiol8AQBARQhwlk8jwkS+AACgLAKc5aNQmcgXAACURYCzFDVwAACgIgQ4SzVMYiktAAAQGAHO9qW0mEYEAACUQYCzfBRqfmGR5HkK3b4cAABgEQKcpRomxktsTPHj/UeYCw4AABxDgLNUbGyMNKlX3A9u32ECHAAAOIYAZ7FmDRLN/d5D+W5fCgAAsAgBzmLN6xcHuN8IcAAAwA8BzmJNvQGOJlQAAOCHAGex5g2K+8D9dpgmVAAAcAwBzmL0gQMAAIEQ4CzWzNeESg0cAAA4hgBnMQYxAACAQAhw0dCEyiAGAADghwBnsWb1vRP50oQKAACOIcBZjEEMAAAgEAJcFPSBO3jUIwWFRW5fDgAAsAQBzmK6Fmp8yYr2e3Lz3L4cAABgCQKc5Qvat2mcbB7/vO+o25cDAAAsQYCzXNsmxQFu534CHAAAKEaAs1xKSYDbsf+I25cCAAAsQYCzXLum9cw9NXAAAMCLAGe5lJI+cDtoQgUAACUIcFHSB44mVAAA4EWAi5I+cDShAgAALwKc5do2Ke4Dt+tgnhQWOW5fDgAAsAABznKtGiVJUnysCW8//XbY7csBAAAWIMBZLi42Rrq2amgeb9qV6/blAAAACxDgokC31sUBbvMvBDgAAECAiwrdSmrgNu8mwAEAAAJcVOjehgAHAACOoQk1mppQd+eK4zASFQCAuo4AFwU6t2hgRqLm5nlkyy+H3L4cAADgMgJcFEiMj5VTU5uaxytz9rp9OQAAwGUEuCjRr0sLc78y51e3LwUAALiMABcl+ndpbu6/zNlLPzgAAOo4AlyUOC21mSQnxMqO/Udl3U/73b4cAADgIgJclKiXGCfDeqWYx/9Z/ZPblwMAAFxEgIsio07rYO7fWrtdDhwtcPtyAACASwhwUeT0bi3luFYN5MBRj2R+stntywEAAC4hwEXZwvb3nHeCeTxrWY58smGX25cEAABcQICLMoN7tpY/9+kghUWOjHtxtUxftEn2H6E5FQCAuiTe7QtAZGJiYuSfF58kRwoK5Z31O2Tawu/lyY+/l5PaN5ET2jaW1o2TpXFyvFm5QScAToqPk9jYmGM/7zuO3zFLtpbeVn6/2q6oyJH9B/ZLk91FpT4zUA51Fd8Je1AWdpXDwMRG0qF5A1evhQAXhRLiYuXpy06VoSe0kRmfbpGNuw6aqUWYXgQAgOr3VAMCXI246KKL5NNPP5UhQ4bI66+/LrWlJm7kqe3N7ed9R8wSW1v3HpZdB47KoTyP5HmKSm6FUlRU/DOOOMX3xXcl28o+CLxfXaBvt6AgXxISEn01kKAc6jK+E/agLOwqh2YNEt2+lLpRA3frrbfK1VdfLXPmzJHaqF3TeibI4fcpKiqS3bt3S+vWrSU2lu6hbqEc7EFZ2IOysK0cWrp9KXVjEMMZZ5whjRo1cvsyAAAAakeAW7p0qYwYMULatWtnmgUXLFhQbp/MzEzp3LmzJCcnS//+/WXlypWuXCsAAIANXA9whw4dkt69e5uQFsi8efNk/PjxMnnyZFmzZo3Zd9iwYaYK0+uUU06RE088sdzt559/rsF3AgAAUEf6wA0fPtzcKjJt2jS59tprZezYseb5zJkz5d1335Xnn39eJkyYYLatXbu2Sq4lLy/P3LwOHDjga/PWW3XQ4zqOU23HB2URbfhO2IOysAdlUXfKoSjMY7se4ILJz8+X1atXy8SJE33btHP50KFDZcWKFVV+vqlTp8qUKVPKbf/ll1/k6NGjUl0FtX//fvMLQcd5d1EWdqAc7EFZ2IOyqDvlcPDgwegPcHv27JHCwkJp06ZNqe36fMOGDWEfRwPfunXrTHNthw4dZP78+TJgwIBy+2lQ1OZa/xq4jh07SqtWraRx48ZSXb8M2vdPz0GAcxdlYQfKwR6UhT0oi7pTDsnJydEf4KrKxx9/HNZ+SUlJ5laWFlJ1hiv9Zajuc4CyiCZ8J+xBWdiDsqgb5RAb5nGtTgwtW7aUuLg42bWr9KLt+jwlJcW16wIAAHCT1QEuMTFR+vTpI4sWLSpVfanPAzWBAgAA1AWuN6Hm5ubK5s2bfc9zcnLMqNLmzZtLamqq6ZM2ZswY6du3r/Tr10+efPJJ05fNOyoVAACgrnE9wK1atUoGDx7se+4dRKChbfbs2TJ69GgzCnTSpEmyc+dOM+fbBx98UG5gAwAAQF0R4+hYWASkQ4WbNm0q27Ztq9ZRqBpQGYXqPsrCDpSDPSgLe1AWdaccDpTMgLFv3z5p0qSJvTVwNvPOxaIfJAAAQE1mkGABjhq4EElbl+Nq1KiRGTZcnUm7Omv5QFlEE74T9qAs7EFZ1J1ycBzHhDddIz5YLR81cEHoB6cT/9YE/UUgwNmBsrAD5WAPysIelEXdKIcmQWreomIaEQAAAJRHgAMAAIgyBDiX6dJdkydPDriEFyiLuojvhD0oC3tQFnZIsujfbAYxAAAARBlq4AAAAKIMAQ4AACDKEOAAAACiDAEOAAAgyhDgXJaZmSmdO3eW5ORk6d+/v6xcudLtS6pVli5dKiNGjDAzWutqGgsWLCg34/WkSZOkbdu2Uq9ePRk6dKhs2rSp1D579+6Vyy+/3EzaqGvjXnPNNZKbm1vD7yS6TZ06VdLT082qJq1bt5aRI0fKxo0bS+1z9OhRycjIkBYtWkjDhg1l1KhRsmvXrlL7bN26Vc477zypX7++Oc5dd90lHo+nht9NdJsxY4acfPLJvolIBwwYIO+//77vdcrBHQ899JD5G3XbbbdRFjXs/vvvN5+9/+3444+3vhwIcC6aN2+ejB8/3gxJXrNmjfTu3VuGDRsmu3fvdvOyapVDhw6Zz1WDciCPPPKITJ8+XWbOnClffvmlNGjQwJSBfmG9NLx9++23snDhQnnnnXdMKBw3blwNvovot2TJEvMH8IsvvjCfY0FBgZx99tmmfLxuv/12efvtt2X+/Plmf13G7uKLL/a9XlhYaP5A5ufny+effy5z5syR2bNnmwCO8OnqMhoWVq9eLatWrZIzzzxTLrzwQvM7Tjm4IysrS5599lkTrP3xnag5vXr1kh07dvhuy5Yts78cHLimX79+TkZGhu95YWGh065dO2fq1KmUSjXQX/c333zT97yoqMhJSUlxHn30Ud+2ffv2OUlJSc6rr75qnmdnZ5ufy8rK8u3z/vvvOzExMc727dspp0ravXu3+VyXLFni+9wTEhKc+fPn+/b57rvvzD4rVqwwz9977z0nNjbW2blzp2+fGTNmOI0bN3by8vIoi9+hWbNmzqxZsygHFxw8eNDp3r27s3DhQmfQoEHOrbfearbznag5kydPdnr37h3wNZvLgRo4l2hS1/8D1iY7/7VX9fmKFSvcuqw6JScnR3bu3FmqDHT9OW3K9paB3muzad++fX376P5aVlpjh8rZv3+/uW/evLm51++C1sr5l4U2YaSmppYqi5NOOknatGnj20drS3VxaW/tESKjNQdz5841NaHalEo51DytmdbaG//ffUVZ1KxNmzaZrjZdu3Y1rS7aJGp7ObCYvUv27Nlj/nj6F7jS5xs2bHDrsuoUDW8qUBl4X9N77c/gLz4+3gQP7z6ITFFRkennc/rpp8uJJ57o+5wTExNNWA5WFoHKyr8sEZ6vv/7aBDbtKqB9et58801JS0uTtWvXUg41SMOzdp/RJtSy+E7UnP79+5smz549e5rm0ylTpsjAgQPlm2++sbocCHAAarzGQf8w+vcxQc3Sf6g0rGlN6Ouvvy5jxowxfXtQc7Zt2ya33nqr6ROqg9jgnuHDh/seaz9EDXSdOnWS1157zQxusxVNqC5p2bKlxMXFlRvJos9TUlLcuqw6xfs5BysDvS87qERHFunIVMopcjfddJMZCLJ48WLTmd6/LLRbwb59+4KWRaCy8i9LhEdrFLp16yZ9+vQxI4R1oM9TTz1FOdQgbZrTvy2nnXaaqdXXm4ZoHVSlj7UGh++EO5o2bSo9evSQzZs3W/2dIMC5+AdU/3guWrSoVNOSPtemDVS/Ll26mC+XfxlonwXt2+YtA73XL67+sfX65JNPTFnp/6UhPDqGRMObNtXp56efvT/9LiQkJJQqC51mRPuh+JeFNv35B2qtvdCpMLT5D5Wnv895eXmUQw0aMmSI+X3WmlDvTfvaav8r72O+E+7Izc2VLVu2mOmlrP7bVG3DIxDS3LlzzYjH2bNnm9GO48aNc5o2bVpqJAt+/wivr776ytz0133atGnm8Y8//mhef+ihh8xn/tZbbznr1693LrzwQqdLly7OkSNHfMc455xznFNPPdX58ssvnWXLlpkRY5dddhlFE4EbbrjBadKkifPpp586O3bs8N0OHz7s2+f66693UlNTnU8++cRZtWqVM2DAAHPz8ng8zoknnuicffbZztq1a50PPvjAadWqlTNx4kTKIgITJkwwo39zcnLM77w+11HVH330EeXgMv9RqIrvRM244447zN8m/U4sX77cGTp0qNOyZUszWt7mciDAuezpp582vxiJiYlmWpEvvvjC7UuqVRYvXmyCW9nbmDFjfFOJ3HfffU6bNm1MmB4yZIizcePGUsf49ddfTWBr2LChGRY+duxYEwwRvkBloLcXXnjBt4+G5htvvNFMaVG/fn3noosuMiHP3w8//OAMHz7cqVevnvkDq394CwoKKIoIXH311U6nTp3M3xz9R0Z/573hjXKwK8DxnagZo0ePdtq2bWu+E+3btzfPN2/ebH05xOh/qq9+DwAAAFWNPnAAAABRhgAHAAAQZQhwAAAAUYYABwAAEGUIcAAAAFGGAAcAABBlCHAAAABRhgAHAAAQZQhwAOqMM844Q2677TaxSUxMjCxYsMDtywAQZViJAUCdsXfvXrMwdaNGjaRz584mzNVUoLv//vtNUNOFyv3t3LlTmjVrJklJSTVyHQBqh3i3LwAAakrz5s2r/Jj5+fmSmJhY6Z9PSUmp0usBUDfQhAqgzjWh6v2PP/4ot99+u2nC1JvXsmXLZODAgVKvXj3p2LGj3HLLLXLo0CHf61pz9+CDD8qVV14pjRs3lnHjxpntf/vb36RHjx5Sv3596dq1q9x3331SUFBgXps9e7ZMmTJF1q1b5zufbgvUhPr111/LmWeeac7fokULc/zc3Fzf61dddZWMHDlSHnvsMWnbtq3ZJyMjw3cuAHUDAQ5AnfPGG29Ihw4d5IEHHpAdO3aYm9qyZYucc845MmrUKFm/fr3MmzfPBLqbbrqp1M9reOrdu7d89dVXJqgpbZbVUJadnS1PPfWUPPfcc/LEE0+Y10aPHi133HGH9OrVy3c+3VaWBsVhw4aZJtWsrCyZP3++fPzxx+XOv3jxYnOtej9nzhxzXm8gBFA30IQKoE42pcbFxZnQ5d+EOXXqVLn88st9/eK6d+8u06dPl0GDBsmMGTMkOTnZbNcaMg1k/u69995StXR33nmnzJ07V+6++25Tm9awYUOJj48P2mT6yiuvyNGjR+XFF1+UBg0amG3PPPOMjBgxQh5++GFp06aN2aYBT7frezj++OPlvPPOk0WLFsm1115bxZ8UAFsR4ACghDZxas3byy+/7PtMHMeRoqIiycnJkRNOOMFs69u3b7nPTGvrNOxpzZg2eXo8HtPEGonvvvvO1Ox5w5s6/fTTzfk3btzoC3Bak6fhzUubUrXpFUDdQYADgBIavK677jrT762s1NRU32P/gKVWrFhhau60n5s2gTZp0sTUvj3++OPV8tnqSFp/2o9OQx6AuoMAB6BO0pGjhYWFpbaddtpppg9bt27dIjrW559/Lp06dZJ77rnHt00HSYQ6X1law6d92bQvnDckLl++XGJjY6Vnz54RXROA2o1BDADqJO2ntnTpUtm+fbvs2bPHN5JUw5gOGtD52jZt2iRvvfVWuUEEZWlfua1bt5paN21C1abUN998s9z5tBlWj6vny8vLK3ccrcXTfnZjxoyRb775xgxSuPnmm+WKK67wNZ8CgCLAAaiTdATqDz/8IMcdd5y0atXKbDv55JNlyZIl8v3335upRE499VSZNGmStGvXLuixLrjgAjMliQa9U045xYRA7+hULx3ZqiNcBw8ebM736quvljuOTkHy4YcfmgmH09PT5ZJLLpEhQ4aYAQsA4I+VGAAAAKIMNXAAAABRhgAHAAAQZQhwAAAAUYYABwAAEGUIcAAAAFGGAAcAABBlCHAAAABRhgAHAAAQZQhwAAAAUYYABwAAEGUIcAAAABJd/h9Wnl39REziWQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "w1, hist1 = gradient_descent(compute_loss, compute_grad, X1, y,\n", " w0=np.zeros(2), gamma=0.1, numIterations=500)\n", "print(f\"a = {w1[0]:.6f}, b = {w1[1]:.6f}\")\n", "\n", "# In class you should have found a = 0.946071, b = 0.000000\n", "np.testing.assert_allclose(w1, [0.9460708318102029, 0.0], atol=1e-8)\n", "print(\"matches the in-class scalar result to 1e-8\")\n", "\n", "plt.figure(figsize=(7, 4))\n", "plt.semilogy(hist1)\n", "plt.xlabel(\"iteration\"); plt.ylabel(\"MSE\")\n", "plt.title(\"1(d) convergence\"); plt.grid(alpha=.3); plt.show()" ] }, { "cell_type": "markdown", "id": "442b78b2", "metadata": {}, "source": [ "---\n", "# Problem 2 — $m$-dimensional linear regression\n", "\n", "Now use **all three** standardized midterm scores to predict the standardized final:\n", "\n", "$$\\hat y_i = w_1 x_{i,1} + w_2 x_{i,2} + w_3 x_{i,3} + b$$\n", "\n", "Four weights instead of two. Your Problem 1 code should work unchanged — that was the whole\n", "point of writing it for a general $m$." ] }, { "cell_type": "markdown", "id": "a88ac4af", "metadata": {}, "source": [ "## 2(a) Ordinary least squares\n", "\n", "One line to write: the design matrix, exactly like 1(a) but with three input columns instead\n", "of one. Then run the two provided cells.\n", "\n", "Afterwards, answer in the markdown cell below: **which midterm carries the most weight, and\n", "why is it meaningful to compare these coefficients directly here when usually it is not?**" ] }, { "cell_type": "code", "execution_count": 9, "id": "d4930e5f", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "X shape: (25, 4) (should be (25, 4))\n" ] } ], "source": [ "# YOUR CODE HERE (1 line): build the design matrix X, of shape (N, 4).\n", "# Columns 0, 1, 2 are EXAM1, EXAM2, EXAM3. Column 3 is all ones (the intercept).\n", "# The three input columns are z[\"EXAM1\"].values, z[\"EXAM2\"].values, z[\"EXAM3\"].values.\n", "# Same tools as 1(a): np.column_stack([...]) and np.ones(N).\n", "X = np.column_stack([z[\"EXAM1\"].values, z[\"EXAM2\"].values, z[\"EXAM3\"].values, np.ones(N)])\n", "\n", "print(\"X shape:\", X.shape, \" (should be (25, 4))\")" ] }, { "cell_type": "code", "execution_count": 10, "id": "506c8413", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " your gradient: [ 0.870169 0.966624 0.724023 -2.606314]\n", " numerical : [ 0.870169 0.966624 0.724023 -2.606314]\n", " relative error = 5.676e-12 -> PASS\n", "\n", "weights [EXAM1, EXAM2, EXAM3, intercept] = [0.181782 0.282127 0.571263 0. ]\n", "final MSE = 0.010338\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAGJCAYAAAAKZg7vAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAN6tJREFUeJzt3Ql4FPX9x/Fvbu77NtwiGpA7UFTkVERFUVupUjm0oBKsFi1KLSD6r3hUpNUUfVTEq4hYRKsVFUSRQ4koIEZQkEvu+76SzP/5/mS3u5tNmMUkM5N5v55n2ezuZHd+89vNfvhdE2dZliUAAADwjHindwAAAACxIcABAAB4DAEOAADAYwhwAAAAHkOAAwAA8BgCHAAAgMcQ4AAAADyGAAcAAOAxBDgAAACPIcABKFReXp60bNlS/vrXv8Z8pGbPni0VKlSQnTt3cpQBoAgR4ACfycrKkhEjRkiLFi2kfPny0qBBA7n++uvl+++/j7r9tGnTZNOmTeZ3YnXZZZfJ2WefLRMmTIjp9xYuXCjXXHON1K5dW1JSUqRRo0Zy6623ysaNG/Nt+8ADD0hcXJzs2rWr0Odcv369DBkyRJo2bSplypSROnXqyMUXXyzjxo2LuVwA4LQ4zoUK+Muvf/1rE5B+85vfSKtWrWTbtm3y9NNPy6FDh+Tzzz83rW2h2rRpI506dZJnn332jF5v8uTJcs8995jXqVix4mm3f+qpp+TOO++UJk2ayODBg6Vu3bry3XffyfPPP28e/+9//ysXXHBBWIAbP368aeWrUaNG1Odcs2aNpKenS9myZeXmm282gXDr1q3y1Vdfyfvvvy/Hjh07o7IBgGM0wAHwj4ULF1rHjx8Pu+/777+3UlJSrAEDBoTd/9VXX1n6Z2LOnDln/Hrbt2+3EhISrBdeeOG02y5YsMCKj4+3unTpYh0+fDjssTVr1li1a9e26tata+3Zsyd4/7hx48w+7ty5s8DnHT58uJWYmGitX78+6v75WeRxBuANdKECPqOtV8nJyWH3NWvWzHSpaktXqFmzZplttasx1IYNG2T48OHSvHlz06pVvXp106Kn3ZSRatWqZVr63n777dPu20MPPWS6Q1966SUpV65c2GPa9fnYY4+ZlrNYWwPXrl0rqamp0rBhw6j7Z8eqVatMV3PNmjVNmbXs999/f9g2X3/9tfTp00cqVapkxv717NnTtGqGmjp1qimjtoKOHDnSPJ92ZWuXcehYwSuvvNK0QkbTuXNn6dChQ9h9r776qrRv397sW7Vq1eS3v/2t6foO1a1bN9PCunTpUlOneoz//Oc/m8d2794tN910k9n3KlWqyKBBg2T58uVmX3WfI4+FtuTq62h3tO7LO++8c0blDNCW0K5du5pWWt0HbTH917/+FbbNF198YbrlK1eubPZdt9fnB/yIAAdAW+Jl+/bt+bogFy1aZL7wk5KS8o2j08c0JPzjH/+Q2267TebOnWsCwpEjR/IdUQ0Wun1h9Pf0Obp06SKNGzeOuk3//v3NmLh33303plrT4KZh5uOPPz6j2l6xYoXpRtbfHzp0qPz973+Xfv36yX/+85/gNt9++63Zdw09o0aNkjFjxsi6devMMdHgEemOO+4w2+oYvNtvv908V+g4Qy2r/r4e68jwrKFQj32ATjAZOHCgCeITJ06Uu+66yxxLDWn79u0L+30NahoytWt80qRJ0r17dzNRpW/fvma8owY3fT4NyvpzJC3nr371KxP277vvPnniiSdMMNPj8dZbb8VczkDYu+KKK2TPnj0yevRoeeSRR8z+6SSYAD32Wp4DBw6Y53r44YdN2Xr06CFLliyxUYtAKeN0EyAA573yyiumGzKymzM1NdW67rrr8m1/5MiRfPctXrzYPMfLL7+c77GHH37YPFZYd+WyZcvMNnfeeWeh+9qqVSurWrVqMXWhrly50ipbtqzZrk2bNuY1Zs2aZbv78OKLL7YqVqxobdiwIez+vLy84M/9+vWzkpOTrbVr1wbv27Jli/k9/f2AF1980exHr169wn7/j3/8o+lq3rdvn7m9f/9+06199913h73mY489ZsXFxQX3RbuF9ff++te/hm33zTffmG7j0Pu7du1qXvuZZ54J2/bf//63uX/SpEnB+3Jzc60ePXqY+3WfA3r27Gmdf/751rFjx8KOwwUXXGA1a9Ys5nLqtR6jTp06WUePHo16fPVan7t3795hz6Xvw8aNG1uXXHJJ2O8BfkALHOBz2h2WkZFhuuUiW1y0taZq1ar5fke76QJOnjxpttPZptr1phMDIgWeo7CZogcPHjTXp5vooI9rK0wstHt42bJl8rvf/c508wZa0HSW63PPPVfo72p33/z5883kB52xG0q7CFVubq58+OGH5jlDuz11AsaNN94oCxYsyLfPw4YNC/6+0tY7fR5tYVPajagtZW+88YZpIQ2YPn26aQEL7MvMmTNNC5p27+rxDVx0lq22yM2bNy/sdbUFU2fjhtKWLm1l1dbFgPj4ePO+CKUtZNoSpq+l9RV4La3/3r17yw8//CCbN2+OqZwfffSReS5tzdPu2GjHV+tOn1uPpb5W4HUPHz5suqm1fvQYAH6S6PQOAHCOzgzVrisdU/Tmm29KQkJCvm1Cw0PA0aNHzdIgL774ovnCDt1m//79BT5H6Bd5pEBwCwS5gujjdmazRjrnnHPklVdeMeEhOzvbdMPqmDoNGNpl26tXr6i/9+OPP5rryNm5kSFPu4B1XFyk8847z4QL7cLVIBkQGQYDIXfv3r1h3ag6DnHx4sVm7KKO5dPxa9r1GaDBRo+vhrVoIru/zzrrrHxjIDVMadiMHHeooTxyNq++lnYP6yWaHTt2mNewW04t0+mOr5ZRRevSDX3fRfvPBlBaEeAAn9IvPG3h0XFEn332mdSrVy/fNjo5ITRQhI5r0vCmY6205U4DoIYzHZcVrSUk8BwFLfMRCAuJiYlmvFlBjh8/LqtXr843gD8WGlLPP/98c9F91zFgr732WoEBrrhEC8sqNAzruDQNVdoKpwFOr7VlTCeMBOjx1mOvkwCiPadOpiio9TRWgbrVZWG0xS2ayNBnp5x2X/fxxx83Y+OiiSwnUNoR4AAf0nXPNBzo4r1z5syRtLS0qNude+65ZiB9JG2t09YQHcAe+pyRA+YD9Dk0vOlMxILoQHgNU9pFpy1C0WaMaoDREKczNItCIAjqgP2CBLpEV65cWeA2Wi4NWhouo3VRa+iqX79+zPunx0TLOmPGDDM5QbtPtQsyNGzr7FwNQ9qKqK2MZ0KPtXa1aitiaCuctrhFOxbaqldUgVf3P3B8I8Nf5DbarVzSQRtwK8bAAT6jXYjaNafdchoMtBWqIPqYfrFqaIpsVYlsQdEFePW5o9Fuv8JeJ+Avf/mLeV5dwFe7aSNDoM7u1K4+PStDLLSFUcfqRdJFgVW0rs/QcKazH6dMmZLvTBCBY6DH49JLLzVLpYQupaIze3UpjIsuusiEjzOhdbVlyxazkLHO5tTboa699lrz+rqYcWSd6G0dM3Y62pqmxyd0PKC2emVmZuZbckVn1eoyLtFC75mcMk2Pm3aJa5d85ILKgfLoLGYNcX/729/MgtNF8bqA19ECB/jM3Xffbdbs0hY4HZSu64eF0oH+AVdffbVZm+3TTz81X7QB2iqk48m061Rb7zQMakuedrlGGxOl3aKRA+Kj0aCkX9K6bpiuHRc4E4O2Ymm40FChoSvaWCdtoYocw6UtX7rO2aOPPmpCpIYdfV6lky1efvlls5aZdgUXRpdK0RDWrl274Jg5DWrvvfeeGWCv/u///s8MyNftdI087Q7WoKPhV8fananLL7/cBBztttSgdt1114U9rsFGX1uX39B90okUur0GXl3WQ/dXf7cw+jsdO3Y07w1tddOWV32P6Psjcuyihjoto3ZB66QHbZXToKrvgZ9++smEzFhosH3yySfl97//vVn7TScqaP3q82iLoK4JqPWoAVa7/HUcoU7C0HF2Ov5SWw71OUKXdAF8welpsABKVmApiYIu0ZbtuOWWW8Lu27t3rzVkyBCrRo0aVoUKFczyDqtWrbIaNmxoDRo0KGzbyZMnW+XKlbMOHDhgex/nz59vXX311eb5k5KSrAYNGlhDhw6NeiaFwDIi0S66XEXg7BMZGRlWy5YtrcqVKwefc/DgwWHLfhRGlyK55pprrCpVqlhlypSxmjdvbo0ZMybfmSv0WOgx0TJ3797dWrRoUdg2geU1srKywu6fN2+euV+vI+kZMgJLchRElwK56KKLrPLly5vLueeea8q8evXqsLpv0aJF1N/XZVhuvPFGs6SHHiM9Nnrc9HVff/31sG31mA0cONCqU6eOOZZnnXWWdeWVV1pvvvnmGZfznXfeMUuR6HIvlSpVsjp27GhNmzYtbJuvv/7auvbaa63q1aubJVb0/Xb99ddbc+fOLfC4AKUV50IFUChtadPWM+0+1GVCYtW2bVvT7aatLPAWnQGrZ07QZVAuvPBCp3cHQAjGwAEo1IABA8xSEJHjoezQ9cV0CQjt3oO7RY451PGMOq5Ruye16xiAu9ACBwAwY9A0xOlkEx23pwsE6+nP9JRVBHDAfQhwAAAzW1aXhdFJDDobVJf00HOXRp63FIA7EOAAAAA8hjFwAAAAHkOAAwAA8BgW8i2ELhqqK6DropiFnYQbAACgKOgZSA4ePGhOmaeLWBeEAFcIDW9ncv5CAACAX2LTpk2Smppa4OMEuEJoy1vgIJ7peQzttPLpefz0fIuFJe3Sxo/l9mOZFeX2T31T1/6pa7/Wd14JlPnAgQOm8SiQQQpCgCtEoNtUw1txBjidsq/P75cPgF/L7ccyK8rtn/qmrv1T136t77wSLPPphm7544gDAACUIgQ4AAAAjyHAAQAAeAwBDgAAwGMIcAAAAB5T6gPcu+++K82bN5dmzZrJ888/7/TuAAAA/GKlehmRnJwcGTlypMybN08qV64s7du3l2uuuUaqV6/u9K4BAACcsVLdArdkyRJp0aKFnHXWWVKhQgXp06ePfPjhh07vFgAAQOkNcPPnz5e+ffua84HpgnazZs3Kt01mZqY0atRIypQpI506dTKhLfRUWBreAvTnzZs3l9j+AwAA+C7AHT58WFq3bm1CWjTTp083XaTjxo2Tr776ymzbu3dv2bFjh3jF3FU75OMf9srBYyed3hUAAOARrh4Dp12eeinIxIkTZejQoTJkyBBz+5lnnpH33ntPpkyZIvfdd59puQttcdOfO3bsWODzHT9+3FxCz0cWOHWGXorDn2askH1HT0q7pnWlYpkk8Qs9npZlFdtxdSM/lllRbv/UN3Xtn7r2a33nlUCZ7T63qwNcYU6cOCFLly6V0aNHB+/T85L16tVLFi9ebG5rWFu5cqUJbjqJ4f3335cxY8YU+JwTJkyQ8ePH57tfT1yr5z4rDnnWzxW1e/du2RFfPK/hRvoG3b9/v/kg+Okcen4rs6Lc/qlv6to/de3X+s4rgTIfPHiwdAe4Xbt2SW5urtSuXTvsfr29atUq83NiYqI88cQT0r17d3PQR40aVegMVA2D2iUb2gJXv359qVmzZrGdzD7BvAFypWq16lKrVvG8hhtpfei4Rj22fvrg+63MinL7p76pa//UtV/rO68Eyqxj+kt1gLPrqquuMhc7UlJSzCWSVlJxVVRc4DouzjcfAIkos5/K7ccyK8rtn/qmrv1T136t77hiLrPd5/XsEa9Ro4YkJCTI9u3bw+7X23Xq1BEvvREAAABi4dkAl5ycbBbmnTt3bljTpt7u3LmzeI3l9A4AAADPcHUX6qFDh2TNmjXB2+vWrZNly5ZJtWrVpEGDBma82qBBg6RDhw5mwsKkSZPM0iOBWaleogMiAQAAPB/gvvzySzMBISAwwUBD29SpU6V///5mhujYsWNl27Zt0qZNG5k9e3a+iQ2x0nXn9KKTJIobPagAAKBUBbhu3bqdtmVqxIgR5lKUMjIyzEVnoeryI8WJMXAAAMA3Y+BKG3pQAQCAXQQ4hwXmoFpMYwAAADYR4BzGGDgAABArApxL0IUKAADsIsC5pgsVAADAHgJcFLqESFpamqSnp0txC8xCpQUOAADYRYCLQpcQyc7OlqysLNsHEgAAoKQQ4FyCWagAAMAuApxbZqEyCA4AANhEgHNY3KlpDOQ3AABgFwHOYawDBwAAYkWAc4nTnfMVAAAggADnMIbAAQCAWBHgXNKFSgMcAACwiwDn8EK+AAAAsSLAObyQL7NQAQBArAhwTgt2oTKJAQAA2EOAc8kkBgAAALsIcC5BAxwAALCLAOewOFbyBQAAMSLAuWUdOJrgAACATQQ4h9EABwAAYkWAcwnmoAIAALsIcA4v5Pu/LtRifykAAFBKEOCcXsj3VB8q+Q0AANhFgHMY68ABAIBYEeBcglmoAADALgKcW06l5fR+AAAAzyDAOYyFfAEAQKwIcC7BLFQAAGAXAc41kxjoRAUAAPYQ4FxyJgZa4AAAgF0EOIexjAgAAIgVAc4l6EAFAAB2EeCcPpVW4EwMJDgAAGATAc7pU2mdurZogwMAADYR4JzGIDgAABAjApxL0IUKAADsIsA5LO5UExxD4AAAgF0EOJesAwcAAGAXAc4t6EMFAAA2EeAc9r9ZqAAAAPYQ4BzGqbQAAECsCHAumcQAAABgFwHOJehCBQAAdhHgXNOFSoQDAAD2EODcMomB/AYAAGwiwDmNheAAAECMCHBRZGZmSlpamqSnp0tJoQEOAADYRYCLIiMjQ7KzsyUrK0tKrguVCAcAAOwhwDmMHlQAABArAhwAAIDHEOAcxixUAAAQKwKcw+JO9aEyAg4AANhFgHMYJ9ICAACxIsC5BLNQAQCAXQQ4t5xKy+kdAQAAnkGAc9ypMXAkOAAAYBMBzmGsAwcAAGJFgHMJi05UAABgEwHOLS1wdKECAACbCHAOI78BAIBYEeAAAAA8hgDnljMx0IUKAABsIsC5pguVBAcAAOwhwEWRmZkpaWlpkp6eLsWNZUQAAECsCHBRZGRkSHZ2tmRlZUlJoQsVAADYRYBz3KkxcE7vBgAA8AwCnFvOhUoTHAAAsIkA55JJDAAAAHYR4FyCLlQAAGAXAc5hnEoLAADEigDnsDgmMQAAgBgR4BzGOnAAACBWBDiXYBYqAACwiwDnllNpMYsBAADYRIBzycnsAQAA7CLAuQQNcAAAwC4CnEswBg4AANhFgHPLqbSc3hEAAOAZBDiHMQIOAADEigDnEsxCBQAAdhHgHMYsVAAAECsCnGvWgWMUHAAAsIcA5zCWgQMAALEiwLkE7W8AAMAuApzD4k51otKDCgAA7CLAOY11RAAAQIwIcC5h0YkKAABsIsC5ZhaqwzsCAAA8gwAXRWZmpqSlpUl6enqxVwCn0gIAALEiwEWRkZEh2dnZkpWVJcWNhXwBAECsCHBuQRcqAACwiQDnljFwJDgAAGATAc5hwTFwtMABAACbCHAuWcgXAADALgKcS9AABwAA7CLAuaYLlQgHAADsIcABAAB4DAHOJWh/AwAAdhHgHMYsVAAAECsCnEtmodICBwAA7CLAuaQFDgAAwC4CnFswCxUAANhEgHPLqbToQwUAADYR4BwWd6oPlfwGAADsIsA5jCFwAAAgVgQ4l+BMDAAAwC4CnNMCp9Jyej8AAIBnEOAcRhcqAACIFQHOJZiFCgAA7CLAOYxZqAAAIFYEOLd0odIEBwAAbCLAOYxTaQEAgFgR4FyCWagAAMAuApzD4k51otKDCgAA7CLAuaQLlRY4AABgFwHOYawDBwAAYkWAcwlOpQUAAOwiwDmNLlQAABAjApxbFvJlEBwAACiOAPfYY4/J0aNHg7cXLlwox48fD94+ePCgDB8+PJan9D3GwAEAgGINcKNHjzYhLaBPnz6yefPm4O0jR47Is88+G/NOQGeh0gQHAACKIcBFDrRn4H0RnomB/AYAAGxiDJxLFvIFAACwiwDnEjTAAQAAuxIlRs8//7xUqFDB/JyTkyNTp06VGjVqmNuh4+MQ45kYSHAAAKA4AlyDBg3kueeeC96uU6eOvPLKK/m2cZtrrrlGPvnkE+nZs6e8+eab4ib/GwJHggMAAMUQ4NavXy9edOedd8rNN98sL730krgOQ+AAAECMfDEGrlu3blKxYkVxM7pQAQBAsQS4xYsXy7vvvht238svvyyNGzeWWrVqybBhw8IW9rVj/vz50rdvX6lXr545K8GsWbPybZOZmSmNGjWSMmXKSKdOnWTJkiVS2mah0oEKAACKJcA9+OCD8u233wZvf/PNN3LLLbdIr1695L777pP//Oc/MmHChFieUg4fPiytW7c2IS2a6dOny8iRI2XcuHHy1VdfmW179+4tO3bsCG7Tpk0badmyZb7Lli1bxDuTGIhwAACgGMbALVu2TB566KHg7ddff920iAUmNtSvX98ErQceeMD2c+rZHPRSkIkTJ8rQoUNlyJAh5vYzzzwj7733nkyZMsWExsB+FQVtPQxtQTxw4IC5zsvLM5diEQhullV8r+FCWlYNrZS59PNjXfu13H4ss6Lc/qnvvBJ4j9t97pgC3N69e6V27drB259++mlY+EpPT5dNmzZJUTlx4oQsXbrUnMIrID4+3rT4aXduUdPWw/Hjx+e7f+fOnXLs2DEpDnr6MXX4yNGwVsXSTt+g+/fvNx8ErVM/8GOZFeX2T31T1/6pa7/Wd14JlNnukmwxBTgNb+vWrTMtbRqutEszNPDoiyYlJUlR2bVrl+Tm5oaFxsB+rFq1yvbzaOBbvny56a5NTU2VGTNmSOfOnfNtp0FRu2tDW+C0rDVr1pRKlSpJcShffq+5Llu2rBlH6Bf6IdAxj3ps/fTB91uZFeX2T31T1/6pa7/Wd14JlFnH+xd5gLv88stNt+Wjjz5qJhuUK1dOunTpEnx8xYoV0rRpU3GbOXPm2NouJSXFXCJpJRVXRcWfGgSnbwi/fAACAmX2U7n9WGZFuf1T39S1f+rar/UdV8xltvu8MQU4Hf927bXXSteuXc3ZGPQsDMnJycHHdVzapZdeKkVFz/CQkJAg27dvD7tfb+siwqUJcxgAAIBdibEGKl32Q/t/NcBpuAqlXZNFud6ahsP27dvL3LlzpV+/fsHmS709YsQIKQ2Cs1BZSAQAABRHgNOzGdihLXF2HTp0SNasWRO8rWPsdFZptWrVzGm5dEzaoEGDpEOHDtKxY0eZNGmSGcsWmJXqdcFTabGKCAAAKI4Ap12mDRs2lLZt2xbZumVffvmldO/ePXg7MIlAQ5u+Xv/+/c0s0LFjx8q2bdvMmm+zZ8/ON7GhKOmadHrRCRQl1gQHAABQHAHu9ttvl2nTpplWMm0B+93vfmdayn7paa5OFwa1u7Qku0wzMjLMRWehVq5cuURekwY4AABgV0xTKLRVauvWrTJq1Chz1gVdYuP666+XDz74gDMJ/OIuVCIcAACwJ+Y5sLrMxg033CAfffSRZGdnS4sWLWT48OHmXKU6ng1nOokBAADAnvhfulaJroeirUclMl6sFGIEHAAAKPYAp+cK1XFwl1xyiZxzzjnmhPZPP/20bNy40SwtgjNEExwAACiOSQzaVaonsNexb7qkiAY5XRsOZ05bMBX5DQAAFEuAe+aZZ8zabE2aNDEnstdLNDNnzhQvK8llROhCBQAAxRrgBg4cGGwxKs0cWUaEWagAAKC4FvJF0WIWKgAAKNFZqCgCgTFwDIIDAAA2EeAcVvo7pAEAQFEjwLmExTxUAABgEwHOLWPg6EIFAAA2EeAcRhcqAACIFQHOYX5YlgUAABQtAlwUuohvWlqapKenS0mhCxUAANhFgItCF/HNzs6WrKwsKW6B9jcmMQAAALsIcA6jBxUAAMSKAOcSdKECAAC7CHAOizvVicoqIgAAwC4CnNOC68AR4QAAgD0EOIexiAgAAIgVAc4laIADAAB2EeDcciotp3cEAAB4BgHO4YV8A5MYSHAAAMAuApzTC/kG8xttcAAAwB4CnMOCZ2IgvwEAAJsIcC45mT35DQAA2EWAc1igCzWPJjgAAGATAc5hdKECAIBYEeAcFs/Z7AEAQIwIcA6jCxUAAMSKAOeWSQzMYgAAADYR4BzGOnAAACBWBDiXTGLIowUOAADYRIBz+lRadKECAIAYEeAcPpVWfKAJjqV8AQCATQQ4hwVOZk8XKgAAsIsA55ZJDExDBQAANhHgXDMLFQAAwB4CnFu6UPOc3hMAAOAVBDiH/e9MWrTBAQAAewhwDgucC5UhcAAAwC4CnMMCDXC0vwEAALsIcA7jZPYAACBWBDiHcSYGAAAQKwKcw+hCBQAAsSLAOSz+1Lm0WMgXAADYRYBzSwscsxgAAIBNBLgoMjMzJS0tTdLT06XkzsRAggMAAPYQ4KLIyMiQ7OxsycrKkpKaxMDJ7AEAgF0EOIfRhQoAAGJFgHMYXagAACBWBDiHcSotAAAQKwKca7pQmcQAAADsIcC5pgsVAADAHgKcw5iFCgAAYkWAcxhdqAAAIFYEOJe0wNGHCgAA7CLAOezUqVAlj0kMAADAJgKcW7pQHd4PAADgHQQ4l3Sh0gAHAADsIsA5LDAEji5UAABgFwHOLS1wTu8IAADwDAKcS8bAkeAAAIBdBDiHMQsVAADEigAXRWZmpqSlpUl6eroUN7pQAQBArAhwUWRkZEh2drZkZWVJyZ2JodhfCgAAlBIEONecC5UEBwAA7CHAuWQZEQAAALsIcA7jZPYAACBWBDiHxbMOHAAAiBEBzjVnYnB6TwAAgFcQ4ByWcGohuDwSHAAAsIkA57CEU01wOQQ4AABgEwHOYQkJPwe4XAIcAACwiQDnkhY4AhwAALCLAOeSMXC5eXlO7woAAPAIApzDEgMBjlmoAADAJgKcw+KDLXCWWJxOCwAA2ECAc0kLnGIeAwAAsIMA55IWOJXDODgAAGADAc5NLXDMYwAAADYQ4FyyjIiiBQ4AANhBgHPJMiKKteAAAIAdBDiHEeAAAECsCHAOi4uLk0AjHC1wAADADgKcm87GwDpwAADABgKcCwRa4HI4HQMAALCBAOeq86FyPi0AAHB6BDgXSDoV4E7kshAcAAA4PQJcFJmZmZKWlibp6elSEsomJZjrIydyS+T1AACAtxHgosjIyJDs7GzJysoqkUool/xzNRw5nlMirwcAALyNAOcCtMABAIBYEOBcoGzSz9Vw+AQtcAAA4PQIcC5QPvnnMXAHjp50elcAAIAHEOBcoHbFZHP9076jTu8KAADwAAKcC9SrdCrA7SHAAQCA0yPAuUC9yinmeuOeI07vCgAA8AACnAvUr/JzgPtx5yHJ42wMAADgNAhwLlC/ahkpkxQvh0/kyo+7Dju9OwAAwOUIcC6QGB8naXUrmZ9Xbt7v9O4AAACXI8C5RMt6lc31NwQ4AABwGgQ4lzg/9ecWuG9+ogUOAAAUjgDnsha4b7fsZyIDAAAoFAHOJZrWLG/OiaoTGX7Yccjp3QEAAC5GgHOJxIR4adewivl5yfo9Tu8OAABwMQKci3RsVN1cL1lHgAMAAAUjwLlIx8bVzPWSdbvFsiyndwcAALgUAc5F2jaoIkkJcbL9wHFOqwUAAApEgHORMkkJ0jr153FwC9fsdnp3AACASxHgXKbrOTXN9cerdji9KwAAwKUIcC7T47xa5nrhml1y7GSu07sDAABciADnMnpO1DqVysjRk7ny+Y90owIAgPwIcC4TFxcXbIX7MHu707sDAABciADnQpe3rGuu31uxVY7n0I0KAADCEeBcqHPT6lK7UorsP3pS5jGZAQAARCDAuVBCfJz0a3uW+fnNpZud3h0AAOAyBDiX+k37VHM9d9V22bD7sNO7AwAAXIQA51Jn16oo3ZrXFD2j1gsL1jm9OwAAwEUIcC42rEsTc/3Gl5tk2/5jTu8OAABwCQKcyycztG9YVY6dzJPHP1jt9O4AAACXIMC5fE24MVemmZ///dVP8uX6PU7vEgAAcAECnMu1qV8lOKHhj28sk4PHTjq9SwAAwGEEOA8Y0zdNUquWlU17jsqdry+TnNw8p3cJAAA4iADnAZXKJMlTN7SVlMR4+XjVDhn5xnI5kUOIAwDArwhwHtG2QVV5+sZ2ZpHfd5ZvkYFTvpDtB5iZCgCAHxHgPOSStNrywqAOUj45QT7/cY9cMvFTeWnRejl2kvOlAgDgJ6U+wG3atEm6desmaWlp0qpVK5kxY4Z4WbfmtWRWxoXSKrWyHDiWI+Pe+VYufmyePDZ7lazedlAsXfkXAACUaolSyiUmJsqkSZOkTZs2sm3bNmnfvr1cfvnlUr58efGqZrUryszbL5BpWZvkn/PWyNb9x+Sfn6w1l1oVU6RTk+rSol4laVargjSpWUHqVCojZZMTnN5tAABQREp9gKtbt665qDp16kiNGjVkz549ng5wKjEhXm76VUO5vkOqfJS9XWZ9vUXmf79Tdhw8Lv9ZvsVcQlVISTThrmr5ZCmXnCDlkxOlfIpeEqRsUoIkJsRJYny8JCXESULwOs68TlJ8nMTFicSJ+ceIO7VO3c/Xpy6nHtSfJfLxkMf0pzzLkv3790vlHXkSH/iFUs6PZfZDuQsqUl6eJfsPnCp3fOkrdzR+LLOi3P6p77xT7/EuyRUltVp5fwe4+fPny+OPPy5Lly6VrVu3yltvvSX9+vUL2yYzM9Nsoy1orVu3lqeeeko6duwY82vpa+Tm5kr9+vWltEhJTJArW9UzFx0L9/XGfWbB3+93HJIfth+U9bsPmzM5HDqeYy6y67DTuwwAgKf9vTwBTg4fPmxC2c033yzXXnttvoM0ffp0GTlypDzzzDPSqVMn0x3au3dvWb16tdSqVctso92jOTk5+X73ww8/lHr16pmftdVt4MCB8txzz0lpVSYpwZx+Sy8BOiZOg5u2zO04cFz2Hz0pR07kyOHjOXL4RK65PnoiV3LyLMnJy5OcXEtO5lqSm5cnJ/W+3J/vs0KeT3/WoXah90nwvpCfT90ODMszV8GfLTl58qQkJSUFW+dKu5Is8/9qzHla/8Fyl7KqLmzIqT508uQJSUpK9sk73J9lVpTbP/VtnXqPa2+W0+IsF4161y63yBY4DW3p6eny9NNPm9t5eXmmBe2OO+6Q++67z9bzHj9+XC655BIZOnSo3HTTTYVup5eAAwcOmNfau3evVKpUSYqDlmfnzp1Ss2ZNiY8v9XNKfF1uP5ZZUW7/1Dd17Z+69mt955VAmTV7VK1a1Qw9KSx7ON6FWpgTJ06Ybs/Ro0cH79MD1qtXL1m8eLGt59B8OnjwYOnRo0eh4U1NmDBBxo8fn+9+raxjx44V25tBK0n30y8fAL+W249lVpTbP/VNXfunrv1a33klUOaDBw/a2s7VAW7Xrl1mzFrt2rXD7tfbq1atsvUcCxcuNN2wuoTIrFmzzH2vvPKKnH/++fm21aCo3bWRLXCatIuzBU5bHv30Pxi/ltuPZVaU2z/1TV37p679Wt95JVDmMmXKeD/AFYWLLrrIHHA7UlJSzCWSVlJxvjn1zVDcr+FGfiy3H8usKLd/6pu69k9d+7W+44q5zHaf19VHXJf8SEhIkO3bt4fdr7d1SRAAAAA/cnWAS05ONgvvzp07N3iftqbp7c6dOzu6bwAAAE5xvAv10KFDsmbNmuDtdevWybJly6RatWrSoEEDMyZt0KBB0qFDB7P2my4jokuPDBkyxNH9BgAA8G2A+/LLL6V79+7B24FJBBrapk6dKv379zezQMeOHWsW8tU132bPnp1vYkNR0oWD9aITKAAAANzG8QCnJ5o/3VJ0I0aMMJeSkpGRYS46C7Vy5col9roAAACeHwMHAACA/AhwAAAAHkOAAwAA8BjHx8C5WWBsno6FKy66LIqeNkNXXvbTQoh+LLcfy6wot3/qm7r2T137tb7zSqDMgcxxuvkBBDgb5yPT02kBAACUZAYpbCJlnHW6iOdjmrS3bNkiFStWNKfOKA6B861u2rSp2M636kZ+LLcfy6wot3/qm7r2T137tb4PlECZNZZpeKtXr16hrXy0wBVCD1xqaqqUBH0j+OUD4Pdy+7HMinL7B3XtL36s70rFXGY7S5j5o9MaAACgFCHAAQAAeAwBzmEpKSkybtw4c+0nfiy3H8usKLd/6pu69k9d+7W+U1xUZiYxAAAAeAwtcAAAAB5DgAMAAPAYAhwAAIDHEOAAAAA8hgDnsMzMTGnUqJE5r1qnTp1kyZIl4lUTJkyQ9PR0c+aKWrVqSb9+/WT16tVh23Tr1s2c1SL0ctttt4Vts3HjRrniiiukXLly5nn+9Kc/SU5OjrjRAw88kK885557bvDxY8eOSUZGhlSvXl0qVKgg1113nWzfvt2z5Q3Q92xkufWiZS1N9Tx//nzp27evWRFdyzBr1qx8K6aPHTtW6tatK2XLlpVevXrJDz/8ELbNnj17ZMCAAWbRzypVqsgtt9wihw4dCttmxYoV0qVLF/N3QFd5f+yxx8SNZT558qTce++9cv7550v58uXNNgMHDjRnrDnd++ORRx5xbZnt1PXgwYPzlemyyy7zdF3bKXe0z7leHn/8cc/W9wQb31VF9bf7k08+kXbt2plZq2effbZMnTq16Aqip9KCM15//XUrOTnZmjJlivXtt99aQ4cOtapUqWJt377dk1XSu3dv68UXX7RWrlxpLVu2zLr88sutBg0aWIcOHQpu07VrV1POrVu3Bi/79+8PPp6Tk2O1bNnS6tWrl/X1119b//3vf60aNWpYo0ePttxo3LhxVosWLcLKs3PnzuDjt912m1W/fn1r7ty51pdffmn96le/si644ALPljdgx44dYWX+6KOP9JR81rx580pVPet+3X///dbMmTNN+d56662wxx955BGrcuXK1qxZs6zly5dbV111ldW4cWPr6NGjwW0uu+wyq3Xr1tbnn39uffbZZ9bZZ59t3XDDDcHH9bjUrl3bGjBggPnsTJs2zSpbtqz17LPPWm4r8759+0ydTZ8+3Vq1apW1ePFiq2PHjlb79u3DnqNhw4bWgw8+GFb/oX8H3FZmO3U9aNAgU5ehZdqzZ0/YNl6razvlDi2vXvT7Ki4uzlq7dq1n67u3je+qovjb/eOPP1rlypWzRo4caWVnZ1tPPfWUlZCQYM2ePbtIykGAc5D+4cvIyAjezs3NterVq2dNmDDBKg30S17/IHz66afB+/SL/c477yzwd/RDEB8fb23bti143+TJk61KlSpZx48ft9wY4PQPdjT6ZZeUlGTNmDEjeN93331njol+8XmxvAXROm3atKmVl5dXKutZRX65aVnr1KljPf7442F1npKSYr6glP7R1t/LysoKbvP++++bL8DNmzeb2//85z+tqlWrhpX73nvvtZo3b245LdoXeqQlS5aY7TZs2BD2hf7kk08W+DtuLrMqKMBdffXVBf6O1+vabn3rMejRo0fYfV6v7x0R31VF9bd71KhR5j/4ofr3728CZFGgC9UhJ06ckKVLl5oul9Bzr+rtxYsXS2mwf/9+c12tWrWw+1977TWpUaOGtGzZUkaPHi1HjhwJPqZl1+6Z2rVrB+/r3bu3OYHwt99+K26kXWba/dCkSRPTfaLN6krrV7ucQutYu1cbNGgQrGMvljfae/nVV1+Vm2++2XSdlNZ6jrRu3TrZtm1bWP3q+Qt1KERo/WpXWocOHYLb6Pb6Wf/iiy+C21x88cWSnJwcdiy0S2fv3r3ihc+51ruWM5R2oWn3U9u2bU13W2jXklfLrN1h2lXWvHlzuf3222X37t3Bx/xQ19qF+N5775mu4Uheru/9Ed9VRfW3W7cJfY7ANkX1Hc/J7B2ya9cuyc3NDat8pbdXrVolXpeXlyd33XWXXHjhheYLPODGG2+Uhg0bmsCjYyJ0PI1+iGfOnGke1y/EaMck8Jjb6Je1jmnQP+hbt26V8ePHm3EeK1euNPurf7Aiv9i0PIGyeK280eiYmX379pkxQqW1nqMJ7Ge0coTWr37hh0pMTDRfFKHbNG7cON9zBB6rWrWquJWOE9K6veGGG8JO7P2HP/zBjPvRci5atMgEeP18TJw40bNl1vFu1157rdnvtWvXyp///Gfp06eP+TJOSEgo9XWtXnrpJTNuTI9DKC/Xd16U76qi+ttd0DYa8o4ePWrGzf4SBDgUCx38qSFmwYIFYfcPGzYs+LP+70UHf/fs2dP8QWzatKnnakP/gAe0atXKBDoNLm+88cYv/nB6xQsvvGCOg4a10lrPyE9bKK6//nozkWPy5Mlhj40cOTLsc6FfhrfeeqsZPO6GUxCdid/+9rdh72ktl76XtVVO39t+MGXKFNPLoBMRSkt9ZxTwXeUFdKE6RLuW9H9tkbNa9HadOnXEy0aMGCHvvvuuzJs3T1JTUwvdVgOPWrNmjbnWskc7JoHH3E7/x3bOOeeY8uj+aveitk4VVMdeL++GDRtkzpw58vvf/95X9Ry6n4V9hvV6x44dYY9r15LOVvTyeyAQ3rT+P/roo7DWt4LqX8u9fv16z5Y5kg6Z0L/joe/p0ljXAZ999plpRT/dZ91L9T2igO+qovrbXdA2+nkpiv/gE+Acov9Dad++vcydOzesKVdvd+7cWbxI/yeuH4i33npLPv7443xN5tEsW7bMXGsLjdKyf/PNN2F/CANfEGlpaeJ2umSAtjJpebR+k5KSwupY/wDqGLlAHXu9vC+++KLpNtKp9H6qZ6Xvb/0DHVq/2jWi451C61e/BHRMTYB+NvSzHgi1uo0u5aChKPRYaLe8G7vUAuFNx35qeNdxT6ej9a9jwQJdjF4rczQ//fSTGQMX+p4ubXUd2dKuf9Nat27t+fq2TvNdVVR/u3Wb0OcIbFNk3/FFMhUCZ7yMiM5Ymzp1qpnBNGzYMLOMSOisFi+5/fbbzZIKn3zySdh08iNHjpjH16xZY6aa65TsdevWWW+//bbVpEkT6+KLL843NfvSSy8107t1unXNmjVdt7xEwN13323Kq+VZuHChmVKuU8l1VlNgKrpOT//4449NuTt37mwuXi1vKJ01rWXT2WShSlM9Hzx40CwRoBf9czlx4kTzc2DGpS4jop9ZLeOKFSvMDL1oy4i0bdvW+uKLL6wFCxZYzZo1C1taQme86RILN910k1nWQP8u6NIDTi2xUFiZT5w4YZZKSU1NNfUW+jkPzLxbtGiRmZGoj+tSE6+++qqp24EDB7q2zKcrtz52zz33mBmI+p6eM2eO1a5dO1OXx44d82xd23mPB5YB0f3UWZaRvFjft5/mu6qo/nYHlhH505/+ZGaxZmZmsoxIaaLrwuibRNeD02VFdP0gr9IPf7SLrrejNm7caL7Eq1WrZoKrrpGkb+zQ9cHU+vXrrT59+ph1gjQMaUg6efKk5UY6Jbxu3bqm/s466yxzWwNMgH6RDx8+3Eyh1w/yNddcY/5QeLW8oT744ANTv6tXrw67vzTVs65rF+09rUtKBJYSGTNmjPly0rL27Nkz3/HYvXu3+RKvUKGCWWJgyJAh5kszlK4hd9FFF5nn0PeRBkM3llnDS0Gf88AagEuXLrU6depkviDLlCljnXfeedbDDz8cFnTcVubTlVu/2PWLWr+gdXkJXTZD1zmM/M+21+razntcadDSz6kGsUherG85zXdVUf7t1uPbpk0b8x2h/5ENfY1fKu5UYQAAAOARjIEDAADwGAIcAACAxxDgAAAAPIYABwAA4DEEOAAAAI8hwAEAAHgMAQ4AAMBjCHAAAAAeQ4AD4BvdunWTu+66S9wkLi5OZs2a5fRuAPAYzsQAwDf27NljTlJdsWJFadSokQlzJRXoHnjgARPU9ETfobZt22ZO6J2SklIi+wGgdEh0egcAoKRUq1atyJ/zxIkTkpycfMa/X6dOnSLdHwD+QBcqAN91oer1hg0b5I9//KPpwtRLwIIFC6RLly5StmxZqV+/vvzhD3+Qw4cPBx/XlruHHnpIBg4cKJUqVZJhw4aZ+++9914555xzpFy5ctKkSRMZM2aMnDx50jw2depUGT9+vCxfvjz4enpftC7Ub775Rnr06GFev3r16ub5Dx06FHx88ODB0q9fP/nb3/4mdevWNdtkZGQEXwuAPxDgAPjOzJkzJTU1VR588EHZunWruai1a9fKZZddJtddd52sWLFCpk+fbgLdiBEjwn5fw1Pr1q3l66+/NkFNabeshrLs7Gz5+9//Ls8995w8+eST5rH+/fvL3XffLS1atAi+nt4XSYNi7969TZdqVlaWzJgxQ+bMmZPv9efNm2f2Va9feukl87qBQAjAH+hCBeDLrtSEhAQTukK7MCdMmCADBgwIjotr1qyZ/OMf/5CuXbvK5MmTpUyZMuZ+bSHTQBbqL3/5S1gr3T333COvv/66jBo1yrSmVahQQRITEwvtMv3Xv/4lx44dk5dfflnKly9v7nv66aelb9++8uijj0rt2rXNfRrw9H4tw7nnnitXXHGFzJ07V4YOHVrERwqAWxHgAOAU7eLUlrfXXnsteEwsy5K8vDxZt26dnHfeeea+Dh065Dtm2lqnYU9bxrTLMycnx3SxxuK7774zLXuB8KYuvPBC8/qrV68OBjhtydPwFqBdqdr1CsA/CHAAcIoGr1tvvdWMe4vUoEGD4M+hAUstXrzYtNzpODftAq1cubJpfXviiSeK5djqTNpQOo5OQx4A/yDAAfAlnTmam5sbdl+7du3MGLazzz47pudatGiRNGzYUO6///7gfTpJ4nSvF0lb+HQsm46FC4TEhQsXSnx8vDRv3jymfQJQujGJAYAv6Ti1+fPny+bNm2XXrl3BmaQaxnTSgK7X9sMPP8jbb7+dbxJBJB0rt3HjRtPqpl2o2pX61ltv5Xs97YbV59XXO378eL7n0VY8HWc3aNAgWblypZmkcMcdd8hNN90U7D4FAEWAA+BLOgN1/fr10rRpU6lZs6a5r1WrVvLpp5/K999/b5YSadu2rYwdO1bq1atX6HNdddVVZkkSDXpt2rQxITAwOzVAZ7bqDNfu3bub15s2bVq+59ElSD744AOz4HB6err8+te/lp49e5oJCwAQijMxAAAAeAwtcAAAAB5DgAMAAPAYAhwAAIDHEOAAAAA8hgAHAADgMQQ4AAAAjyHAAQAAeAwBDgAAwGMIcAAAAB5DgAMAAPAYAhwAAIB4y/8Dj5IwXvDSYmwAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# --- PROVIDED --- gradient check, then fit, then plot. Do not modify.\n", "\n", "rng = np.random.default_rng(1)\n", "err = grad_check(compute_loss, compute_grad, rng.normal(size=4), X, y)\n", "assert err < 1e-7\n", "\n", "w2, hist2 = gradient_descent(compute_loss, compute_grad, X, y,\n", " w0=np.zeros(4), gamma=0.1, numIterations=2000)\n", "print(\"\\nweights [EXAM1, EXAM2, EXAM3, intercept] =\", np.round(w2, 6))\n", "print(\"final MSE =\", round(hist2[-1], 6))\n", "\n", "plt.figure(figsize=(7, 4))\n", "plt.semilogy(hist2)\n", "plt.xlabel(\"iteration\"); plt.ylabel(\"MSE\")\n", "plt.title(\"2(a) OLS convergence\"); plt.grid(alpha=.3); plt.show()" ] }, { "cell_type": "markdown", "id": "7ee00ed3", "metadata": {}, "source": [ "**EXAM3 dominates** (weight $\\approx 0.571$, versus $0.282$ for EXAM2 and $0.182$ for\n", "EXAM1). The intercept is $0$ to machine precision, which is exactly what standardizing both\n", "sides guarantees.\n", "\n", "Comparing coefficient magnitudes directly is normally meaningless, because each one is\n", "denominated in the units of its own input — halve an input's units and its coefficient\n", "doubles, with no change to the model at all. Here every column was standardized to unit\n", "variance first, so all three coefficients are in the same units (\"standard deviations of\n", "final exam score per standard deviation of midterm score\") and the comparison is legitimate.\n", "This is one of the main practical reasons to standardize.\n", "\n", "A caveat worth stating: the midterms are strongly correlated with each other, so these\n", "weights should not be read as \"EXAM3 causes the final grade.\" With collinear inputs the\n", "regression can shift weight between them fairly arbitrarily." ] }, { "cell_type": "markdown", "id": "198bd747", "metadata": {}, "source": [ "## 2(b) Ridge regression\n", "\n", "Ridge regression is a linear model with a slightly different loss function. It is the same MSE,\n", "plus a penalty on the size of the weights:\n", "\n", "$$e_{\\text{ridge}}(\\bar w) = \\frac{1}{N}\\sum_{i=1}^{N}(y_i - \\hat y_i)^2 + \\lambda \\sum_{j=1}^{m-1} w_j^2$$\n", "\n", "**Note the sum runs to $m-1$: the intercept $b = w_m$ is *not* penalized.** Shrinking the\n", "intercept would mean penalizing the model for predicting a nonzero average, which is not\n", "something we want to discourage.\n", "\n", "**(i)** Derive $\\partial e_{\\text{ridge}} / \\partial \\bar w$ by hand. It is your Problem 2(a)\n", "gradient plus one extra term. Be careful about the intercept component.\n", "**(ii)** Implement it and pass gradient checking.\n", "**(iii)** Fit with $\\lambda = 1$ and report the weights.\n", "**(iv)** Now fit with $\\lambda = 100$ and report those weights. Compare them to your\n", "$\\lambda = 1$ weights: what changed, and what does that tell you about what $\\lambda$ does?" ] }, { "cell_type": "markdown", "id": "5454d6b0", "metadata": {}, "source": [ "**(i)** For a non-intercept weight $j < m$:\n", "$$\\frac{\\partial e_{\\text{ridge}}}{\\partial w_j} = \\left[-\\frac{2}{N}X^T(\\bar y - X\\bar w)\\right]_j + 2\\lambda w_j$$\n", "and for the intercept $j = m$ the penalty contributes nothing:\n", "$$\\frac{\\partial e_{\\text{ridge}}}{\\partial b} = \\left[-\\frac{2}{N}X^T(\\bar y - X\\bar w)\\right]_m$$" ] }, { "cell_type": "markdown", "id": "46430521", "metadata": {}, "source": [ "### 2(b)(ii) Implement the ridge loss and gradient\n", "\n", "Two lines to write. Both functions reuse the ones you already wrote, so neither needs to\n", "recompute anything about the data.\n", "\n", "The one idea to get right is the intercept. In numpy, `w[:-1]` means \"all of w except the\n", "last entry\" — that is exactly the set of weights the penalty applies to." ] }, { "cell_type": "code", "execution_count": 11, "id": "bd52bc9c", "metadata": {}, "outputs": [], "source": [ "def compute_ridge_loss(X, y, w, lam):\n", " \"\"\"The ridge objective: mean squared error, plus lam times the sum of the squared\n", " NON-intercept weights.\n", "\n", " Inputs\n", " ------\n", " X, y : the data, exactly as in compute_loss\n", " w : the weights, shape (m,). w[-1] is the intercept.\n", " lam : lambda, the penalty strength. A single number.\n", "\n", " Output\n", " ------\n", " A single number.\n", " \"\"\"\n", " mse = compute_loss(X, y, w) # the first term -- already written in Problem 1\n", " penalized = w[:-1] # every weight EXCEPT the intercept\n", "\n", " # YOUR CODE HERE (1 line): return mse plus lam times the sum of the squared\n", " # entries of \"penalized\". np.sum(a) adds up an array.\n", " return mse + lam * np.sum(penalized ** 2)\n", "\n", "\n", "def compute_ridge_grad(X, y, w, lam):\n", " \"\"\"Gradient of compute_ridge_loss with respect to w.\n", "\n", " Inputs\n", " ------\n", " X, y, w, lam : exactly as in compute_ridge_loss above.\n", "\n", " Output\n", " ------\n", " An array of shape (m,), same shape as w.\n", " \"\"\"\n", " g = compute_grad(X, y, w).copy() # the MSE part of the gradient, from Problem 1\n", "\n", " # YOUR CODE HERE (1 line): add the penalty term to the first m-1 entries of g, and\n", " # leave the last entry (the intercept) alone. Write it as\n", " # g[:-1] = g[:-1] + \n", " g[:-1] = g[:-1] + 2.0 * lam * w[:-1]\n", "\n", " return g" ] }, { "cell_type": "markdown", "id": "30a6405a", "metadata": {}, "source": [ "Your two ridge functions take a fourth argument, `lam`, but `grad_check` and\n", "`gradient_descent` expect functions of exactly `(X, y, w)`. The fix is a pair of tiny\n", "wrapper functions that pin `lam` to one value." ] }, { "cell_type": "code", "execution_count": 12, "id": "05f5c224", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ " your gradient: [-2.815819 -4.308325 -4.144223 -4.882935]\n", " numerical : [-2.815819 -4.308325 -4.144223 -4.882935]\n", " relative error = 3.991e-12 -> PASS\n" ] } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "def ridge_loss_lam1(X, y, w):\n", " \"\"\"compute_ridge_loss with lambda pinned to 1.\"\"\"\n", " return compute_ridge_loss(X, y, w, 1.0)\n", "\n", "\n", "def ridge_grad_lam1(X, y, w):\n", " \"\"\"compute_ridge_grad with lambda pinned to 1.\"\"\"\n", " return compute_ridge_grad(X, y, w, 1.0)\n", "\n", "\n", "rng = np.random.default_rng(2)\n", "err = grad_check(ridge_loss_lam1, ridge_grad_lam1, rng.normal(size=4), X, y)\n", "assert err < 1e-7" ] }, { "cell_type": "markdown", "id": "e9545305", "metadata": {}, "source": [ "### 2(b)(iii) Fit with $\\lambda = 1$" ] }, { "cell_type": "code", "execution_count": 13, "id": "3d7fec7d", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "lambda = 1 weights: [0.241794 0.238395 0.27735 0. ]\n", "OLS weights : [0.181782 0.282127 0.571263 0. ]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAogAAAGJCAYAAAAJ/cKCAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAQG5JREFUeJzt3QmcTXX/wPHv2HfZxdhJ9p1UsoeKKOWJInkojYhKqSdKhRaKEiWiRaRQzyNkJ0vWsTSRLSp7izWGmfN/fX/Pc+//njt3Zu4dd2buvefzfp7bzD33zDnnd3733vP1W74nyrIsSwAAAID/yeL6BQAAACBABAAAQBK0IAIAAMCGABEAAAA2BIgAAACwIUAEAACADQEiAAAAbAgQAQAAYEOACAAAABsCRCDEJCYmSs2aNeWVV15xL3vhhRckKipKTp06leLfXr58WcqUKSPvvvvuVR/HypUrzT71Z2patGhhHkAwTJ8+3bz3fv755ww/of5+1lKif6/bAcIZASKQzjZt2iQDBgyQGjVqSN68eaVs2bJy7733yk8//eRz/c8++0x++eUX8zeByp49uwwZMsQElxcvXgzC0QOIJN9++6306dPH/CM0a9asUr58+cw+JIQoAkQgnb366qvy5ZdfSuvWrWX8+PHSr18/Wb16tdSvX1927dqVZP3XX39d/vGPf0jBggXTtL/evXub1o+ZM2de1XHfcsst8vfff5ufACKDfi/oQ79fSpUqldmHgxBGgAikM23RO3TokEyYMEH++c9/yr/+9S9Zs2aNXLlyRcaMGWNbd9u2bbJ9+3bTwphW11xzjdx6662mmy4ttOVRu7mzZMkiuXLlMj+R8c6fP89pR9CNGjVKzpw5I2vXrpU6depwhpEsvvmBdHbjjTdKjhw5bMuqVKliupx//PFH2/L58+ebdZNrtdOWQQ0eCxQoIEWKFJFBgwb57Epu27atfPfdd/LHH3/4Nc5w1qxZJnAtXbq05MmTx1xAkhuD+P7770ulSpUkd+7c0rhxYxPs+qJBcadOnUy3evHixWXw4MGyePFin9v8/vvvpX379qZVQ/ffvHlzcwHzh5Zfx3tdd911JqC99tpr5a677pL9+/fbgq0nnnjCjM/MmTOnVK1aVd544w2xLMu2LT027drXetAuOF1X62nRokXudb744guz3qpVq5Icy3vvvWde82wZ3r17t3Tt2lUKFy5sjq9hw4by9ddf+xxzp9t89NFHzfmKjo52vz5x4kSpWLGi7Zz7Gvd56dIlGTFihFSuXNkcu5Z36NChZnmg5XT57bffTJektjbpehUqVJD+/ftLfHy8e52//vpLHn/8cff51f1ry7n+QyNYvvrqK7n99tvdx6HvwZdeekkSEhJs6+k50TLt2LHDvI/0/aTHo/Wm9Bw3adLEnEt9HyxdujTNnzU9r/q+LlasmOTPn9+833/99VefnwWtV92f7le3d88992TKGEs9fzoUBUhNtlTXABB0GpgcP37cXJQ9rVu3zlzckvsC1wuWjhkaPXq0bNiwwbRK/vnnn/LRRx/Z1mvQoIHZh27vjjvuSPV49EKrgemTTz5pLnreAa3L1KlT5eGHHzZBrwYEBw4cMBdFDX40OPAMyFq1aiVHjx41F9aSJUuabq0VK1Yk2eby5culQ4cO5pg1uNEWyw8//ND8vQZCGhAlR4MDLd+yZctMt7zu6+zZs7JkyRITpGkQoedBj1H3rYFO3bp1TaD61FNPmeDnzTfftG1TA+u5c+eaC7pe9PUc33333XL48GFzYdcgJV++fPL555+bAMTT7NmzTZ1qHaoffvhBbrrpJhN4P/PMMyZY1r/r3LmzGXbQpUsX29/rPjXYGD58uLsFcdKkSSaYa9asmQlGNKjQvy9UqJAtiNRgTMupx6/DGKpVqyY7d+405dPxrhoMBlJOdeTIEXP+NQDUbV5//fXmnGmwdeHCBfM+0Z96HnS5vjd0jK2+74YNG2bq/6233pJg0CBaz7u2yOtPfd/oedJ/zOiwDE/6mdD3hb4nNBDTc6i/f/rpp+Z9+8gjj0j37t3N32nwrmN+9RwE+lnTHoFPPvnEbEs/E3pM+v7wNQ5Zz4keg9aZ1qEekwazcXFxJohNie7XOxD2RbeT2rYAv1kAMtzHH3+sTVfW1KlTbcujo6Otu+++O8n6I0aMMOt36tTJtvzRRx81y7dv325bfuTIEbP81VdfTfE4VqxYYdarWLGideHCBZ+v6U8VHx9vFS9e3Kpbt6516dIl93rvv/++Wa958+buZWPHjjXL5s+f7172999/W9dff71tm4mJiVaVKlWsdu3amd9d9FgqVKhgtW3bNsXjnzZtmtneuHHjkrzm2p4eg67z8ssv217v2rWrFRUVZe3bt8+9TNfLkSOHbZmeW13+9ttvu5fdd9995lxcuXLFvezo0aNWlixZrJEjR7qXtW7d2qpVq5Z18eJF23HdeOONptwuH374odnHzTffbNumnuciRYpYjRo1si5fvuxePn369CTnXN9Tuv81a9bYyjl58mSz7tq1awMuZ8+ePc02N23alOz5femll6y8efNaP/30k+31Z555xsqaNat1+PBhK1Cu83Hw4EH3Mu/3p3r44YetPHny2M6vnhP925kzZ7qX7d692yzTsmzYsMG9fPHixWa57i/Qz1psbKx5rss9de/e3SzX7aR07OvXrzfrffTRR6mej3Llypl1U3t47tMft99+u9k24AtdzEAG0y7HmJgYadq0qfTq1cv22u+//25ahpKjf+fpscceMz+/+eYb23LXNvxN1aHHoV1fKdm8ebOcOHHCtL54tjA++OCDSSbUaFeltpppi5aLdq/27dvXtl5sbKzs3bvXtMBo2fV49aGtZzqpRyfzpNRNqa1wRYsWdZ8H725U17nR2ZoDBw60va5dzhorLVy40La8TZs2puXRpXbt2qabUVtLXbp162bOhWdXubaq6bHqa0q797VFSVuitFXTVTYtZ7t27Uy5tdXNk54fPVbPc67r6/Js2f6/w6dHjx5J3idz5swxrYbayufalz60JVZ5t96mVk4ti7Y6duzY0XSLJ3d+db/auqnH47lf3b62emkdBoPn+9N1PnW/2oKpnylP2sKorXUu2rWrY3P1/Gj3sovrd8+69fez5vrp/b7SFsqUjl1TUWmdare3HtPWrVtTLbu2fGqreGqPnj17protwF90MQMZ6NixY6YLSgMqDSg8gwEX73Fx3mMXPekFXrtkvccyubbhuoinRseVpUbHUfk6Bu0O1/Fx3uvqsXnvXy+KnjRIUt6BsqfTp08nGzTrOEO9+HsGT76OW8ddeXcharDgWS4X7SL1pvvXbj4X13hJ7VLWQFbp79p9rWMh1b59+0w9PP/88+bhiwaZGkgnVw+uY/M+b1pe7/Qkei51TKt2USe3r0DKefLkSdN96+ouT47uV8f7+bvftNLueh0nq0G3Hpf3e8STduN6v/e0vjyHQbiWKc+69fezpnWjzz2DbKXvR2+aDUC7qnXohP6jwPMz7n3svugwBSCjESACGUQvBDrWTsdz6dg6XykmdOyXr4tVcpILAF3b0NY1f6TWepheXK2DOhZMgytftDUoI/kK2pXnRV0nSeg4wHnz5pmk5DqeVCfV6AxR77LpuE5tMfTFO/C7mnrQ/dWqVUvGjRvn83Xv4Mifcvq7X50UpZNhfHEFzFdDPzM6zlFbOEeOHGmCMm2R1ta3p59+Okkrc3Jlu5oy+/uPLV+09VGDQ21d1J4DDUx1e9rK6c9EHg3W/RmDqJ+VjP68IHIRIAIZQGc/aledThbQWZPVq1f3uZ52Dx48eDDF1hrPViZtpdILjHdrkmsbrlayYChXrpz7GFzdlq4uM92fZ8oMXVcH3+uF1/PCqsfrydX6ohd+7ZIMlP69zoDWY0huYo8ei55z7Zb0bEV0dUu6yhUo7UqeMWOGmSCjLXdaVlf3snK1qupxpaVsnsem561ly5bu5ZoiSVuytFvY81xoiiRt0byaYMZFWwS1Xnzl6vSk+z137lyay+gP7crXblmdVOM5wz+lz8rVSu2zpnWjz12t2C579uxJsi3tLdBW8rFjx9q+EzTw9UejRo2StHT7opO8uIMLgoUxiEA603/5a+Cwfv16M15LWxCSo6/pBdk7LYlnuhNPb7/9tvmpLZOetmzZYoKElPYVKB2HpkHD5MmTbSlOdHap94VOW8y0K80znYteEKdMmWJbT2cua4ChKWc0yPDVcpISnXWrY9HeeeedZFuFbrvtNlMH3uvo7F49R97nzl8aEOnsbe1a1ofO9vUMKDRVjc5S1dQ3Ops30LK5zrm2Kut506DQc0yad0uzjnXUc+59jl1dnIHmVdTuU20l/fe//23GQiZ3fnW/+t7WmeHe9H3hedxp5Wr582zp0/dgMG4pmZzUPmuunzq72ZOvWdt6/N6tlLo9f1oFFWMQkRloQQTSmU6G0EBJWxB14oKmxfB0//33u3+/8847TcoZzdWmya69aYuJTvzQMXB6UXal2PBOeKsD1nXckitdSTBoS9jLL79sUploC6IGvXo82nXmPQZR19GA7L777jOpZzQ3oV7ktFtQuVq4NAj54IMPzMVW08PoXWB0TJ4GOjqpQluwNEBJjg7K17Qjmvpk48aNZtKCBkLaYqjpW/R86nnX1rfnnnvOtLrpudLbjWlePe3y8x5DFsj50HyLmkNS96lBrq8g4+abbzZdvzrRRM+Tdkdr3Wm+PG3xS4lOBtIWIe2i1HOuwZiWQYNy7zGeDzzwgEmho5OI9Nxp/WsAoi2lulwDOF+TTVKiXeZ6rrR715U6R4Nd/YeOpsnRSRaaLkjf35pWRicsadCv50NT7GjLmR6va6iDvq6trvq+CeQWb5pCRsdHaiucTgrRcn/88ccBd4cHIrXPmg6J0Pe3Bqk6fESPUVuTvVvJlZ4bPV7tWtbeA92evkf9/XwGcwyijhd1/cNNj1WPXT/XSsumnxfA8Dm3GUDQuNJuJPfwVrt2batPnz62Za7UG3FxcSY9S/78+a1ChQpZAwYMMOljPP31118mhckHH3yQ6rG5UtnMmTMn2ddcKWlc3n33XZOCJmfOnFbDhg2t1atXmzJ6plxRBw4cMGk0cufObRUrVsx64oknrC+//NJs0zPViNq2bZt11113mZQuul1NvXHvvfday5YtS7UMmkLkueeeM8eUPXt2q2TJkuYc7d+/373O2bNnrcGDB1ulSpUy62iKmddff92WWkfpscXExCTZhx5Pr169kixfsmSJ+RtNl/PLL7/4PD49Dk0Xo8el+y5durR1xx13WF988UWStC6+0smoCRMmmGPQc9O4cWOTsqZBgwZW+/btbetpKiJNbVSjRg2zrr5HdL0XX3zROn36dJrKeejQIXP8Woe6TU2JpH/rmepIz++wYcOsypUrm/de0aJFTSqfN954wxyTi6Zw0vfDn3/+6bOcKaW50TLfcMMN5u+1HocOHepOU+P5HtX3oZbfV9n0/ejN+1wE8lnT5wMHDjTvW03107FjR/M+8E45o+Xt3bu3OS/58uUzaZ009U5y76v05Dq3vh4ZfSwIbVH6H2JlIHRoS4Om2NCExdpCEyjt4nrttdfM2KjMmnyS0rFpsmdtPfOcvYvA6Ng37e7XFkxfXcqhqkSJEqbV1zuxNYDQwxhEIMRojjtNQeI9BsofOllDZ7FqOpDMDg513JsnHYOo4/E0fQjBof/0vHn/O1671XW4gvet9kKZpqnR94TOOgYQ+mhBBJAudFyhBro6VkvHOekYLg0SdCyijuWC/zN4tdVVbxmnY9Y0tYve8lDHA+pkpORuiwgAV4NJKgDShc5k1gkoGhDqZAkdnK8TOjxTwSB1OplDcxjqbFltNdSZ09pNO2bMGIJDAOmGFkQAAADYMAYRAAAANgSIAAAAsGEMYianqjhy5Ii5/Vcwbo0FAACQEs2KoLceLVWqlLlZQXIIEDORBoc6+BwAACAj/fLLLxIdHZ3s6wSImUhbDl2VpLcUS69WSr3nqybVTelfCpHEiWVWlNs59U1dO6eunVrfTixzRpX7zJkzpnHKFYMkhwAxE7m6lTU4TM8AURPt6vad8iFzYpkV5XZOfVPXzqlrp9a3E8uc0eVObWibc846AAAA/EKACAAAABsCRAAAANgQIAIAAMCGABEAAAA2BIhX6T//+Y9UrVpVqlSpIh988MHVbg4AACDTkebmKly5ckWGDBkiK1askIIFC0qDBg2kS5cuUqRIkeDVEAAAQAajBfEqbNy4UWrUqCGlS5eWfPnySYcOHeTbb78NXu0AAABkAkcHiKtXr5aOHTua+xFqwsj58+cnWWfixIlSvnx5yZUrlzRp0sQEhZ63ytPg0EV//+233zLs+AEAANKDowPE8+fPS506dUwQ6Mvs2bNNF/KIESNk69atZt127drJiRMnJBycOHtRFu06Jpt/OZPZhwIAAMKIo8cgapewPpIzbtw46du3r/Tu3ds8nzx5sixYsECmTZsmzzzzjGl59Gwx1N8bN26c7PYuXbpkHp73Q3TdWkcfwbbr19Py6Mxtcn3xPNK+XkVxCj2XlmWlyzkNZZTbOfVNXTunrp1a304sc0aV299tOzpATEl8fLxs2bJFhg0b5l6m90Vs06aNrF+/3jzXYHDXrl0mMNRJKgsXLpTnn38+2W2OHj1aXnzxxSTL9cbceu/FYPvrr9PmZ0LCFdPq6ZT7Weqb//Tp0+ZD5pQyK8rtnPqmrp1T106tbyeWOaPKffbsWb/WI0BMxqlTpyQhIUFKlChhW67Pd+/e/d+Tly2bjB07Vlq2bGkqdejQoSnOYNZgU7usPVsQy5QpI8WKFTM35g62a/787424s2bNJsWLF3fMh0zrQseU6nl1SpkV5XZOfVPXzqlrp9a3E8ucUeXWORX+IEC8Sp06dTIPf+TMmdM8vOmbID3eCFFZ/hsgWum4j1ClHzCnlVlRbufUN3XtnLp2an07scwZUW5/t+ussx6AokWLStasWeX48eO25fq8ZMmSEg7+Gx4CAAAEhgAxGTly5DCJr5ctW2Zr+tXnTZs2DfA0AwAAhA9HdzGfO3dO9u3b535+8OBBiY2NlcKFC0vZsmXNeMFevXpJw4YNzYSUt956y6TGcc1qBgAAiESODhA3b95sJpi4uCaQaFA4ffp06datm5lhPHz4cDl27JjUrVtXFi1alGTiSqjT2VAAAAD+cnSA2KJFi1SDpwEDBphHuA50BQAACBRjEAEAAGBDgAgAAAAbAkQAAADYECBGMEYgAgCAtCBABAAAgA0BIgAAAGwIEB2ANIgAACAQBIgRjDSIAAAgLQgQAQAAYEOACAAAABsCRAfgTswAACAQBIgRLIpMiAAAIA0IEAEAAGBDgOgAdDEDAIBAECACAADAhgAxE0ycOFGqV68ujRo1Stf9kAcRAACkBQFiJoiJiZG4uDjZtGlTZuweAAAgRQSITsC99gAAQAAIEAEAAGBDgBjBojL7AAAAQFgiQAQAAIANAaIDkAcRAAAEggARAAAANgSIkYxBiAAAIA0IEAEAAGBDgOgApEEEAACBIEAEAACADQFiBItiECIAAEgDAkQAAADYECA6AHkQAQBAIAgQAQAAYEOAGMGiyIMIAADSgAARAAAANgSIDkAeRAAAEAgCRAAAANgQIEYwhiACAIC0IEAEAACADQGiI5AJEQAA+I8AMYJFkecGAACkAQEiAAAAbAgQAQAAYEOAmAkmTpwo1atXl0aNGmXI/hiBCAAAAkGAmAliYmIkLi5ONm3alK77YQgiAABICwJEAAAA2BAgOgC32gMAAIEgQAQAAIANAWIE41Z7AAAgLQgQAQAAYEOA6ACkuQEAAIEgQAQAAIANAWIEIw8iAABICwJEAAAA2BAgOgGDEAEAQAAIEAEAAGBDgBjRyIQIAAACR4AIAAAAGwJEB7AYhAgAAAJAgAgAAAAbAsQIRh5EAACQFgSIAAAAsCFAdACLPIgAACAABIgAAACwIUCMYGRBBAAAaUGACAAAABsCRAdgCCIAAAgEASIAAABsCBAjWBSJEAEAQBoQIAIAAMCGANEByIMIAAACQYAIAAAAGwLETDBx4kSpXr26NGrUKF33Qx5EAACQFgSImSAmJkbi4uJk06ZNmbF7AACAFBEgAgAAwIYAEQAAADYEiBGMNIgAACAtCBABAABgQ4DoABZ3YwYAAAEgQAQAAIANAWIEiyITIgAASAMCRAfgVnsAACAQBIgAAAAIToAYHx8ve/bskStXrqR1EwAAAIiEAPHChQvSp08fyZMnj9SoUUMOHz5slj/22GMyZsyY9DhGpBF5EAEAQIYEiMOGDZPt27fLypUrJVeuXO7lbdq0kdmzZ6fpIJC+LE4wAAAIQDYJ0Pz5800geMMNN0iURxOVtibu378/0M0BAAAg3FsQT548KcWLF0+y/Pz587aAEQAAAA4JEBs2bCgLFixwP3cFhR988IE0bdo0uEcHAACA0O9iHjVqlHTo0EHi4uLMDObx48eb39etWyerVq1Kn6PE1WEQIgAASM8WxJtvvlliY2NNcFirVi359ttvTZfz+vXrpUGDBoFuDgAAAOHegqgqVaokU6ZMCf7RAAAAIPxaEDWdzfTp0+XMmTPpc0QIGuYMAQCADAkQNZ2N5kIsWbKk3HPPPfLVV1/J5cuX07RzZAyGIAIAgHQNEHVSym+//WbyIebNm1d69uwpJUqUkH79+jFJBQAAwKn3Ys6SJYvceuutpqv5+PHj8t5778nGjRulVatWwT9CAAAAhP4kFZdjx47JrFmz5JNPPpEdO3ZI48aNg3dkuGpRQuJyAACQAS2IOjnlww8/lLZt20qZMmVk0qRJ0qlTJ9m7d69s2LAhDYeA9GZZjEIEAADp2IKo4w0LFSok3bp1k9GjR5s7qwAAAMDBAeLXX38trVu3NuMQAQAAEHkCjvK0a5ng0LcuXbqY1tWuXbtKKCAPIgAASLcWxPr168uyZctM8FOvXj2JSiHy2Lp1qzjVoEGD5KGHHpIZM2ZIKGEEIgAACHqAeOedd0rOnDndv6cUIDpZixYtZOXKlZl9GAAAAOkfII4YMcL9+wsvvCDBpom3n376aVm4cKFcuHBBKleubGZKB2sCzOrVq+X111+XLVu2yNGjR2XevHnSuXPnJOtNnDjRrKfpe+rUqSNvv/02qXsAAIDjBDwGsWLFivL7778nWf7XX3+Z1wL1559/yk033STZs2c3AWJcXJyMHTvWdGf7snbtWp+39tO/06Tdvpw/f94EfBoAJmf27NkyZMgQEwxrN7mu365dOzlx4oR7nbp160rNmjWTPI4cOSKhiIZeAACQIbOYf/75Z0lISEiy/NKlS/Lrr78GfACvvvqqyaeoLYYuFSpU8LluYmKixMTESJUqVUyC7qxZs5rle/bsMXdx0QBv6NChSf6uQ4cO5pGScePGSd++faV3797m+eTJk2XBggUybdo0eeaZZ8yy2NhYCUeMQQQAAOkSIGp6G5fFixdLwYIF3c81YNRJLMkFdqltV1vq7rnnHnMv59KlS8ujjz5qgjVvOnv6m2++kVtuucXcA/rjjz+WgwcPmuBQu4x9BYf+iI+PN93Pw4YNs+2rTZs2sn79egk2bcnUh69AGwAAIGwCRNeYPZ2g0qtXL9tr2j1cvnx50zUcqAMHDpi7sWjr37PPPiubNm2SgQMHSo4cOZLsR5UqVUqWL18uzZo1k+7du5sATgM53UZanTp1ygRrmgTckz7fvXu339vR49i+fbvp0o6OjpY5c+ZI06ZNk6ynraD60LvSeAbawcat9gAAQLoGiNq9q7SVUIO4okWLpmmHvrark1FGjRplnmsanV27dpkuXl8BoipbtqxpPWzevLkZ9zh16tSQmFm9dOnSzD4EAACAjJ+kol26wQoO1bXXXivVq1e3LatWrZocPnw42b/RySj9+vWTjh07mlnPgwcPvqpj0PLoeEbvSS76vGTJkhL2GIQIAADSM0DU7t8JEyYkWf7OO+/I448/HujmzAxmnWTi6aeffpJy5col2x2st/rTIHLu3Llm7KPOQH7yySclrbQ7u0GDBmZbni2b+txXFzEAAEAkCzhA/PLLL01Q5+3GG2+UL774IuAD0Na/DRs2mC7mffv2ycyZM+X99983Y/S8adCms5E1eNSgMFu2bKb1ccmSJWYW9JtvvulzH+fOnTMzkF2zkLUVVH/3bKXUMZBTpkwxd0H58ccfpX///mYsoWtWczgKgV53AADghDQ3mgPR18SKAgUKmNa9QDVq1MgkrtYZxCNHjjRjHN966y3p0aNHknV1ZrEGkjpBRVv9XDRnoY7/K1asmM99bN68WVq2bGkLBpWOcZw+fbr5vVu3bnLy5EkZPny4SZStOQ8XLVqUZOJKOKKHGQAApGuAqHc50cBpwIABtuWa5DotibLVHXfcYR7+aNu2rc/lOrklpVvgWVbqYZKWybtcAAAAThNwgKitbxpEaWub5h9UOlZPU9xoyx8AAAAcFiA+9NBD5q4pr7zyirz00ktmmeZA1DyEmrwaoYMhiAAAIEMCRKUTOPShrYi5c+eWfPnypWnnyBj+dK8DAACkeRazunLlipkUomlmXMHHkSNHzGxhAAAAOKwF8dChQ9K+fXuTIka7mnXSSP78+eXVV181z/UOKAAAAHBQC+KgQYPMrfH+/PNP073s0qVLF1uiaWQ+8iACAIAMaUFcs2aNrFu3zpaH0DVR5bfffkvTQSB9MQIRAACkawui3s0kISEhyfJff/3VdDUDAADAYQHirbfeast3GBUVZSanjBgxQm677bZgHx8AAABCvYtZE2K3a9fO3AP54sWL0r17d9m7d68ULVpUPvvss/Q5SqQRmRABAEAGBIjR0dGyfft2mTVrluzYscO0Hvbp08fcO9lz0gpCCIMQAQBAeifKzpYtm9x///1p+VMAAABEQoD49ddfS4cOHSR79uzm95ToXVWuv/56KVWqVLCOEQAAAKEWIHbu3FmOHTsmxYsXN7+nJmvWrPLaa6/J4MGDg3GMSCPyIAIAgHSbxaypbTQ4dP2e0kMnrkyZMsUEiAgNDEEEAADpPgYxJZpA++677zYTWAAAAOCQAHHPnj3y9ttvy48//mieV6tWTQYMGGDGHipNmD1u3LjgHikAAABCM1H2l19+KTVr1pQtW7ZInTp1zGPr1q1Sq1Yt8xpCB1kQAQBAhrQgDh06VIYNGyYjR460Ldc7qehr2r2M0GIxCBEAAKRnC+LRo0elZ8+eSZZrXkR9DQAAAA4LEFu0aCFr1qxJsvy7776TZs2aBeu4AAAAEOqJsl06deokTz/9tBmDeMMNN5hlGzZskDlz5siLL76YfkeKgEWRCBEAAKRnomxv7777rnl4iomJkUceeSQtx4F0ZJEJEQAABDtA1ATYAAAAcIaAxyACAAAgsqUpQFy1apV07NhRKleubB46LtHXxBVkLvIgAgCADAkQP/nkE2nTpo3kyZNHBg4caB65c+eW1q1by8yZM9N0EEhfpEEEAADpmij7lVdekddee00GDx7sXqZBot5a76WXXpLu3bsHukkAAACEcwvigQMHTPeyN+1mPnjwYLCOCwAAAOESIJYpU0aWLVuWZPnSpUvNawgd7jSI9DEDAID07GJ+4oknTJdybGys3HjjjWbZ2rVrZfr06TJ+/PhANwcAAIBwDxD79+8vJUuWlLFjx8rnn39ullWrVk1mz54td955Z3ocIwAAAEI5QFRdunQxD6TNxIkTzSMhIYFTCAAAQg6JsjOB3pIwLi5ONm3alK77ifpfJkSGIAIAgEAQIAIAAMCGABEAAAA2BIgAAAAIToAYHx8ve/bskStXrqR1E8igPIgWgxABAEB6BogXLlyQPn36mHsx16hRQw4fPmyWP/bYYzJmzJhANwcAAIBwDxCHDRsm27dvl5UrV0quXLncy9u0aWNyIQIAAMBheRDnz59vAsEbbrhBotz3chPTmrh///5gHx8AAABCvQXx5MmTUrx48STLz58/bwsYETosMiECAID0DBAbNmwoCxYscD93BYUffPCBNG3aNNDNAQAAINy7mEeNGiUdOnQwdwLRGczjx483v69bt05WrVqVPkcJAACA0G1BvPnmmyU2NtYEh7Vq1ZJvv/3WdDmvX79eGjRokD5HCQAAgNBtQVSVKlWSKVOmBP9oEFTuIaHkQQQAAOkZIJ45c8bnch2LmDNnTsmRI0egmwQAAEA4B4jXXHNNirOVo6Oj5cEHH5QRI0ZIlizcyQ8AACDiA8Tp06fLc889Z4LAxo0bm2UbN26UGTNmyL/+9S+TBueNN94wrYnPPvtsehwzAAAAQilA1EBw7Nixcu+997qXdezY0UxYee+992TZsmVStmxZeeWVVwgQM5mrpZchiAAAIBAB9wFrOpt69eolWa7LdCaza6az6x7NAAAAiPAAsUyZMjJ16tQky3WZvqZ+//13KVSoUHCOEAAAAKHdxazjC++55x5ZuHChNGrUyCzbvHmz7N69W7744gvzfNOmTdKtW7fgHy0AAABCL0Ds1KmTCQbff/992bNnj1mmd1aZP3++lC9f3jzv379/8I8UASMNIgAAyLBE2RUqVJDRo0enaYcAAACIgABxx44dfm+wdu3aV3M8AAAACIcAsW7duiZlimVZtiTZ+lx5LktISEiP4wQAAEAozWI+ePCgHDhwwPz88ssvTRfzu+++K7Gxseahv+v9mfU1hA5X3P6/OB4AACB4LYjlypVz/64zmCdMmCC33XabrVtZU9w8//zz0rlzZ//2DAAAgMjIg7hz507TguhNl8XFxQXruAAAABAuAWK1atXMDOb4+Hj3Mv1dl+lrCB1R7kQ3AAAA6ZjmZvLkyebey9HR0e4ZyzrLWSeq/Pvf/w50c8gAFndjBgAA6RkgNm7c2ExY+fTTT03CbKV3TenevbvkzZs30M0BAAAgEhJlayDYr1+/4B8NAAAAwiNA/Prrr83t9LJnz25+T+1WfAgN7vSUpLkBAADBDhA1dc2xY8ekePHiKaax0XGIJMoGAABwQICYmJjo83cAAAA4PM3N5cuXpXXr1rJ37970OyIAAACET4CoYxA1pQ3CA0MQAQBAhiTKvv/++2Xq1Klp2hkAAAAiMM3NlStXZNq0abJ06VJp0KBBktyH48aNC+bxAQAAINQDxF27dkn9+vXN7z/99FOSWcwAAABwWIC4YsWK9DkSBN//4nWLPIgAACA9xyACAAAgshEgAgAAwIYAEQAAADYEiEHUpUsXKVSokHTt2lVCQdT/BiEyBBEAAASCADGIBg0aJB999FEwNwkAAJDhCBCDqEWLFpI/f/5gbhIAAMDZAeKYMWNMLsXHH388qNtdvXq1dOzYUUqVKmW2P3/+fJ/rTZw4UcqXLy+5cuWSJk2ayMaNG4N6HAAAAOEgZALETZs2yXvvvSe1a9dOcb21a9fK5cuXkyyPi4uT48eP+/yb8+fPS506dUwAmJzZs2fLkCFDZMSIEbJ161azfrt27eTEiRPuderWrSs1a9ZM8jhy5IiEIvKWAwCAsA0Qz507Jz169JApU6aYSR7JSUxMlJiYGOnevbskJCS4l+/Zs0datWolM2bM8Pl3HTp0kJdfftlMIkmO3iKwb9++0rt3b6levbpMnjxZ8uTJY24r6BIbG2vuJOP90JbJQGigqvto1KhRQH8HAADgmABRg77bb79d2rRpk+J6WbJkkW+++Ua2bdsmPXv2NAHj/v37TXDYuXNnGTp0aJr2Hx8fL1u2bLHtX/elz9evXy/pUV5t8dRWUwAAgLC/1V6wzZo1y3Tp+hssaWvd8uXLpVmzZqYlUQM4DeQmTZqU5mM4deqUaZEsUaKEbbk+3717t9/b0ePYvn276dKOjo6WOXPmSNOmTdN8XAAAAI4LEH/55ReTGmbJkiVmYoi/ypYtKx9//LE0b95cKlasKFOnTjWTTzLb0qVLJZR4nhGLGzIDAIBw6GLWbl2dBFK/fn3Jli2beaxatUomTJhgfvccZ+hJJ6P069fPzEy+cOGCDB48+KqOo2jRopI1a9Ykk1z0ecmSJa9q2wAAAOEmUwPE1q1by86dO83kD9ejYcOGZsKK/q5Bm6/uYP27atWqydy5c2XZsmVmBvKTTz6Z5uPIkSOHNGjQwGzLRcc36nO6iAEAgNNkahezJpXWNDGe8ubNK0WKFEmy3BW06YzkcuXKmaBQWxl1NrB2UetEldKlS/tsTdRZ0vv27XM/P3jwoAlACxcubLqrlaa46dWrlwlQGzduLG+99ZYZS6izmgEAAJwk0yepBEJnFo8aNcpMUNFWPxfNWajj/4oVK+bz7zZv3iwtW7Z0P9dgUGlAOH36dPN7t27d5OTJkzJ8+HA5duyYyXm4aNGiJBNXwonnuEyGIAIAgLANEFeuXJni623btvW5vF69eineAs+fSRoDBgwwDwAAACcLiTyIAAAACB0EiAAAALAhQIxgtjyImXgcAAAgvBAgAgAAwIYAEQAAADYEiA7BrfYAAIC/CBAjWAjcnhoAAIQhAkQAAADYECACAADAhgDRIUhzAwAA/EWAGMGibJkQAQAA/EOACAAAABsCRAAAANgQIDqExSBEAADgJwLESMYQRAAAkAYEiAAAALAhQAQAAIANAaJDWGRCBAAAfiJAjGDcixkAAKQFASIAAABsCBABAABgQ4DoFORBBAAAfiJAjGCkQQQAAGlBgAgAAAAbAkQAAADYECA6BEMQAQCAvwgQI1gUiRABAEAaECACAADAhgARAAAANgSIDmExCBEAAPiJADGCkQcRAACkBQEiAAAAbAgQAQAAYEOA6BAWmRABAICfCBAjGGkQAQBAWhAgAgAAwIYA0SFIcwMAAPxFgBjBojwS3ZAGEQAA+IsAMYJl8ajdRJoQAQCAnwgQI1gWj1kqxIcAAMBfBIgOCRBpQQQAAP4iQIxgWTzutZeYyChEAADgHwLECBZla0HM1EMBAABhhADRIa2IFoMQAQCAnwgQHTIOkRZEAADgLwJEh7QgMkkFAAD4iwAxwmX5X4RIDzMAAPAXAaJjupiZpQIAAPxDgOiQLuYEAkQAAOAnAsQg6tKlixQqVEi6du0qoZbqhkkqAADAXwSIQTRo0CD56KOPJBS7mC0iRAAA4CcCxCBq0aKF5M+fX0IJs5gBAEDYBYiTJk2S2rVrS4ECBcyjadOmsnDhwqDuY/Xq1dKxY0cpVaqU6XKdP3++z/UmTpwo5cuXl1y5ckmTJk1k48aNEu7IgwgAAMIuQIyOjpYxY8bIli1bZPPmzdKqVSu588475YcffvC5/tq1a+Xy5ctJlsfFxcnx48d9/s358+elTp06JgBMzuzZs2XIkCEyYsQI2bp1q1m/Xbt2cuLECfc6devWlZo1ayZ5HDlyREKV6257zGIGAAD+yiaZTFv2PL3yyiumVXHDhg1So0YN22uJiYkSExMjVapUkVmzZknWrFnN8j179pjAUgO8oUOHJtlHhw4dzCMl48aNk759+0rv3r3N88mTJ8uCBQtk2rRp8swzz5hlsbGxEm7cYxDJcgMAAMKlBdFTQkKCCfy0xU+7mr1lyZJFvvnmG9m2bZv07NnTBIz79+83wWHnzp19Bof+iI+PNy2Ybdq0se1Ln69fv16CTVsyq1evLo0aNZL0lvV/gxBpQQQAAGHTgqh27txpAsKLFy9Kvnz5ZN68eSaA8kXHES5fvlyaNWsm3bt3NwGcBnLa6phWp06dMsFpiRIlbMv1+e7du/3ejh7H9u3bTYCrXedz5szxGehqK6g+zpw5IwULFpSM6WJO190AAIAIEhIBYtWqVU337enTp+WLL76QXr16yapVq5INEsuWLSsff/yxNG/eXCpWrChTp0515/vLTEuXLpXQ7WImQgQAAGHUxZwjRw6pXLmyNGjQQEaPHm0miIwfPz7Z9XUySr9+/cz4xQsXLsjgwYOvav9FixY14xm9J7no85IlS0o4I80NAAAIywDRm44tvHTpUrLdwa1bt5Zq1arJ3LlzZdmyZWYG8pNPPnlVAaoGp7otz2PQ5766iMMJd1IBAABh18U8bNgwM8NYu43Pnj0rM2fOlJUrV8rixYuTrKtBm65brlw5ExRmy5bNdEMvWbLETFQpXbq0z9bEc+fOyb59+9zPDx48aLq0CxcubPardAa0dm03bNhQGjduLG+99ZYZS+ia1RyuXF3MCQxCBAAA4RIgap5BnZF89OhRM2FDk2ZrcNi2bdsk6+rM4lGjRpkJKtrq56Jd0jr+r1ixYj73ofkVW7Zs6X6uwaDSgHD69Onm927dusnJkydl+PDhcuzYMZPzcNGiRUkmroRrFzNjEAEAQNgEiDrBJBC+AkdVr169FG+B50+ANGDAAPOIJNxJBQAARMQYRAQPd1IBAACBIkCMcP+fKDuzjwQAAIQLAsQIRx5EAAAQKALECMedVAAAQKAIECNczmxZzc8L8Vcy+1AAAECYyPRZzEhf1xbIZX7O2vSLuzkxSv8XpT//v4UxkiQmWnL6zGkpeCJRsrjy/DgA5XZOfVPXzqlrp9a3E8vsKveZM6fl3uLFJbMRIEa4KiXyiewU+W7f7+YBAABCV9YsIvfeWDWzD4MAMdI9eGM5OfbHafn1bKJcupL434WW/t8SP1JDhiUt1uXL8ZI9ew7TSuoUlNs59U1dO6eunVrfTiyzq9yJVy5LKKAFMcLlz5VdBt1SRooXL27uROMEektGvUOPk8qsKLdz6pu6dk5dO7W+nVhmz3KHAuecdQAAAPiFABEAAAA2BIgAAACwIUAEAACADQEiAAAAbAgQAQAAYEOACAAAABsCRAAAANgQIAIAAMCGABEAAAA2BIgAAACw4V7Mmciy9LbcImfOnEnX+zqePXtWcuXK5Zj7WTqxzIpyO6e+qWvn1LVT69uJZc6ocrtiDlcMkhwCxEykbwJVpkyZzDwMAADgwBikYMGCyb4eZaUWQiJd/6Vw5MgRyZ8/v0RFRaXbvxQ0AP3ll1+kQIEC4gROLLOi3M6pb+raOXXt1Pp2Ypkzqtwa9mlwWKpUqRRbKWlBzERaMdHR0RmyL32jOelD5tQyK8rtHNS1szixvp1Y5owod0othy7O6dgHAACAXwgQAQAAYEOAGOFy5swpI0aMMD+dwollVpTbOfVNXTunrp1a304sc6iVm0kqAAAAsKEFEQAAADYEiAAAALAhQAQAAIANASIAAABsCBAj2MSJE6V8+fLmno5NmjSRjRs3SrgaPXq0NGrUyNx1pnjx4tK5c2fZs2ePbZ0WLVqYO9J4Ph555BHbOocPH5bbb79d8uTJY7bz1FNPyZUrVyRUvfDCC0nKdP3117tfv3jxosTExEiRIkUkX758cvfdd8vx48fDusxK37fe5daHljVS6nr16tXSsWNHczcDPf758+cnudvB8OHD5dprr5XcuXNLmzZtZO/evbZ1/vjjD+nRo4dJqHvNNddInz595Ny5c7Z1duzYIc2aNTPfA3qHhtdee01CtdyXL1+Wp59+WmrVqiV58+Y16/Ts2dPccSq198eYMWPCttzqwQcfTFKm9u3bh3V9p1ZmX59xfbz++uthW9ej/bhWBet7e+XKlVK/fn0z47ly5coyffr04BZGb7WHyDNr1iwrR44c1rRp06wffvjB6tu3r3XNNddYx48ft8JRu3btrA8//NDatWuXFRsba912221W2bJlrXPnzrnXad68uSnn0aNH3Y/Tp0+7X79y5YpVs2ZNq02bNta2bdusb775xipatKg1bNgwK1SNGDHCqlGjhq1MJ0+edL/+yCOPWGXKlLGWLVtmbd682brhhhusG2+8MazLrE6cOGEr85IlS/SWoNaKFSsipq71mJ577jlr7ty5pmzz5s2zvT5mzBirYMGC1vz5863t27dbnTp1sipUqGD9/fff7nXat29v1alTx9qwYYO1Zs0aq3LlytZ9993nfl3PSYkSJawePXqYz85nn31m5c6d23rvvfesUCz3X3/9Zeps9uzZ1u7du63169dbjRs3tho0aGDbRrly5ayRI0fa6t/zuyDcyq169epl6tOzTH/88YdtnXCr79TK7FlWfej1Kioqytq/f3/Y1nU7P65VwfjePnDggJUnTx5ryJAhVlxcnPX2229bWbNmtRYtWhS0shAgRij9Uo2JiXE/T0hIsEqVKmWNHj3aigQaQOgXzqpVq9zLNGgYNGhQsn+jH7IsWbJYx44dcy+bNGmSVaBAAevSpUtWqAaIekHwRS+m2bNnt+bMmeNe9uOPP5rzohfWcC2zL1qvlSpVshITEyOyrr0vnlrOkiVLWq+//rqtvnPmzGkugEovCvp3mzZtcq+zcOFCc4H97bffzPN3333XKlSokK3MTz/9tFW1alUrFPgKGrxt3LjRrHfo0CFb0PDmm28m+zfhWG4NEO+8885k/ybc69ufutbyt2rVyrYs3Ov6hNe1Kljf20OHDjWNB566detmAtRgoYs5AsXHx8uWLVtMl5TnfZ/1+fr16yUSnD592vwsXLiwbfmnn34qRYsWlZo1a8qwYcPkwoUL7te07Np1VaJECfeydu3amZuj//DDDxKqtFtRu2gqVqxoupe060FpHWuXnGc9a/dz2bJl3fUcrmX2fj9/8skn8tBDD5nupUiua5eDBw/KsWPHbHWr907VoSKedavdjA0bNnSvo+vrZ/377793r3PLLbdIjhw5bOdBu7z+/PNPCZfPuta7ltWTdjNqF129evVMl6Rn91u4llu7DLU7sWrVqtK/f3/5/fff3a9Fen1rF+uCBQtMt7m3cK7r017XqmB9b+s6nttwrRPMa3y2oG0JIePUqVOSkJBge3Mpfb57924Jd4mJifL444/LTTfdZIIDl+7du0u5cuVMMKVjUnQsk35JzJ0717yuF1xf58T1WijSgEDHlegF4+jRo/Liiy+asTa7du0yx6xfit4XTi2TqzzhWGZvOm7pr7/+MmO0IrmuPbmO0VcZPOtWgwlP2bJlMxciz3UqVKiQZBuu1woVKiShTMdqad3ed999Ztydy8CBA83YKy3runXrzD8Q9PMxbty4sC23jje86667zHHv379fnn32WenQoYO54GfNmjXi63vGjBlm3J6eA0/hXNeJPq5VwfreTm4dDSL//vtvM275ahEgIuzo4F4NkL777jvb8n79+rl/13996eD+1q1bmy/bSpUqSTjSC4RL7dq1TcCogdHnn38elC+AcDB16lRzHjQYjOS6hp22stx7771mss6kSZNsrw0ZMsT2udAL7sMPP2wmCITCLcrS4h//+IftPa3l0veytirqezvSTZs2zfSQ6ESTSKnrmGSuVeGCLuYIpN1u+i9O71lR+rxkyZISzgYMGCD/+c9/ZMWKFRIdHZ3iuhpMqX379pmfWnZf58T1WjjQf3Ved911pkx6zNr9qq1rydVzuJf50KFDsnTpUvnnP//pqLp2HWNKn2H9eeLECdvr2vWmM13Dvf5dwaHW/5IlS2yth8nVv5b9559/Dutye9IhJfpd7vmejtT6XrNmjekBSO1zHk51PSCZa1WwvreTW0c/K8FqPCBAjED6L6wGDRrIsmXLbE3d+rxp06YSjrQVQT9w8+bNk+XLlyfpUvAlNjbW/NTWJaVl37lzp+1L1nXxqV69uoQDTWmhrWRaJq3j7Nmz2+pZv2R1jKKrnsO9zB9++KHpVtN0D06qa31/6wXAs26160jHmnnWrV5kdEyTi3429LPuCph1HU01ogGX53nQIQuh2t3oCg517K3+40DHnqVG61/H4rm6YMOx3N5+/fVXMwbR8z0difXt6iXQ77M6deqEfV1bqVyrgvW9ret4bsO1TlCv8UGb7oKQS3OjMx6nT59uZr/169fPpLnxnBUVTvr3729SfqxcudKW7uDChQvm9X379plUCJoy4ODBg9ZXX31lVaxY0brllluSpA649dZbTfoBTQdQrFixkEp94u2JJ54wZdYyrV271qQ90HQHOjPOlS5BUygsX77clL1p06bmEc5l9px5r2XTGYmeIqWuz549a1JY6EO/iseNG2d+d83W1TQ3+pnV8u3YscPM8PSV5qZevXrW999/b3333XdWlSpVbGlPdMakpgB54IEHTNoN/V7Q1BiZme4lpXLHx8ebdD7R0dGm3jw/667Zm+vWrTOzWvV1TYfyySefmLrt2bNn2JZbX3vyySfNLFZ9Ty9dutSqX7++qc+LFy+GbX2n9h53panRY9RZut7Csa77p3KtCtb3tivNzVNPPWVmQU+cOJE0N/Cf5kXSN6HmQ9S0N5o7K1zpl4uvh+abUocPHzYBQuHChU1grPnB9IPjmRtP/fzzz1aHDh1MniwNtDQAu3z5shWqNG3Btddea+qwdOnS5rkGSC4aLDz66KMmzYN+WXTp0sV8GYVzmV0WL15s6njPnj225ZFS15rT0dd7WtOduFLdPP/88+bip+Vs3bp1knPx+++/mwAhX758JgVG7969zUXZk+ZQvPnmm8029D2kgWeolluDo+Q+664cmFu2bLGaNGliLsK5cuWyqlWrZo0aNcoWSIVbuTV40GBAgwBNgaKpXTTPp/c/6MOtvlN7jysN5PQzqoGet3Csa0nlWhXM7209v3Xr1jXXB/1Hsuc+giHqfwUCAAAADMYgAgAAwIYAEQAAADYEiAAAALAhQAQAAIANASIAAABsCBABAABgQ4AIAAAAGwJEAAAA2BAgAkAQtGjRQh5//PGQOpdRUVEyf/78zD4MAGGIO6kAQBD88ccfkj17dsmfP7+UL1/eBIsZFTC+8MILJhCMjY21LT927JgUKlRIcubMmSHHASByZMvsAwCASFC4cOGgbzM+Pl5y5MiR5r8vWbJkUI8HgHPQxQwAQexi1p+HDh2SwYMHmy5efbh899130qxZM8mdO7eUKVNGBg4cKOfPn3e/ri2PL730kvTs2VMKFCgg/fr1M8uffvppue666yRPnjxSsWJFef755+Xy5cvmtenTp8uLL74o27dvd+9Pl/nqYt65c6e0atXK7L9IkSJm++fOnXO//uCDD0rnzp3ljTfekGuvvdasExMT494XAOcgQASAIJo7d65ER0fLyJEj5ejRo+ah9u/fL+3bt5e7775bduzYIbNnzzYB44ABA2x/r8FZnTp1ZNu2bSYQVNptrUFfXFycjB8/XqZMmSJvvvmmea1bt27yxBNPSI0aNdz702XeNBBt166d6XLetGmTzJkzR5YuXZpk/ytWrDDHqj9nzJhh9usKOAE4B13MABDkruasWbOaoM6zi3f06NHSo0cP97jEKlWqyIQJE6R58+YyadIkyZUrl1muLXwa8Hn617/+ZWtlfPLJJ2XWrFkydOhQ0xqYL18+yZYtW4pdyjNnzpSLFy/KRx99JHnz5jXL3nnnHenYsaO8+uqrUqJECbNMA0hdrmW4/vrr5fbbb5dly5ZJ3759eZ8ADkKACAAZQLuAteXw008/dS+zLEsSExPl4MGDUq1aNbOsYcOGSf5WWxs1mNSWPe0SvnLliumCDsSPP/5oWiZdwaG66aabzP737NnjDhC1JVKDQxftatauaQDOQoAIABlAA7uHH37YjDv0VrZsWffvngGcWr9+vWl51HGG2kVcsGBB03o4duzYdDlOnYntSccxahAJwFkIEAEgyHTmcUJCgm1Z/fr1zRjCypUrB7StdevWSbly5eS5555zL9NJMKntz5u2UOpYQh2L6ApC165dK1myZJGqVasGdEwAIh+TVAAgyHSc4OrVq+W3336TU6dOuWcia7Cnk0I0X+HevXvlq6++SjJJxJuOVTx8+LBpNdQuZu1qnjdvXpL9aTe1blf3d+nSpSTb0VZIHefYq1cv2bVrl5mE8thjj8kDDzzg7l4GABcCRAAIMp3B/PPPP0ulSpWkWLFiZlnt2rVl1apV8tNPP5lUN/Xq1ZPhw4dLqVKlUtxWp06dTMocDSTr1q1rgkzX7GYXnRmtM6Rbtmxp9vfZZ58l2Y6myFm8eLFJ6N2oUSPp2rWrtG7d2kxIAQBv3EkFAAAANrQgAgAAwIYAEQAAADYEiAAAALAhQAQAAIANASIAAABsCBABAABgQ4AIAAAAGwJEAAAA2BAgAgAAwIYAEQAAADYEiAAAABBP/wei4gO9D5yx1AAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "w_ridge1, hist_r1 = gradient_descent(ridge_loss_lam1, ridge_grad_lam1, X, y,\n", " w0=np.zeros(4), gamma=0.1, numIterations=2000)\n", "print(\"lambda = 1 weights:\", np.round(w_ridge1, 6))\n", "print(\"OLS weights :\", np.round(w2, 6))\n", "\n", "plt.figure(figsize=(7, 4))\n", "plt.semilogy(hist_r1)\n", "plt.xlabel(\"iteration\"); plt.ylabel(\"ridge objective\")\n", "plt.title(\"2(b) ridge convergence, lambda = 1\"); plt.grid(alpha=.3); plt.show()" ] }, { "cell_type": "markdown", "id": "5ecb2bae", "metadata": {}, "source": [ "### 2(b)(iv) Fit with $\\lambda = 100$\n", "\n", "Same thing with a much larger penalty. Run the cell, then compare the three rows of weights\n", "it prints and answer the question in the markdown cell below." ] }, { "cell_type": "code", "execution_count": 14, "id": "772843ef", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "lambda = 100 weights: [0.009203 0.009041 0.00947 0. ]\n", "lambda = 1 weights: [0.241794 0.238395 0.27735 0. ]\n", "OLS weights: [0.181782 0.282127 0.571263 0. ]\n" ] } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "def ridge_loss_lam100(X, y, w):\n", " \"\"\"compute_ridge_loss with lambda pinned to 100.\"\"\"\n", " return compute_ridge_loss(X, y, w, 100.0)\n", "\n", "\n", "def ridge_grad_lam100(X, y, w):\n", " \"\"\"compute_ridge_grad with lambda pinned to 100.\"\"\"\n", " return compute_ridge_grad(X, y, w, 100.0)\n", "\n", "\n", "w_ridge100, hist_r100 = gradient_descent(ridge_loss_lam100, ridge_grad_lam100, X, y,\n", " w0=np.zeros(4), gamma=0.004, numIterations=20000)\n", "print(\"lambda = 100 weights:\", np.round(w_ridge100, 6))\n", "print(\"lambda = 1 weights:\", np.round(w_ridge1, 6))\n", "print(\"OLS weights:\", np.round(w2, 6))" ] }, { "cell_type": "markdown", "id": "3b082fdd", "metadata": {}, "source": [ "**(iv)** At $\\lambda = 1$ the three midterm weights have already shrunk noticeably below\n", "their OLS values. At $\\lambda = 100$ they are squashed to nearly zero — the model has been\n", "penalized into predicting roughly a constant, and it barely looks at the midterms at all.\n", "\n", "That is what $\\lambda$ does: it sets the exchange rate between two competing goals, fitting\n", "the training data (the first term) and keeping the weights small (the second term). Turn\n", "$\\lambda$ up and the second goal wins. At $\\lambda = 0$ we would recover OLS exactly.\n", "\n", "The intercept does **not** shrink, because we deliberately left it out of the penalty. (Here\n", "it sits at zero anyway, since both sides were standardized.)\n", "\n", "Why would anyone want small weights? Small weights mean the prediction cannot swing wildly in\n", "response to a small change in any one input, which is a hedge against overfitting —\n", "particularly here, with 25 data points, 4 parameters, and three strongly correlated midterm\n", "columns. Notice also that the shrinkage is not uniform: EXAM3, which OLS leaned on hardest,\n", "gives up the most in absolute terms." ] }, { "cell_type": "markdown", "id": "d69f1309", "metadata": {}, "source": [ "---\n", "# Problem 3 — Parameter estimation for a nonlinear model\n", "\n", "Everything so far has been linear in the parameters, which makes the gradient easy and the\n", "loss surface a bowl with exactly one minimum. Real models are usually neither.\n", "\n", "You are given noisy measurements of the response of an underdamped second-order system:\n", "\n", "$$y = A\\,e^{-bx}\\sin(\\omega x + \\varphi)$$\n", "\n", "with four unknown parameters: amplitude $A$, decay rate $b$, angular frequency $\\omega$, and\n", "phase $\\varphi$. Estimate all four by gradient descent.\n", "\n", "The structure is the same as Problems 1 and 2, and you will reuse the *same*\n", "`gradient_descent` function. The only genuinely new thing is that the model is nonlinear in\n", "its parameters, so the four partial derivatives take real work to derive.\n", "\n", "Throughout Problem 3 the parameters live in one array `p = [A, b, omega, phi]`, playing\n", "exactly the role `w` played before.\n", "\n", "### What to turn in\n", "\n", "**(a)** Derive $\\partial e/\\partial A$, $\\partial e/\\partial b$, $\\partial e/\\partial \\omega$,\n", "$\\partial e/\\partial \\varphi$ by hand, for the MSE loss. You will need the chain rule and,\n", "for two of them, the product rule.\n", "\n", "**(b)** Implement the model and the gradient, and pass gradient checking.\n", "\n", "**(c)** Fit the model. Plot the loss vs iteration for three different learning rates that all\n", "converge.\n", "\n", "**(d)** Plot the data with your fitted curve overlaid.\n", "\n", "### Notes — read these, they will save you time\n", "\n", "- $\\varphi$ is only defined modulo $2\\pi$, and flipping the sign of $A$ while shifting\n", " $\\varphi$ by $\\pi$ gives the *identical* curve. If your parameters look different from a\n", " classmate's but your curves lie on top of each other, you are both right.\n", "- A single $\\gamma$ shared across four parameters of quite different scales will converge\n", " slowly. That is expected here." ] }, { "cell_type": "markdown", "id": "a5b59326", "metadata": {}, "source": [ "### Provided: the data" ] }, { "cell_type": "code", "execution_count": 15, "id": "7e6d1ab6", "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnEAAAGJCAYAAADlpGXRAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAO/xJREFUeJzt3Ql4VOW9x/F/EkIWIEEk7BGCIKBsIoKAV0A2Fa303lpFK4uglZJeFpfKvVYLeqW2KlpBqdclLvUBl4JeRDTsZXFhU6SKgBEQWULFbJAQMuc+/1cnZplJMpDMzDvn+3meYZgzZybnvDlz5pd3O1GO4zgCAAAAq0SHegMAAAAQOEIcAACAhQhxAAAAFiLEAQAAWIgQBwAAYCFCHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhyAiBIVFSXp6enVrpeRkWHW/frrryWSrV692uyn3gOILIQ4AHXOG5i8t/j4eDnvvPNM2Dp8+DC/gR9NmzZNevXqJU2aNJHExETp0qWL/OEPf5D8/PyQlNHSpUvNzwcQnuqFegMAuMesWbMkLS1NCgsLZd26dfL000+boPDZZ5+Z0OJ2H3/8sfzbv/2bjB8/3gTdrVu3yh//+EdZvny5rF27VqKjg/t3t/5u5s2bR5ADwhQhDkDQXHnlldK7d2/z/4kTJ8rZZ58tjz32mLz11lsyevRon68pKCiQBg0auOK3pMG2onPPPVfuvPNO+eijj+SSSy4JyXYBCE80pwIImcsvv9zcZ2Vlmftx48ZJw4YNZc+ePXLVVVdJo0aN5KabbioNc3fccYekpqZKXFycdOrUSR555BFxHMfne//tb38z62iN1kUXXWRqsmri3XffNbVhGhz1548cOVJ27NhRbh3vdu7bt0+uvvpq8//WrVubWiu1fft2s2/6Hm3btpVXX331tMuoXbt25v7777+vdt1vvvlGRo0aZX5us2bNTPNsUVFRpfX+8Y9/yHXXXSfnnHOOKUstU133xIkT5fbRuz9lm8K9tOz79+9vgnhCQoIp4zfeeOO09xNA4KiJAxAyGtaUBgGvU6dOyYgRI+TSSy81QUGbWTWo/exnP5NVq1bJhAkTpGfPnvLee+/JXXfdJQcOHJA5c+aUe981a9bIwoUL5T//8z9NSHnqqafkiiuuMLVZXbt29bs9L7/8sowdO9b8/IcffliOHz9umnx1W7Rp0xuoVElJialZvOyyy+RPf/qTCY3ax08D1H//93+b8Pnv//7vMn/+fBkzZoz069fPNCVXR/dfA9vJkydNM/O9995rwmSfPn2qfJ0GsCFDhphgqfvdqlUrsz8rV66stO7rr79u9m3SpEmm7LVcnnzySRMC9Tn161//Wr799lvJzMw071PRE088YX4nup+6rQsWLDDBcMmSJSb4AggCBwDq2AsvvKDVZc7y5cud7OxsZ//+/c6CBQucs88+20lISHC++eYbs97YsWPNevfcc0+51y9evNgsf/DBB8st/8UvfuFERUU5u3fvLl2m6+lt06ZNpcv27t3rxMfHOz//+c8rbVNWVpZ5nJeX5zRu3Ni59dZby/2MQ4cOOcnJyeWWe7fzoYceKl127Ngxsy+6PbpvXl988YVZ9/77769RWW3cuLF0H/TWqVMnZ9WqVdW+7vHHHzfrv/baa6XLCgoKnA4dOpjlZd/j+PHjlV4/e/Zss+1aVl6TJ082r/Wl4nucPHnS6dq1q3P55ZfXaD8BnDmaUwEEzdChQyUlJcU0391www2mGXLRokWmKbIsrSGq2ME+JibG1DCVpc2rmtu0CbQsrfXS5j0vbTa89tprTe2d1qD5ojVOWgOmffOOHj1aetOf27dvX1MLWJH26/Nq3Lixab7Vmrhf/vKXpct1mT731Vdf1aiMzj//fLMtixcvlrvvvtu8X01Gp2oZtWzZUn7xi1+ULtNazNtuu63Sutr86aXN1Lqf2jSqZak1jjVR9j2OHTsmOTk5phl6y5YtNXo9gDNHcyqAoNE+Vjq1SL169aR58+Ym4FQccanPtWnTptyyvXv3muZBbVYsS6fg8D5fVseOHSv9bP252oSYnZ0tLVq0qPT8rl27yvXTqygpKancY+1rp4G0rOTkZLPtZfuOeZdr0KkJ/TkadpUGT+1Pp/cajnr06OH3dVoGHTp0qPSztYwr0ibX++67T95+++1K26VhrCa02fTBBx+Ubdu2let3V/HnA6g7hDgAQaP9uryjU/3RPmzBnkpDeTwec6/9v3yFPA2XZWkNnS/+lvsbgFEd7Vd38803mz5nVYW4mtKayGHDhsl3330nv/vd76Rz586mtk/7FupgBm85VEUHRmh/OO0PqP0NtQYwNjZWXnjhhTMaxAEgMIQ4AGFPR3jqXGl5eXnlauO++OKL0ud91aqV9eWXX5rmxYq1Z2Wn8lA6qtNbExYOtJZLg1V1NWRaBjoQQsNi2dqwnTt3lltPR85qWbz44otmwIWXNuFW5K9W7c033zQ1kdo8raHbS0McgOChTxyAsKfTjWgN0ty5c8st11GpGjR0lGhZGzduLNc3a//+/WYuuuHDh/utKdMRqdqU+dBDD0lxcXGl57UZti5pfzxfP/fZZ58199XVYGoZ6WjSstN8aPPxM888U2497/6XrRnU/+to04q88/NVnN5E30PLvWz/Qr18mfbjAxA81MQBCHvXXHONDB482EzdoWFBmxXff/99E8ymTp1aWovmpdOIaCgrO8WImjlzpt+foQFOpxPRpku99JUOvNBaO+0/9s4778iAAQMqhcjapNc21e3VgQnap0+n7dBmy7///e8mwP3qV7+q8vW33nqr2T6tXdu8ebNp4tSm4YpXwtDmU+8EwtqEqvutNWu++ux5B4fodml5anjTctEpRHSSZp225cYbb5QjR46Y/o7aJ+/TTz+t5ZIB4FctjHAFgCp5p/P4+OOPq1xPp+5o0KCBz+d0CpBp06Y5rVq1cmJjY52OHTs6f/7znx2Px1NuPf05OjXGK6+8YtaJi4tzLrzwwkrTdFScYsRL1xsxYoSZVkSnJTn33HOdcePGlZuyxN92Dhw40LngggsqLW/btq0zcuTIKvddp0kZM2aM0759ezNVif5sfS+dmiQ/P9+pCZ0e5Gc/+5mTmJjoNG3a1JkyZYqzbNmySlOM/POf/3SGDh3qNGzY0Kyn06d88sknZj0tF69Tp045v/3tb52UlBQz/UjZr4znnnuutHw7d+5sXqfbytcKEDxR+o//iAcAAIBwRJ84AAAACxHiAAAALESIAwAAsBAhDgAAwEKEOAAAAAsR4gAAACzEZL/V0Mvd6CzoeqkfLuwMAADqks78ppcYbNWqVbXXkSbEVUMDXGpqam3+fgAAAKqklwts06ZNlesQ4qrhvdi2FqZensYttY96nUi95FB1fwVEKsqAcuB44HPBuYFzZCi+K3Jzc03lkTd/VIUQVw1vE6oGODeFuMLCQrO/bg5xbi8DRTlQDhwPfCY4N4TmHFmTLlzu/XYCAACwGCEOAADAQoQ4AAAACxHiAAAALESIAwAAsBAhDgAAwEKEOAAAAAsR4sJcYXGJ7P/uuLkHAADwYrLfMFXicWRO5pfy3LosOVFcIgmxMTLh0jSZNuw8iYmufgJAAAAQ2QhxYUoD3NxVu0sfa5DzPr5zRKcQbhkAAAgHNKeGIW061Ro4X55fn0XTKgAAIMSFo+y8IlPz5svxkyXmeQAA4G7UxIWhlEZxpg+cL4n1Y8zzAADA3QhxYSj+x0EMvtwyIM08DwAA3I2BDWHSB06bSLWGzRvQdBSqtw+cNqFqDZwGOO9yAADgboS4MJ5GREehpl/eoVLAAwAAIMSF+TQiGtxSmySGbBsBAEB4ok9ciDCNCAAAOBOEuBBhGhEAAHAmCHEhwjQiAADgTBDiQoRpRAAAwJlgYEMIMY0IAAA4XYS4EGIaEQAAcLoIcWGAaUQAAEDE9ombPXu2XHzxxdKoUSNp1qyZjBo1Snbu3FnlazIyMiQqKqrcLT4+PmjbDAAAIG4PcWvWrJHJkyfLBx98IJmZmVJcXCzDhw+XgoKCKl+XlJQkBw8eLL3t3bs3aNsMAAAgbm9OXbZsWaVaNq2R27x5s1x22WV+X6e1by1atAjCFgIAAASPNSGuopycHHPfpEmTKtfLz8+Xtm3bisfjkV69eslDDz0kF1xwgd/1i4qKzM0rNzfX3Ovr9VZXV284ml8kTRuGx/VRdT8dx6mz/bUBZUA5cDzwueDcwDkyFN8VgbyXlSFOd3Dq1KkyYMAA6dq1q9/1OnXqJM8//7x0797dhL5HHnlE+vfvLzt27JA2bdr47Xs3c+bMSsuzs7OlsLCwlvfDkbe2HZDMfx6WohKPxMVEy7Dzm8u1PVtLdHRUrf6swLbLY8pLD8zoaGta3GsVZUA5cDzwueDcwDkyFN8VeXl5NV43ytGfbJlJkybJu+++K+vWrfMbxnzRfnRdunSR0aNHywMPPFDjmrjU1FQ5duyY6V9Xmx57/0t5avUPF7wv6zeDOsj04edJKA9KDa0pKSmuDnFuLwNFOVAOHA98Jjg3BPccqbnjrLPOMuGwutxhXU1cenq6LFmyRNauXRtQgFOxsbFy4YUXyu7dlYOTV1xcnLlVpL+c2vwy1ybU59Z/LR6pXOP2/IavJX1Ix5A2rWpfwtreZ9tQBpQDxwOfC84NnCOD/V0RyPtY8w2tFYYa4BYtWiQrV66UtLS0gN+jpKREtm/fLi1btpRQy84rkhPFJT6fO36yxDwPAABgfU2cTi/y6quvyltvvWXmijt06JBZnpycLAkJCeb/Y8aMkdatW5t+bWrWrFlyySWXSIcOHeT777+XP//5z2aKkYkTJ0qopTSKk4TYGJ9BLrF+jHkeAADA+pq4p59+2rQPDxo0yNSkeW8LFy4sXWffvn1mLjgv7cd26623mn5wV111lWln3rBhg5x//vkSatpUOuFS37WJtwxIq1FTqjbJ7v/uuLkHAADuYk1NXE3GX6xevbrc4zlz5phbuJo27IfBC8+vzzJNqFoDpwHOu9yfEo8jczK/lOfWZZmavIQfA6G+Tq/HCgAAIp81IS4SaeC6c0QnSb+8g+kDp02oNamB0wA3d9VPgzM0yHkf6/sBAIDIZ01zaiTT4JbaJLHGTahaA+eL1ujRtAoAgDsQ4izDqFYAAKAIcZbxjmr1hVGtAAC4ByHOMrUxqhUAANiPgQ0WOt1RrQAAIHIQ4lw0qhUAAEQOQlwEjGoFAADuQ584AAAACxHiAAAALESIAwAAsBAhDgAAwEKEOAAAAAsR4gAAACxEiAMAALAQIQ4AAMBChDgAAAALEeIAAAAsRIgDAACwECEOAADAQoQ4AAAACxHiAAAALESIAwAAsBAhDgAAwELWhLjZs2fLxRdfLI0aNZJmzZrJqFGjZOfOndW+7vXXX5fOnTtLfHy8dOvWTZYuXRqU7QUAAKhL1oS4NWvWyOTJk+WDDz6QzMxMKS4uluHDh0tBQYHf12zYsEFGjx4tEyZMkK1bt5rgp7fPPvssqNsOAABQ2+qJJZYtW1bucUZGhqmR27x5s1x22WU+X/PEE0/IFVdcIXfddZd5/MADD5gAOHfuXJk/f35QthsAAMDVIa6inJwcc9+kSRO/62zcuFGmT59ebtmIESNk8eLFfl9TVFRkbl65ubnm3uPxmJsb6H46juOa/fWFMqAcOB74XHBu4BwZiu+KQN7LyhCnOzh16lQZMGCAdO3a1e96hw4dkubNm5dbpo91eVV972bOnFlpeXZ2thQWFoobaPlqSNYDMzramhb3WkUZUA4cD3wuODdwjgzFd0VeXl5khzjtG6f92tatW1fr7z1jxoxytXdaE5eamiopKSmSlJQkbjkoo6KizD67OcS5vQwU5UA5cDzwmeDcENxzpA7EjNgQl56eLkuWLJG1a9dKmzZtqly3RYsWcvjw4XLL9LEu9ycuLs7cKtJfjpu+zPWgdNs+V0QZUA4cD3wuODdwjgz2d0Ug72PNN7RWVWqAW7RokaxcuVLS0tKqfU2/fv1kxYoV5ZbpwAZdDgAAYLN6NjWhvvrqq/LWW2+ZueK8/dqSk5MlISHB/H/MmDHSunVr069NTZkyRQYOHCiPPvqojBw5UhYsWCCbNm2SZ555JqT7AgAAcKasqYl7+umnTefBQYMGScuWLUtvCxcuLF1n3759cvDgwdLH/fv3N8FPQ1uPHj3kjTfeMCNTqxoMAQAAYIN6NjWnVmf16tWVll133XXmBgAAEEmsqYkDAADATwhxAAAAFiLEAQAAWIgQBwAAYCFCHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIUIcAACAhQhxAAAAFiLEuVBhcYns/+64uQcAAHaqF+oNQPCUeByZk/mlPLcuS04Ul0hCbIxMuDRNpg07T2Kio/hVAABgEUKci2iAm7tqd+ljDXLex+mXd5DsvCJJaRQn9WMIdAAAhDtCnEto06nWwPkyf80eeXbdV1JY7Pmhdm5AO7mxe3LQtxEAANQcfeJcQmvZtObNl1MexwQ4pes8tXq3vLXtQJC3EAAABIIQ5xLaTKq1bDWV+flhBj4AABDGCHEuEf/jIIaaKjrlkaP5RXW6TQAA4PTRJ85FdBSqen59lhw/qaNTo6W4xDHNqRXF1YuWpg3jQrCVAACgJghxLqLTiNw5olO5kahzV+4uN2LVa1iX5qb2DgAAhCdCnAtpOEttkuizdi6xfozc0r+dXMvoVAAAwppVfeLWrl0r11xzjbRq1UqioqJk8eLFVa6/evVqs17F26FDh4K2zbbUzm35/TD5x92Dzf304edJNJP/AgAQ1qwKcQUFBdKjRw+ZN29eQK/buXOnHDx4sPTWrFmzOttG22vnaEIFAMAOVjWnXnnlleYWKA1tjRs3rpNtAgAACAWrQtzp6tmzpxQVFUnXrl3lD3/4gwwYMMDvurqe3rxyc3PNvcfjMTc30P10HMc1++sLZUA5cDzwueDcwDkyFN8VgbxXRIe4li1byvz586V3794mmD377LMyaNAg+fDDD6VXr14+XzN79myZOXNmpeXZ2dlSWFgobqAHUE5Ojjkwo6OtanGvNZQB5cDxwOeCcwPnyFB8V+Tl5dV43ShHf7KFdIDCokWLZNSoUQG9buDAgXLOOefIyy+/XOOauNTUVDl27JgkJSWJWw5KDa0pKSmuDnFuLwNFOVAOHA98Jjg3BPccqbnjrLPOMuGwutwR0TVxvvTp00fWrVvn9/m4uDhzq0h/OW76MteQ7LZ9rogyoBw4HvhccG7gHBns74pA3sd139Dbtm0zzawAAAA2s6omLj8/X3bv/unqAllZWSaUNWnSxDSRzpgxQw4cOCAvvfSSef7xxx+XtLQ0ueCCC0x/Nu0Tt3LlSnn//fdDuBcAAAAuC3GbNm2SwYMHlz6ePn26uR87dqxkZGSYOeD27dtX+vzJkyfljjvuMMEuMTFRunfvLsuXLy/3HgAAADayKsTpyNKqxmFokCvr7rvvNjcAAIBI47o+cQAAAJGAEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIUIcAACAhQhxAAAAFiLEAQAAWIgQBwAAYCFCHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIUIcAACAhQhxACJKYXGJ7P/uuLkHgEhWL9QbAAC1ocTjyJzML+W5dVlyorhEEmJjZMKlaTJt2HkSEx1VaX0Nedl5RZLSKE7iY2P4JQCwDiEOQETQADd31e7SxxrkvI/vHNHptMMeAIQrmlMBWE9r1TSU+fL8+qxyTavesKcBrmzY0+UAYBOrQtzatWvlmmuukVatWklUVJQsXry42tesXr1aevXqJXFxcdKhQwfJyMgIyrYCCB5tFvWGsoqOn/yh2bS6sPfcuq/oRwfAKlaFuIKCAunRo4fMmzevRutnZWXJyJEjZfDgwbJt2zaZOnWqTJw4Ud57770631YAwaP92rRZ1JfE+jHm+erC3olijzy09HPT3AoANrCqT9yVV15pbjU1f/58SUtLk0cffdQ87tKli6xbt07mzJkjI0aMqMMtBRBM8T/2ayvbJ87rlgFppQMXvGHPX5B7aeNeSYqPLdeHDgDClVUhLlAbN26UoUOHllum4U1r5PwpKioyN6/c3Fxz7/F4zM0NdD8dx3HN/vpCGdhXDlOGdBBxHMnYkGWaULUGblz/NLPcu/31Y6JkwoB28tTqymHPK2P9V/KbQe3LjVi1qRzqEuVAGXAs1P1nIpD3iugQd+jQIWnevHm5ZfpYg9mJEyckISGh0mtmz54tM2fOrLQ8OztbCgsLxQ30AMrJyTEHZnS0VS3utYYysLMcftWzsfyyaw/JOVEsyQmxUr9etPzraHa5dW7snixRhSmycucRP+9ySvbs+7a0CdbGcqgrlANlwLFQ95+JvLy8Gq8b0SHudMyYMUOmT59e+lgDX2pqqqSkpEhSUpK45aDUgSO6z279wqIM7C6HNtU8P+nKpvLc5kwpLK78F6/W4J17TqtKNXE2lkNtoxwoA46Fuv9MxMfH13jdiA5xLVq0kMOHD5dbpo81jPmqhVM6ilVvFekvx00nbz0o3bbPFVEGkVsOiXHRcsul5/rsQzduQHtJjIt1RTmcDsqBMuBYqNvPRCDvE9Ehrl+/frJ06dJyyzIzM81yAO6mk/t655Hz9qHTQRDe5QAQ7qwKcfn5+bJ79+5yU4jo1CFNmjSRc845xzSFHjhwQF566SXz/O233y5z586Vu+++W2655RZZuXKlvPbaa/LOO++EcC8AhAO9OoOOQk2/vAOX3wJgJatC3KZNm8ycb17evmtjx441k/gePHhQ9u3bV/q8Ti+igW3atGnyxBNPSJs2beTZZ59lehEApbTvW2qTREoEgHWsCnGDBg0yI0D88XU1Bn3N1q1b63jLAAAAgsvdPXQBAAAsRYgDAACwECEOQNDphej3f3c87C84b8t2AnAnq/rEAbCbXlx+TuaX8ty6LHP90oQfr3mq03roaNFw3c4GsdGS3i9FJg7XCT1DvXUA8ANORwCCRoORTrDrvQC93utjXR7u27lk+0F5YvmuUG8aAJQixAEICm2S1JotX3TC3XBpsqxqOzM2hM92AgAhDkBQZOcVldZsVaRXTNDnw4Et2wkAhDgAQZHSKM70gfNFL3mlz4cD3Y742Oiw304AIMQBCNqVEXQQgy96zVJ9PhwGNMxduVtOlfieVHxc//DYTgA4rRCnl7hau3YtpQcgYDoKNX1wB1OjpfReH4fLRee9AxpOecqHuHrRUXJ1t5YyZWjHkG0bAJzxFCM5OTkydOhQadu2rYwfP96EutatWwf6NgBcKJwvOl/VgIbYmCgZ2b1VWE2DAgAB18QtXrxYDhw4IJMmTZKFCxdKu3bt5Morr5Q33nhDiouLKVEANb7ofLgEuOoGNJwo9kjOCc5vACKgT1xKSopMnz5dPvnkE/nwww+lQ4cOcvPNN0urVq1k2rRpsmsXcykBiKyBF8kJsUHfJgCos4ENBw8elMzMTHOLiYmRq666SrZv3y7nn3++zJkz50zeGgDCZuCFDmioX49xYADCS8BnJW0yffPNN+Xqq682/eJef/11mTp1qnz77bfy4osvyvLly+W1116TWbNm1c0WA0CQB14woAFARAxsaNmypXg8Hhk9erR89NFH0rNnz0rrDB48WBo3blxb2wgAIR14oec8ALA+xGkz6XXXXSfx8fF+19EAl5Xle5QXANgy8AIAIirE6QAGAAAAhBY9dQEAACxEiANQ65Pm7v/uuLkHAIRRcyoA+LvuqF62Sq96oJPmJvw4ZYeO+ORKBwBQ+whxAGr1uqNeGuS8j3XEJwD7aQ27jtw+uwGTX4cDQhyAM1bVdUefX59lpuwIp0tsATizmvYGsdGS3i9FJg5PkWg6ZoUMRQ+gTq87evzkD3+5A7C/pt37Odf7JdsPyhPLucxmKFkX4ubNmyft2rUz89T17dvXTDjsT0ZGhkRFRZW7VTW/HYC6ue6oPg8g8mraMzZkMYgphKwKcQsXLpTp06fL/fffL1u2bJEePXrIiBEj5MiRI35fk5SUZK7x6r3t3bs3qNsMuP26o7cMSKMpFbAYNe3hy6oQ99hjj8mtt94q48ePl/PPP1/mz58viYmJ8vzzz/t9jda+tWjRovTWvHnzoG4z4PbrjupyMPUK7EVNe/iyZmDDyZMnZfPmzTJjxozSZdHR0TJ06FDZuHGj39fl5+dL27ZtzbUPe/XqJQ899JBccMEFftcvKioyN6/c3Fxzr693y/UTdT8dx3HN/vpCGQReDlEiMn1YR/nNoPZyNL9Imjb84bqjIvp6R9x6PGiHcO039ML6n6ZeGT8gTaYM7Wjd1Ct8LtxZBvVjomTCgHby1OqfRp9HiyNR4si4fu3M824qj7o+HgJ5L2tC3NGjR6WkpKRSTZo+/uKLL3y+plOnTqaWrnv37pKTkyOPPPKI9O/fX3bs2CFt2rTx+ZrZs2fLzJkzKy3Pzs6WwsJCcQM9gLS89MDUoOxGlMGZlUN9/QPoWL788CeQu4+HRVu+kVXbD0q7ht4lp2TVJ7ukgSdfft7L93koXPG5cG8Z3Ng9WRp4Wkjm54el6JRHEupFy1WdkmR49+QquzRFOk8dHA95eXmRF+JOR79+/czNSwNcly5d5K9//as88MADPl+jNX3a765sTVxqaqqkpKSY/nVuOSi1GVr32U0nqbIoA8qhNo4H7RA+d+NWOVFcucZt7wfZMnZID6v6C/K5cHcZ/PqK5jJ2SImpaW+SGCt533/nynKo6+MhkAGY1oS4pk2bSkxMjBw+fLjccn2sfd1qIjY2Vi688ELZvfunKuGK4uLizK0i/eW46UDVg9Jt+1wRZUA5nOnx8K+CQiko1qaRyiEu/6RH/lVQLKlN7Jo0lc+Fu8sgMS5azomLNeEl38XlUJfHQyDvY03J169fXy666CJZsWJF6TI9iPRx2dq2qmhz7Pbt26Vly5Z1uKUA8AM6hAOoS9aEOKXNnP/7v/8rL774onz++ecyadIkKSgoMKNV1ZgxY8oNfJg1a5a8//778tVXX5kpSX71q1+ZKUYmTpwYwr0A4BZMvQKgLlnTnKquv/56M8Dgvvvuk0OHDknPnj1l2bJlpYMd9u3bV64a8tixY2ZKEl33rLPOMjV5GzZsMNOTAEAweKdY0cuP6dUrdOoVnTuPqVcAnKkoR4dUwC8d2JCcnGxGn7hpYIOONmrWrJlr+zpQBpRDbR8P3guHaxOrTYMZyuJzQRlwLNT9ZyKQ3OHOb2gACDINbhrgNMhpoAMAVzWnAoCNdMJfvYC4Xn/SO+GvXqZMm1Rtm/AXQPigJg4A6pgGuLmrdpsAp/ReH+tyIFJpjfP+745T81yHqIkDgDr+ItMaOF90sEP65R2s7SMH+ELNc/BQEwcAdUj7wHlr4CrS0ar6PBBJqHkOHkIcANRhjcTfPtjr93mdbkQHOwBuqXlmUE/tIsQBQB3WSMxf+5Xf53W+OJpSEUmoeQ4uQhwABLlGQt0+sD0T/sLVl5pj4MOZY2ADAAS5RkLd1Lct04sgYi81p6Ov/dU8M/Ch9hDiAKAOayR8BTn6wrlDJFyloy4uNecd+ODlnXJH3TmiU4i22k6EOAAIUY0EIpOvmqax/drKDX3OkRbJ8RH/u9cJrDWM6fQ5FUMsU+7ULkIcAISoRgKRyVdNkw5w0ZubrtahwS21SWLAAx8qvgb+EeIAIAQ1EnDngBa3Nx3SzaB2MToVAIJUI0GAi3zVDWhx+5xp3m4GvtDNIHCEOAAAgjDFRlluvlqHNiWnD+5guhcovdfHdDMIHM2pAAAEYUBLWW4eoUw3g9pDiAMAoBZpjdIpj0cyNnwthcUen+vQdOh74AMCQ3MqAAC1PL3Iixv2mgBXPyZKeqU2lgSaDlEHqIkDAKCOphc5WeLIlv3fy+2XtZebLmnrmhHKbp3oONgIcQAA1PH0Ii99sFemDjsv4gMNl9QKLppTAQCoBTWZyNYtNZHecvDOi6fLUfsIcQAA1PH0Im4YjVrdJbXcOC9eXSPEAQBQC9w+kS01kcFHiAMAoJa4eSJbW2oiC4tLZP93xyOiZtC6EDdv3jxp166dxMfHS9++feWjjz6qcv3XX39dOnfubNbv1q2bLF26NGjbCgBw50S2W34/TP5x92Bzr48j/WL3NtRElngceeS9nXLhrEz5tz+tMvf6WJfbyqoQt3DhQpk+fbrcf//9smXLFunRo4eMGDFCjhw54nP9DRs2yOjRo2XChAmydetWGTVqlLl99tlnQd92ADgTkVR74AZuvV5uONdEzonAQRdRjuNYE0G15u3iiy+WuXPnmscej0dSU1Plt7/9rdxzzz2V1r/++uuloKBAlixZUrrskksukZ49e8r8+fNr9DNzc3MlOTlZcnJyJCkpSdxAy1WDcbNmzSQ62qqcX2soA8ohXI6HcJqygc8FZVDTYyHc5okrLC4xNW++Rg9r0NQa09PZzrr4TASSO6yZJ+7kyZOyefNmmTFjRukyLbChQ4fKxo0bfb5Gl2vNXVlac7d48WK/P6eoqMjcyham9xelNzfQ/dRs75b99YUyoBzC5XiY8/6X8tTqHyaP1a+IouJT8tSqXSKOI9OHB7d2g88FZVDTY0GvVNG6cXzpuqF2JPeE+ez4ilmFJ0+Z59uclRgWn4lA3suaEHf06FEpKSmR5s2bl1uuj7/44gufrzl06JDP9XW5P7Nnz5aZM2dWWp6dnS2FhYXiBnoA6V8AemC6uSYu0svg5CmP5JwoluSEWKlfL9q15VAToSoH/R1t+GyPdDmrcoPJhh175Jddk/z+7uoCxwNlYO2xcMojPZtGSVFJ5YAUp5+hE7lypDg/LMohLy8v8kJcsGhNX9naO62J0ybblJQUVzWnRkVFmX224sNZByK5DLR57onlu+SF9T81z40fkCZThnas1DwXyeUQiFCVwzfHjsu2oxrgfDWbOiIJSdLsNGoPThfHA2Vg87HQv+u5pbXaZf1m0LnSplWLsCkHHYgZcSGuadOmEhMTI4cPHy63XB+3aOG78HV5IOuruLg4c6tIfzm2HKi1QQ9Kt+2zW8rgscydMnf1nh8fRUlBseeHx1E/jKpzSzkEKhTl0CwpQeJi6/ntx6PPB/v3wvFAGdh6LEwb3smc53TiYb2Chn6GdNSs9i+NPoP+pbVdDoG8jx0lr+3r9evLRRddJCtWrCiXgPVxv379fL5Gl5ddX2VmZvpdH4h0zKhul3CfsgGwSUwETv9iTU2c0mbOsWPHSu/evaVPnz7y+OOPm9Gn48ePN8+PGTNGWrdubfq1qSlTpsjAgQPl0UcflZEjR8qCBQtk06ZN8swzz4R4T4DwnVFdp0VA+PBOzeCr9gDA6U//EgmsCnE6ZYgOMLjvvvvM4ASdKmTZsmWlgxf27dtXrhqyf//+8uqrr8q9994r//Vf/yUdO3Y0I1O7du0awr0AQj+jur/muXCZUR2Vaw/SL+8QVlM2AAg9q0KcSk9PNzdfVq9eXWnZddddZ24Afmqe0wkuK6J5LrxFUu1BOPM3v5l3+dkNYkO6fYDVIQ7AmaF5Dqj5pMr/OaSj/GXFrtLlDWKjJb1fikwcrqMRKUmEFiEOcBma5wD/l2Ty8l6S6YOv/iWb9h4rt3zJ9oNSEN1Q7ryiM0WJkOLvCMCl3HptRyCQUdtlA1xZGRuyuI6tyxSG4fWLqYkDALhaVaO2/WE0t3uUVHH94lBPTkJNHADA1byjtgPBaG73NbWf+DHoe5vadXmoEeIAAK5W1aTKvdue5XP5uP5Mtuz2pvbn1n0V8qZVQhwAwPW0aSx9cAdTw6b0Xh+/eusllZZf3a2ludYw3N3UfqLYI39c9oV4PHp949CgTxwAwPWqGrVddrnOE5d77F9WX6oJtTNBunpl415pGVskv77ih4sOBBs1cQAAVDNqm9Hc7hRfRVO7V+bnh0PWrEqIAwAAqKKpfcwlbf09LUWnPHI0v0hCgRAHAADghzad/9fILhIf6zsyxdWLlqYNQ3PdaUIcAABANc2qEy9t7/O5YV2ah2zSdAY2AAAAnM51p/u3k2u7J0uoEOIAAABOYwRz/ZgoOXLkiIQKIQ4AAKCGvCOVlcfjkVCiTxwAIOwv9A2gMmriAADVXuibyW2B8EOIAwCUu9C3l/dC30r7AgEILzSnAkAEq2nTaFUX+tbReDStAuGHmjgAiECBNo1WdaFvnU5Bn2/dOD4IWw6gpqiJA4AIbhr1BjNv06gur+pC377ofFj6PIDwQogDgAhTVdPoc+u+kt1H8io1j1Z1oe9bBqSFbEZ6AP7RnAoAEaaqptETxR4Z+than82rPmekH/DDOgDCDyEOACKMt2nUX5DzN/LU14z01MAB4cua5tTvvvtObrrpJklKSpLGjRvLhAkTJD8/v8rXDBo0SKKiosrdbr/99qBtMwCEQlVNozUZeeqdkZ4AB4Q3a2riNMAdPHhQMjMzpbi4WMaPHy+33XabvPrqq1W+7tZbb5VZs2aVPk5M/OFSGQAQySo2jfrjHXnqvYwQAHtYEeI+//xzWbZsmXz88cfSu3dvs+zJJ5+Uq666Sh555BFp1aqV39dqaGvRokWNf1ZRUZG5eeXm5pZeHy3U10gLFt1Px3Fcs7++UAaUg+3Hg/Zymz6so/xmUHs5cOy4XDtvvRQWV95+7fd2doPYGu2bjeVQ2ygDyqGuj4dA3suKELdx40bThOoNcGro0KESHR0tH374ofz85z/3+9q//e1v8sorr5ggd80118jvf//7KmvjZs+eLTNnzqy0PDs7WwoLC8UN9ADKyckxB6aWsRtRBpRDJB0PjUTkt/2ayZLtBys9d3W3FMk99i/54c/VyC6H2kAZUA51fTzk5eVFVog7dOiQNGvWrNyyevXqSZMmTcxz/tx4443Stm1bU1P36aefyu9+9zvZuXOn/P3vf/f7mhkzZsj06dPL1cSlpqZKSkqK6Y/nloNS+w/qPrv5RO32MlCUQ+SUw8ThKVIQ3VAyNvw08nRc/zSZOLRjja+LejrloP3tjuYXSdOGkTFIIhKOhdpAOdRdOcTHx9sR4u655x55+OGHq21KPV3aZ86rW7du0rJlSxkyZIjs2bNHzj33XJ+viYuLM7eK9Jfjpg+sHpRu2+eKKAPKIZKOB93sO6/oLOlDOp7RyNOalkOgV4zwFf7KbmfFx6Fk+7FQWyiHuimHQN4npCHujjvukHHjxlW5Tvv27U1T6JEjR8otP3XqlBmxGkh/t759+5r73bt3+w1xABDJvCNPg3XFiKqmNKlJ+IuvFy1dWyfLjm9zTysMApEspCFOqx/1Vp1+/frJ999/L5s3b5aLLrrILFu5cqWpxvQGs5rYtm2budcaOQBA8K8YoaNldR46f7VpFcNf4SmPbNp7LOAwCLiBFXXBXbp0kSuuuMJMF/LRRx/J+vXrJT09XW644YbSkakHDhyQzp07m+eVNpk+8MADJvh9/fXX8vbbb8uYMWPksssuk+7du4d4jwDAnVeM8E5pEmj4q8n8doDbWBHivKNMNaRpnzadWuTSSy+VZ555pvR5nTtOBy0cP37cPK5fv74sX75chg8fbl6nTbf/8R//If/3f/8Xwr0AAPdcMcKXhNho83yg4S+QMAi4hRWjU5WORK1qYt927dqZIb5eOqJ0zZo1Qdo6AEDFK0aUbRb1Ki5xZO7K3T77tNXkcmFeOrrWXxgE3MKamjgAgD00pKUP7iD1KgS1Ux7HhDvt+3Ymlwu7ZUBayEepAqFGiAMA1DqtZdMBDPViogLq0+YNf1rT5m1+7d32LEn48bEu1+e9lxUD3Mya5lQAgF20z5qvS31Vdc1WDX866lQDYLjOEweEC0IcAKBOVNXHrbo+bRXnswvW/HaATWhOBQDUiar6uNGnDThz1MQBAKrkbco8u0FswCXl7bumfeC812zVAEefNuDMEeIAADW6BFaD2GhJ75ciE4frxb5rVmj++rgBOHM0pwIAqrwElrdPm94v2X5Qnli+K+AS8/ZpI8ABtYcQBwAI6BJYGRu45BUQDghxAIBau/4pgOAhxAEAArr+KZe8AsIDIQ4AEND0IOP6c8krIBwwOhUAUOPpQa7uliITh3akxIAwQIgDANRoehCdJy732L/McgChR3MqAKBKTA8ChCdCHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIWtC3P/8z/9I//79JTExURo3blyj1ziOI/fdd5+0bNlSEhISZOjQobJr164631YAAIC6Zk2IO3nypFx33XUyadKkGr/mT3/6k/zlL3+R+fPny4cffigNGjSQESNGSGFhYZ1uKwAAQF2z5tqpM2fONPcZGRk1roV7/PHH5d5775Vrr73WLHvppZekefPmsnjxYrnhhhvqdHsBAADqkjUhLlBZWVly6NAh04TqlZycLH379pWNGzf6DXFFRUXm5pWbm2vuPR6PubmB7qeGYLfsry+UAeXA8cDngnMD58hQfFcE8l4RG+I0wCmteStLH3uf82X27NmltX5lZWdnu6YZVg+gnJwcc2BGR1vT4l6rKAPKgeOBzwXnBs6RofiuyMvLsyPE3XPPPfLwww9Xuc7nn38unTt3Dto2zZgxQ6ZPn16uJi41NVVSUlIkKSlJ3HJQRkVFmX12c4hzexkoyoFy4HjgM8G5IbjnyPj4eDtC3B133CHjxo2rcp327duf1nu3aNHC3B8+fNiMTvXSxz179vT7uri4OHOrSH85bvoy14PSbftcEWVAOXA88Lng3MA5MtjfFYG8T0hDnCZXvdWFtLQ0E+RWrFhRGtq0Vk1HqQYywhUAACAcWVPNsm/fPtm2bZu5LykpMf/XW35+fuk62uy6aNGi0mQ8depUefDBB+Xtt9+W7du3y5gxY6RVq1YyatSoEO4JAADAmbNmYINO2vviiy+WPr7wwgvN/apVq2TQoEHm/zt37jQdDL3uvvtuKSgokNtuu02+//57ufTSS2XZsmUBtTcDAACEI2tCnM4PV90ccTo6pCytjZs1a5a5AQAARBJrmlMBAADwE0IcEGKFxSWy/7vj5h4AgIhrTgUiTYnHkTmZX8pz67LkRHGJJMTGyIRL02TasPMkJjoq1JsHAAhzhDggRDTAzV21u/SxBjnv4ztHdOL3AgCoEs2pQAho06nWwPny/PosmlYBANUixAEhkJ1XZGrefDl+ssQ8DwBAVQhxQAikNIozfeB8SawfY54HAKAqhDggBOJ/HMTgyy0D0szzAABUhYENQIjoKFRvHzhtQtUaOA1w3uUAAFSFEIeIHjygfcu0aTIca7Z0GhEdhZp+eYew3k4AQHgixCHi2Db/mga31CaJod4MAIBlCHGIOMy/BgBwAwY2IKIw/xoAwC0IcYgozL8GAHALQhwiCvOvAQDcghCHiML8awAAt2BgAyIO868BANyAEIeIw/xrAAA3IMQhYjH/GgAgktEnDgAAwEKEOAAAAAsR4gAAACxEiAMAALAQIQ4AAMBChDgAAAALEeIAAAAsxDxx1XAcx9zn5uaKW3g8HsnLy5P4+HiJjnZnzqcMKAeOBz4XnBs4R4biu8KbN7z5oyqEuGroL0elpqbWxu8GAACgRvkjOTm5ynWinJpEPRfTlP3tt99Ko0aNJCoqStxA/wrQ0Lp//35JSkoSN6IMKAeOBz4XnBs4R4biu0JjmQa4Vq1aVVu7R01cNbQA27RpI26kB6RbQ5wXZUA5cDzwueDcwDky2N8V1dXAebmzwxMAAIDlCHEAAAAWIsShkri4OLn//vvNvVtRBpQDxwOfC84NnCPD/buCgQ0AAAAWoiYOAADAQoQ4AAAACxHiAAAALESIAwAAsBAhzqXmzZsn7dq1M9d769u3r3z00Ud+183IyDBXqyh709fZbO3atXLNNdeYGbF1fxYvXlzta1avXi29evUyo5A6dOhgysV2gZaDlkHFY0Fvhw4dElvNnj1bLr74YnNVlmbNmsmoUaNk586d1b7u9ddfl86dO5vPQrdu3WTp0qVis9Mph0g8Nzz99NPSvXv30slb+/XrJ++++66rjoVAyyASjwNf/vjHP5p9mzp1qoTL8UCIc6GFCxfK9OnTzbDoLVu2SI8ePWTEiBFy5MgRv6/RD/LBgwdLb3v37hWbFRQUmP3WMFsTWVlZMnLkSBk8eLBs27bNfIgnTpwo7733nripHLz0y73s8aBf+rZas2aNTJ48WT744APJzMyU4uJiGT58uCkbfzZs2CCjR4+WCRMmyNatW03g0dtnn30mbiqHSDw36BV69Mt68+bNsmnTJrn88svl2muvlR07drjmWAi0DCLxOKjo448/lr/+9a8m3FYl6MeDXjsV7tKnTx9n8uTJpY9LSkqcVq1aObNnz/a5/gsvvOAkJyc7kUo/BosWLapynbvvvtu54IILyi27/vrrnREjRjhuKodVq1aZ9Y4dO+ZEqiNHjph9XLNmjd91fvnLXzojR44st6xv377Or3/9a8dN5RDp5wavs846y3n22WddeyxUVwaRfhzk5eU5HTt2dDIzM52BAwc6U6ZM8btusI8HauJc5uTJk+avq6FDh5a7Pqw+3rhxo9/X5efnS9u2bc2Ffqv7iywSadmULTOltZdVlVkk69mzp7Rs2VKGDRsm69evl0iSk5Nj7ps0aeLq46Em5RDp54aSkhJZsGCBqY3UJkU3Hgs1KYNIPw4mT55sWmIq/p7D4XggxLnM0aNHzYeyefPm5ZbrY3/9mjp16iTPP/+8vPXWW/LKK6+Ix+OR/v37yzfffCNuoWXjq8xyc3PlxIkT4hYa3ObPny9vvvmmuekJe9CgQaZZPhLosa1N5QMGDJCuXbsGfDzY3DfwdMohUs8N27dvl4YNG5r+r7fffrssWrRIzj//fFcdC4GUQaQeB0oDrJ7ftM9oTQT7eKhXJ++KiKJ/fZX9C0w/nF26dDH9Ax544IGQbhuCS0/Weit7LOzZs0fmzJkjL7/8ckT8xa19V9atWyduVtNyiNRzgx7j2vdVayPfeOMNGTt2rOkz6C/ERKJAyiBSj4P9+/fLlClTTB/RcB2oQYhzmaZNm0pMTIwcPny43HJ93KJFixq9R2xsrFx44YWye/ducQstG19lpp15ExISxM369OkTEaEnPT1dlixZYkbsasfu0zkeavoZipRyiNRzQ/369c0IdHXRRReZTu1PPPGECSVuORYCKYNIPQ42b95sBvzprARe2pKln425c+dKUVGR+T4N5fFAc6rL6AdTP5ArVqwoXaZV3/q4qv4OZelBrFXt2rTmFlo2ZctM6V9nNS2zSKZ/rdt8LOiYDg0u2ly0cuVKSUtLc+XxcDrl4JZzg54j9QvbLcdCoGUQqcfBkCFDzH7oOc576927t9x0003m/xUDXEiOhzoZLoGwtmDBAicuLs7JyMhw/vnPfzq33Xab07hxY+fQoUPm+Ztvvtm55557StefOXOm89577zl79uxxNm/e7Nxwww1OfHy8s2PHDsfm0UZbt241N/0YPPbYY+b/e/fuNc/r/ms5eH311VdOYmKic9dddzmff/65M2/ePCcmJsZZtmyZY7NAy2HOnDnO4sWLnV27djnbt283o7Sio6Od5cuXO7aaNGmSGVm3evVq5+DBg6W348ePl65T8TOxfv16p169es4jjzxijof777/fiY2NNWXipnKIxHOD7p+OyM3KynI+/fRT8zgqKsp5//33XXMsBFoGkXgc+FNxdGqojwdCnEs9+eSTzjnnnOPUr1/fTDnywQcflDtIx44dW/p46tSppes2b97cueqqq5wtW7Y4NvNOlVHx5t1vvddyqPianj17mnJo3769GVZvu0DL4eGHH3bOPfdcc4Ju0qSJM2jQIGflypWOzXztv97K/n4rfibUa6+95px33nnmeNDpZ9555x3HbeUQieeGW265xWnbtq3Zp5SUFGfIkCGl4cUtx0KgZRCJx0FNQ1yoj4co/adu6vgAAABQV+gTBwAAYCFCHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIUIcAACAhQhxAAAAFiLEAUAtyc7OlhYtWshDDz1UumzDhg1Sv359WbFiBeUMoFZFOY7j1O5bAoB7LV26VEaNGmXCW6dOnaRnz55y7bXXymOPPRbqTQMQYQhxAFDLJk+eLMuXL5fevXvL9u3b5eOPP5a4uDjKGUCtIsQBQC07ceKEdO3aVfbv3y+bN2+Wbt26UcYAah194gCglu3Zs0e+/fZb8Xg88vXXX1O+AOoENXEAUItOnjwpffr0MX3htE/c448/bppUmzVrRjkDqFWEOACoRXfddZe88cYb8sknn0jDhg1l4MCBkpycLEuWLKGcAdQqmlMBoJasXr3a1Ly9/PLLkpSUJNHR0eb///jHP+Tpp5+mnAHUKmriAAAALERNHAAAgIUIcQAAABYixAEAAFiIEAcAAGAhQhwAAICFCHEAAAAWIsQBAABYiBAHAABgIUIcAACAhQhxAAAAFiLEAQAAiH3+H2VrAtOZq0uoAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "def generate_data(seed=190):\n", " rng = np.random.default_rng(seed)\n", " A_true, b_true, w_true, p_true = 3.0, 0.6, 4.0, 0.5\n", " xs = np.sort(rng.uniform(0, 4, size=40))\n", " ys = A_true * np.exp(-b_true * xs) * np.sin(w_true * xs + p_true)\n", " ys = ys + rng.normal(0, 0.10, size=xs.shape)\n", " return xs, ys, np.array([A_true, b_true, w_true, p_true])\n", "\n", "xs, ys, p_true = generate_data() # xs and ys both have shape (40,)\n", "\n", "plt.figure(figsize=(7, 4))\n", "plt.scatter(xs, ys, s=18)\n", "plt.xlabel(\"x\"); plt.ylabel(\"y\")\n", "plt.title(\"Problem 3 data\"); plt.grid(alpha=.3); plt.show()" ] }, { "cell_type": "markdown", "id": "0cee4488", "metadata": {}, "source": [ "### 3(a) Derivations\n", "\n", "With $\\hat y_i = A e^{-b x_i}\\sin(\\omega x_i + \\varphi)$ and\n", "$e = \\frac{1}{N}\\sum_i (y_i - \\hat y_i)^2$, the chain rule gives\n", "\n", "$$\\frac{\\partial e}{\\partial \\theta} = \\frac{-2}{N}\\sum_i (y_i - \\hat y_i)\\frac{\\partial \\hat y_i}{\\partial \\theta}$$\n", "\n", "for any one of the four parameters $\\theta$, so we only need the four model derivatives.\n", "Writing $E_i = e^{-b x_i}$, $S_i = \\sin(\\omega x_i + \\varphi)$,\n", "$C_i = \\cos(\\omega x_i + \\varphi)$:\n", "\n", "$$\\frac{\\partial \\hat y_i}{\\partial A} = E_i S_i \\qquad\n", "\\frac{\\partial \\hat y_i}{\\partial b} = -x_i A E_i S_i$$\n", "\n", "$$\\frac{\\partial \\hat y_i}{\\partial \\omega} = x_i A E_i C_i \\qquad\n", "\\frac{\\partial \\hat y_i}{\\partial \\varphi} = A E_i C_i$$\n", "\n", "The $b$ and $\\omega$ derivatives are the ones needing the product rule together with the\n", "chain rule; the $x_i$ factor comes from differentiating the inner function." ] }, { "cell_type": "markdown", "id": "b8387ad4", "metadata": {}, "source": [ "### 3(b) The model, the loss, and the gradient\n", "\n", "Five short lines to write in total, spread over three functions.\n", "\n", "The third function is the heart of the assignment. Notice that its structure is identical to\n", "`compute_grad` back in Problem 1 — residuals times a derivative, summed over the data, scaled\n", "by $-2/N$. The only difference is that here there are four different derivatives instead of\n", "one matrix multiply. The three repeated quantities $E$, $S$, $C$ from your 3(a) derivation are\n", "computed for you, and the $A$ derivative is worked out as an example of the pattern." ] }, { "cell_type": "code", "execution_count": 16, "id": "9a257121", "metadata": {}, "outputs": [], "source": [ "def forward(x, p):\n", " \"\"\"Evaluate the model at the points in x.\n", "\n", " Inputs\n", " ------\n", " x : an array of x values, any shape.\n", " p : the four parameters, in the order [A, b, omega, phi].\n", "\n", " Output\n", " ------\n", " An array the same shape as x: the predicted y at each x.\n", " \"\"\"\n", " A, b, omega, phi = p # unpack the four parameters into named variables\n", "\n", " # YOUR CODE HERE (1 line): return the model outputs corresponding to the inputs in x\n", " # Use np.exp(...) and np.sin(...).\n", " return A * np.exp(-b * x) * np.sin(omega * x + phi)\n", "\n", "\n", "def model_loss(x, y, p):\n", " \"\"\"Mean squared error between the measurements y and the model output at x.\n", "\n", " Inputs\n", " ------\n", " x : the measured x values, shape (N,)\n", " y : the measured y values, shape (N,)\n", " p : the four parameters [A, b, omega, phi]\n", "\n", " Output\n", " ------\n", " A single number.\n", " \"\"\"\n", " residuals = y - forward(x, p) # shape (N,) -- same idea as in Problem 1\n", "\n", " # YOUR CODE HERE (1 line): return the mean of the squared residuals.\n", " return np.mean(residuals ** 2)\n", "\n", "\n", "def model_grad(x, y, p):\n", " \"\"\"Gradient of model_loss with respect to p.\n", "\n", " Inputs\n", " ------\n", " x, y, p : exactly as in model_loss above.\n", "\n", " Output\n", " ------\n", " An array of shape (4,): the partial derivatives with respect to A, b, omega, phi,\n", " in that order.\n", " \"\"\"\n", " N = len(y)\n", " A, b, omega, phi = p\n", "\n", " E = np.exp(-b * x) # shape (N,)\n", " S = np.sin(omega * x + phi) # shape (N,)\n", " C = np.cos(omega * x + phi) # shape (N,)\n", " residuals = y - A * E * S # shape (N,) -- this is y - forward(x, p)\n", "\n", " # Each of these is d(yhat)/d(parameter), evaluated at every data point, shape (N,).\n", " # The first one is done for you as an example; write the other three from your 3(a)\n", " # derivations. Note that x, E, S, and C are all arrays, so ordinary * multiplies\n", " # them entry by entry -- there is no loop over the data anywhere here.\n", " dyhat_dA = E * S\n", "\n", " # YOUR CODE HERE (3 lines): set dyhat_db, dyhat_domega, and dyhat_dphi.\n", " dyhat_db = -x * A * E * S\n", " dyhat_domega = x * A * E * C\n", " dyhat_dphi = A * E * C\n", "\n", " # Combine: each entry is -2/N times the sum over data points of residual * derivative.\n", " # The @ operator between two 1-D arrays does exactly that sum of products.\n", " return -2.0 / N * np.array([residuals @ dyhat_dA,\n", " residuals @ dyhat_db,\n", " residuals @ dyhat_domega,\n", " residuals @ dyhat_dphi])" ] }, { "cell_type": "markdown", "id": "5a9e20c8", "metadata": {}, "source": [ "Gradient check, at three random starting points. Do not move on until it passes — a\n", "wrong gradient will cost you far more time in 3(c) than it costs you to find it here." ] }, { "cell_type": "code", "execution_count": 17, "id": "2dc3c395", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "trial 0, p = [3.313 1.794 7.869 1.415]\n", " your gradient: [ 0.28724 -0.402883 0.226843 0.97568 ]\n", " numerical : [ 0.28724 -0.402883 0.226843 0.97568 ]\n", " relative error = 1.545e-11 -> PASS\n", "trial 1, p = [1.851 1.747 0.55 5.16 ]\n", " your gradient: [ 0.437014 -0.233942 -0.183986 -0.571009]\n", " numerical : [ 0.437014 -0.233942 -0.183986 -0.571009]\n", " relative error = 3.081e-11 -> PASS\n", "trial 2, p = [4.087 0.936 3.379 1.749]\n", " your gradient: [-0.056604 0.177974 0.436748 1.179488]\n", " numerical : [-0.056604 0.177974 0.436748 1.179488]\n", " relative error = 3.721e-11 -> PASS\n", "\n", "gradient check passes at every test point\n" ] } ], "source": [ "# --- PROVIDED --- do not modify\n", "\n", "rng = np.random.default_rng(7)\n", "for trial in range(3):\n", " p_random = np.array([rng.uniform(0.5, 5.0), rng.uniform(0.0, 2.0),\n", " rng.uniform(0.5, 10.0), rng.uniform(0.0, 2 * np.pi)])\n", " print(f\"trial {trial}, p = {np.round(p_random, 3)}\")\n", " err = grad_check(model_loss, model_grad, p_random, xs, ys)\n", " assert err < 1e-7\n", "\n", "print(\"\\ngradient check passes at every test point\")" ] }, { "cell_type": "markdown", "id": "d05f0131", "metadata": {}, "source": [ "### 3(c) Fit the model with three different learning rates\n", "\n", "One line to write: the call to your `gradient_descent` from Problem 1. It works here without\n", "any changes, because `model_loss` and `model_grad` follow the same `(data, data, parameters)`\n", "convention that `compute_loss` and `compute_grad` did.\n", "\n", "The starting point is provided. It is not arbitrary: the scatter plot shows about 2.5 cycles\n", "over $x \\in [0, 4]$, so the period is roughly $1.6$ and $\\omega = 2\\pi/1.6 \\approx 3.9$. Make\n", "sure you see where that came from." ] }, { "cell_type": "code", "execution_count": 18, "id": "c0b29bd6", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "gamma = 0.002 -> p = [3.0002 0.5841 4.0034 0.4775], final MSE = 0.0085\n", "gamma = 0.01 -> p = [3.001 0.5843 4.0032 0.4779], final MSE = 0.0085\n", "gamma = 0.05 -> p = [3.001 0.5843 4.0032 0.4779], final MSE = 0.0085\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAnAAAAGJCAYAAAAKZg7vAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAXI1JREFUeJzt3Qd8U+X6B/An3S0thTJa9p4yhTIEAQFlOdEL4mApIBTH5SrKVRFceFERFf4oegFRlHEFVGQIiICADGUIZQiUIXsWSndz/p/fm56QpGmaQtucNL8vn5AmOefkvOckPU+fd5k0TdOEiIiIiLyGn6d3gIiIiIjyhwEcERERkZdhAEdERETkZRjAEREREXkZBnBEREREXoYBHBEREZGXYQBHRERE5GUYwBERERF5GQZwRERERF6GARzRDTCbzdKoUSN566238rXe8uXLJTw8XM6dO3dTx/3IkSNiMpnkvffeu6ntUNH65Zdf1HnDfVGbNWuWem98dozMk8eIyJswgCMSkT179sg//vEPqVmzpoSFhUnZsmWlQ4cO8sMPPzg9Pt98840cP35cRo4cma/j1717d6ldu7ZMmDDBreWXLl0q48aN4zki8hIbN25U39nLly97eleomGMARyQiR48elatXr8qAAQPkww8/lFdffVUdl3vvvVemT5+e4xi9++678vDDD0tkZGS+j9+wYcPk008/Ve/nTgA3fvx4nqNiAn8UpKSkqHsqnscIARy+swzgqLAxgCMSkZ49e6rqzddee02GDBkizz77rKxZs0aaNm0qkyZNsjtG27dvl507d0qfPn1u6Ng9+OCDkpaWJgsWLPDYsb927ZrH3rs4ye9x9PPzk5CQEHXvK7z9GPG7QkZljG8IkQH5+/tLlSpVcvwlvXjxYgkKCnKaIThx4oQ88cQTUrFiRQkODpYaNWrI8OHDJT093bpM+fLlpUmTJvLdd9+5fP+BAwfK1KlT1c9oE6TfHCFDWKtWLfV+sbGxsnXr1hzbQbu7Q4cOqUA1IiJCHn30UWtbvsmTJ8stt9yiLprR0dEqQ3jp0qUc77Ns2TK5/fbbpUSJEmobvXr1UlXP7sAx/Oc//ynVq1dX+1m5cmXp37+/nD9/3rrM2bNn1bHDPmBfEDx/8cUXubb9c1VuvI7lkFl1NGbMGHX+bMu4efNmVb2NjCqq0Dt27CgbNmywWw/VYthmfHy8PPLII1K6dGlp37699TjidZx3rH/HHXeo5VBeHH9X7bs6deqk2lNieayH9StVqiQTJ07Mse8oD7LCOAf4HOGYrlix4qbajLlzXnft2qXKgSYGODcxMTEyePBguXDhgtvHCMfi7rvvll9//VVatWqltoPtzZ49224bnjxGrvbfnWOA9V944QX1M777+nfWtt3hV199JS1atJDQ0FCJiopSmXw0x7D1119/qT/08B54L3xfsFxiYqLL/SffEuDpHSAyEvy1jeob/KL8/vvv1cWtb9++OapIcDEJDAy0e/7kyZPqwoRgZejQoVK/fn0V0P3vf/+T5ORkFTTo8AscgaArCKSwzZUrV8qXX37pdJmvv/5aVcViWVwocEHr3bu3HD582G7/MjMzpVu3bupihOAGF0D9PdC4fdCgQfLMM89IQkKCTJkyRWUZEcDo28D7o3oZ2/jPf/6jyjNt2jS1PSyLi3NukpKSVICwd+9edcG79dZbVeCG4/v333+r9oY45rhIHzx4ULUrxMUPGUpcMHE8kRHNT7mRHR09erTMnz/fekHV4bm77rpLXZzh559/lh49eqhzggwsMj8zZ86Uzp07y/r169U5tYW2knXq1JG3335bNE2zBoXYh3vuuUcdI2RocZ+amiruQDCJABJlwL7jM/Piiy9K48aN1b7pn03s06lTp9TxwMUdxwGZ4hvl7nnFZxDHFp8TvC8CPATQuP/tt99y/GHh7BgBzu9DDz2kAnW874wZM9Q5xrHHHxFGOUbO9t+dY4B9O3DggGoj+8EHH6jPNpQrV07do9MTmmdg/5988knVmenjjz9WfwzieJcqVUr9sYfzgSz9008/rd4Lv0eWLFmivgs30myDiimNiKyGDRuG39bq5ufnpz300EPaxYsX7Y5Q5cqVtQcffDDHUevfv79aZ+vWrTleM5vNdo/ffvtt9R5nzpxxefTj4uLUco4SEhLU82XKlLHbv++++049/8MPP1ifGzBggHrupZdestvG+vXr1fNz5syxe3758uV2z1+9elUrVaqUNmTIELvlTp8+rUVGRuZ43tHYsWPV9hYuXJjrcZk8ebJa5quvvrK+lp6errVt21YLDw/Xrly5ku9yY90WLVrYvd+WLVvUcrNnz7a+f506dbRu3brZnaPk5GStRo0a2p133ml97rXXXlPr9uvXL8dxCAgI0O6//36758eNG6eWx/HXrVmzRj2He13Hjh3t9gnS0tK0mJgYu8/Z+++/r5ZbvHix9bmUlBStfv36ObbpzMyZM9VyOIb5Pa84Ho6++eYbtb1169bleYygWrVqOZY/e/asFhwcrP3rX/8yxDFytf/uHoN3333X7jjrjhw5ovn7+2tvvfWW3fN//vmn+vzoz2/fvl2tv2DBApf7SsQqVCIbzz33nPpLG1V3+Ks+KyvLrvoTUGWiZ290qEJDRg0ZmJYtW+Y4po4ZCn192yrEG4HsoO2+INMFyBQ4QlWuLWS48Nf8nXfeqfZDvyEbgipXPWuB44G//Pv162e3HKqYW7dunWd249tvv1XVoQ888ECuxwWdNZBpwHvokElDVhAZvLVr1+a73Fjm999/V1XHunnz5qkq1/vuu0893rFjh6quQnUZzqteNmRyunTpIuvWrVPn1tZTTz1l93j16tUqwzlixAi755E9cReO92OPPWZ9jGwtMn+25UEbTVQbonpQh+o1tNm8Efk5r6ju0yGriOXatGmjHv/xxx85tu14jHQNGza0nis9M1WvXj2nn1dPHiNn+5/fY+Bo4cKF6rOE7Jvt8cbnHtk+/XjrGTZU+yIjSpQbVqES2UC1J26ANlqoakNQhjZStkGYbbUQoCrkypUrqmrVHfr6ztq05UfVqlXtHutBjWMbtoCAANWOxhYCF1QVo52QM2iTpi8HqJpypmTJki73EQEU2vO4gnZLuIg5Nlxv0KCB9fX8lhvVYKNGjVJB27///W91zBG0IjDX91kvG6rzcoNjZBssonrXcd8Bw8PYQvsmx0A/Nzg3zoJ8tLuyfR+0+XNczvF93ZWf83rx4kXVs3Lu3LnWz4XOWbssx2OU23nTy+mszaUnj5Gz/c/vMXB2vPEZxOfcGb25At4bn1t0npozZ44KeBGQInhl9SnZYgBH5ALa66CdFdq1IFMAZcqUceuC44q+vt5G5kYhW+KMY4CJrJNjcIRsAII3XCSc0dvt6BkotJdCtsARgsOi5k650aEAFz+0eUMAh3ZKx44dU229dHrZMCxMs2bNcs385JaJKSjunseClJ/ziqwR2n6iPSGOE44J1kebNMcMpatjdDPlLMpj5Gz/83sMHGEZBJZoV+usLLafs/fff1+1DURHp59++kllojF2JD7Djn+Ike9iAEfkAhrXO/6FjQwdGvs7BjvIWOzevdut44n1EbzpQVJubjZD5woyFatWrZJ27dq5DEqwHCDY69q16w29T17HpVq1aiqTgoucbaC5b98+6+s3AtWoqNrcv3+/ysSh8wYyqrb7Bjh3N1I2231DA33bzA2qZG820Hd8H/SORMBi+7nA+94Id88ryoBqYmSfxo4dmyODZyQFfYxu5Bjk9p3F8cZ+4TNSt27dPN8TnTNwe+WVV1TgiO/pJ598Im+++eZNlYWKD7aBI7KpLrSVkZGhhjhAcIO2O7q2bduqgAS9xKxfJD8/uf/++9XMDdu2bcszS4C2WdhOXjAUAhTGoKDIKKCN3xtvvJHjNbTp0t8TPeIQ4KBHHo6Jo7ymBUP1KXplLlq0KNfjguFNTp8+rYIs231ADz1kJjCsx43AeyPbgV6BqD7FMBb6MQW098OFFT1z0dYuv2UDtJVDtgq9N22hN29BwnlAb0T03rVti/XZZ5/d8PbcOa96tsjxM4zhZ4ymoI+RLj/HILfvLHqoYjsIAh23g8f6cCRoioHPvi0EcvgdY/s7h4gZOKLs4TTwixPd+dEIGsEEqhaRAUJ1hm31BhrAI+hBw3q0kdPhQojqDgQbGEYE7bcwnAECB4x9hSEC9GAR2aa4uLg8jz0CDEAVCi5OuABgPKiCgP1EuVE1g8b8KAva4SCrgH3GjBSoQsZFHsHJ448/roYAwfsjc4jqyB9//FFlBlwFK6hywpAPaJOGYURQJrQnwkUWGQV0cMDxwuwUqDZCcIvhK7AOhjLBRRLjk90IZJcwbhjaE2HYEcchYXBR/Pzzz1W7OAxjgSEicP4RBKBROcqe23RqOoxbhyEr8DlBWyVUqSFgRVUZsqwFlUXFucJxRqcDvF+FChXUZxSN9CG/7+PuecVy+F5gmBQEejg++Jw7ZqGNoKCPkS4/x0D/zr788svqmOI7hawv/lBA9gxDzmBcOPzBh881toE/bvAdeP7559WwNhhKB98XZOoQzKGaG9/9vNqSko9hR1wiy3AAXbt21aKjo1WX/tKlS6vHGJ7CmSZNmmhPPPFEjuePHj2qhhMpV66cGh6hZs2aaigQDHmgmzZtmhYWFmYdGsOVzMxM7emnn1bbM5lM1iFF9OE0MGSBIzyP4RB0GMaiRIkSub7H9OnT1XAboaGhWkREhNa4cWNt9OjR2smTJ+2WwxAMGG4DQ0yEhIRotWrV0gYOHKht27Ytz3JcuHBBGzlypFapUiUtKChIDcWC/Tp//rx1GQypMmjQIK1s2bJqGewHhr6wlZ9y6z777DP1GsqGISWcwdANvXv3VsOT4LxhyIs+ffpoq1evzjHExLlz55yep1dffVUNa4Hj2LlzZ23v3r1qe0899VSeQ2TccsstObaJ44P9sHX48GGtV69e6j3wmcDwG99++63a5m+//ablZxiR/JzXv//+W3vggQfUsCNY7h//+If6fDgec1fHCGXBvjtC+XEzwjFytf/uHgN444031Gcdwwo5HnPsS/v27dV3EjcMcYLfEfv377fu/+DBg9V5wPmIiorS7rjjDm3VqlUu9518jwn/eTqIJPI2+IsYGTRkK/TMmruaN2+uBq3FQJ9UfKEKDb0kkXVBNqawIEOJ2QYwKDIyQ8RjRL6BbeCIbgCmosKQCPpUV+7COFWookQ1ChW/zi7O2kchWC+s90H7LlQ9Y2gKBm88RuRb2AaO6Aag7ZS7PU5toX2Us8by5N3Q+QJTkqEzBtpLos0jOk6gXSHakhUUNITHHw4YxgI9ozGvJtpp5jYUjC/iMSJfwQCOiOgmNWnSRPVERSN3dIbROzYU9JAP6MiCThcI2NCDGL2jMbCsY+cMX8ZjRL6CbeCIiIiIvAzbwBERERF5GQZwRERERF6GbeBcwLQ+J0+eVIMtFuaURkRERESA0d0w8Djmc3acw9oWAzgXELxVqVLF1SJEREREBe748eNSuXLlXF9nAOeCPn0PDiKmUimsLB/mHMQUNq4i7eLGF8vti2UGltt3zjfPte+ca1893+YiKDN6siN5lNcUggzgXNCrTRG8FWYAh8E4sX1f+QL4arl9sczAcvvO+ea59p1z7avn21yEZc6r6ZZvHHEiIiKiYoQBHBEREZGXYQBHRERE5GXYBo6IiHxumIbMzEw1HVlBtYvKyMhQbaN8pS2Yr5bbXABl9vf3V1Pv3ezwZAzgiIjIZ6Snp8upU6ckOTm5QANCXNgxdpcvjRnqi+XWCqjMYWFhUqFCBQkKCrrhbTCAIyIin4ALb0JCgsqAYJBUXDwLIvDQM3oFkVXxJr5Ybu0my4z18UcEhiLBZ7FOnTo3nMljAEdERD4BF04EcRhjCxmQguKLgYyvllsrgDKHhoZKYGCgHD16VH0mQ0JCbmg7vlFpTURElM1X2mtR8f4M8lNMRERE5GUYwHnYrB/fkvlr35UzF054eleIiIjIS7ANnIfNODdXEv39pPnxTlKhXBVP7w4RERF5AWbgDCKzgMYjIiIi8lXHjh2TXr16qU4q5cuXlxdeeEF1OnDl4sWL8uijj6r5TUuVKiVPPPGEJCUl2S2za9cuuf3221UHhJo1a8rEiRPtXv/ss8/U66VLl1a3rl27ypYtW6QwMYAziEyz5uldICIi8lpZWVkqeEPPzo0bN8oXX3whs2bNkrFjx7pcD8Hbnj17ZOXKlbJkyRJZt26dDB061Pr6lStX5K677pJq1arJtm3bZMKECTJ+/HiZPn26dZlffvlF+vXrJ2vWrJFNmzapns5Y58SJQmwepVEOU6ZM0Ro0aKDVrVsXUZWWmJhYaEep3X8bao1mNdKWb5zvU2ciKytLO3XqlLr3Fb5YZmC5fed8G/1cp6SkaPHx8epeZzabtWtpGTd1S0pN1y4nJat7d9fB++bHlStXtEceeUQLCwvTYmJitEmTJmkdO3bUnn32Wesys2fP1lq0aKGFh4dr0dHRWr9+/bQzZ85YX1+zZo26pi1fvlxr1qyZFhISot1xxx1qmaVLl2r169fXIiIi1HrXrl2zrof3GTlypHqvUqVKaeXLl9emT5+uXb16Vevfv796v1q1aqlt6DIzM7XBgwdr1atXV++D6+nkyZO1wrR06VLNz89PO336tPW5adOmaSVLltTS0tKcroPPA47J1q1brc8tW7ZMM5lM2okTJ9Tj//u//9NKly6ttoHzlp6ero0ePVqrV69ervuC8uNYfvHFF25/FnWIOdyJPdgGzom4uDh1Q9QdGRkpRYE1qERERS8lI0sajl1R5O8b/3o3CQty/xI8atQo2bBhg3z//fcSHR2tskp//PGHNGvWzLoMpnh64403pF69enL27Fm1zsCBA2Xp0qV22xo3bpxMmTJFVTP26dNH3YKDg+Xrr79WVYcPPPCAfPzxx/Liiy9a10E2a/To0apacN68eTJ8+HBZtGiR3HvvvfLyyy/L5MmT5fHHH1dVmNguxturXLmyLFiwQMqUKaMyYshqYfYBvF9uwsPDXR6Hxx57TD755BOnryHz1bhxY3V8dN26dVP7igxb8+bNna6DatOWLVtan0P1J4b52Lx5szoWWKZDhw5q4GeMA6dvF9Woly5dUlWmjjDTB85HVFSUFBYGcAaRaWYbOCIiygnTNiGAQoDVpUsX9dzMmTPVbBK2Bg8ebP0Z7bQ++ugjiY2NVUGZbWD05ptvSrt27dTPaO81ZswYOXTokFoHHnroIVUVaBvANW3aVF555RX1M5Z/5513pGzZsmp9DGqLgHLatGmqrVibNm3UQLWoZtTVqFFDBULz5893GcDt2LHD5UcA7dRyc/r0abvgDfTHeC23ddBWzhbKg8BLXwf32P/ctussgMOxw/lBMFhYGMB5mD6Oc1aW2cN7QkTke0ID/VU2rKhH58f7uuvw4cMqm9OqVSvrc6gdQqbN1u+//66yazt37lSZIWTBAFmxhg0bWpdr0qSJXSCCjJkevOnPOTbAt10HU5Ehq9aoUSO7dQCZP93UqVNlxowZ6v1TUlJU2zTbjKEztWvXFm/3zjvvyNy5c1W7uBudZcEdDOA8zvJlz2IGjoio6H8Dm0z5qsrMNYDzE49OKXXt2jVVrYfbnDlzpFy5cipwwmMETraQHdNhf20f68/pwZ+zdZytp5dbXw8BzPPPPy/vv/++tG3bViIiIuTdd99V1ZKu3EwVakxMTI7A88yZM9bXclvHNugEBOPomaqvg3t9O3lt97333lMB3KpVq+yC3sLAAM4gmIEjIiJnkB1DsLR161apWrWqei4xMVEOHDig2mbBvn375MKFCyp4QA9IQI9JT0F7vdtuu01GjBhhfQ7VtHm5mSrUtm3byltvvaUCMr1aFD1LsY5tBtJxncuXL6vsZYsWLdRzP//8swpEW7dubV0G7fyQBUWQrm8XGVDb6lO0icP7r1ixwq5NXWHhMCIepv+txmFEiIjIGWSvBgwYoMY0Q9s0NMhH2zM0tNczXwjs0MgenQ9Q5YrODujQ4Cl16tRRASSCGQSar776qgpA84IqVFc3x/ZqtjBsBwI1dKZANTLeG+320CkRnTQAGbr69etbh/do0KCBdO/eXYYMGaJeQ+A5cuRIefjhh61tDB955BF1bHHMcezRjg/tC9FJRPef//xHlRFVxtWrV1dt43BzHE+uIDGAM4gsh3Q1ERGRbtKkSSoTdPfdd6uG8eiEgOBDb2OFKlOMeYZenwhikIlDdZ6nDBs2THr37i19+/ZVmSxkB22zcYXB399fjeOGexwrVLf2799fXn/9dbveofv371fZNB2qnBHUoYNIz549pX379nZjvKG94U8//SQJCQkqs4YOCgjWbMeKQwcOVFWjAwh62uq3wjwHJowlUmhb93L6MCJIVbtK296MDjMaySV/k/yrzDMy8O4h4iuQntbT3Pgr0hf4YpmB5fad8230c52amqouwuhRWJCNy2+kE0NBtHmrVKmSamOGzJAneKLcnqYVUJldfRbdjT3YBs4g2AaOiIhys337dtXODT1RcWHXs0r33XcfD5qPYgBnEFkaq1CJiCh3qI5D9R/aY6HB/fr169VYbOSbGMAZRCbbwBERUS4wiwB6ShLpjNdIwcfoNegaB/IlIiIiNzGAM4hMVqESERGRmxjAGWUmBmbgiIiIyE0M4AxShWo2czQXIiIicg8DOIMwcy5UIiIichMDOIPgTAxERETkLgZwRqlC5YQYRERE5CYGcAaRxSpUIiKim3Ls2DHp1auXhIWFqSndXnjhBTX1lSsXL16URx99VE1bVapUKTU1me0k9Jj2auDAgdK4cWMJDAyUBx980BBniQP5GgQ7MRAREd24rKwsFbzFxMTIxo0b5dSpU2oyewRdb7/9dq7rIXjDsitXrlST3A8aNEhNVP/1119btxsaGirPPPOMfPvtt4Y5RczAGaQKlW3giIg8AM1X0q8V/S2fzWauXr2qAo0SJUpIhQoV5IMPPpBOnTrJc889Z13myy+/lJYtW0pERIQKYh555BE5e/as9fVffvlFTcC+YsUKNbMDgpLOnTurZZYtWyYNGjRQWSisl5ycbF0P7/P000+r9ypdurRER0fLZ599JteuXZMnn3xSrVO7dm21DR2CHmSyMFk73qdevXry4YcfSmH66aefJD4+Xr766itp1qyZ9OjRQ9544w2ZOnWqpKenO11n7969snz5cvn888+ldevW0r59e/n4449l7ty5cvLkSbUMjvm0adNkyJAh6rgaBTNwhgjhNDFzKi0ioqKXkSzydsWb/i0emN+V/n1SJKiE24uPGjVKNmzYIN9//70KoMaOHSt//PGHClR0yB4hYEGwhKAM66Dqb+nSpXbbGjdunEyZMkVVM/bp00fdgoODVcYJVYcPPPCACmJefPFF6zpffPGFjB49WrZs2SLz5s2T4cOHy6JFi+Tee++Vl19+WSZPniyPP/64qsLEdnFNq1y5sixYsEDKlCmjMmLIaiH4xPvlJjw83OVxeOyxx+STTz5x+tqmTZtUNSeOj65bt25qX/fs2aOCVmfroNoUga+ua9eu4ufnJ5s3b1bHwqgYwBkkBccqVCIiyi37hgAKAVaXLl3UczNnzpSKFe0Dz8GDB1t/rlmzpnz00UcSGxurgjLbwOjNN9+Udu3aqZ+RJRszZowcOnRIrQMPPfSQrFmzxi6Aa9q0qbzyyivqZyz/zjvvSNmyZdX6AQEBKqBElmrXrl3Spk0bVW05fvx46/rIxCFYmj9/vssAbseOHS4/BMj25eb06dN2wRvoj/FabuugrZwtlCcqKirXdYyCAZxBMANHROQBgWGWbNhN0DRNNZTHhR9VlG6/r5sOHz6ssmutWrWyPhcZGakybbYw2T2yazt37pRLly5ZryvIijVs2NC6XJMmTewCHGTM9OBNfw6ZNlu26/j7+6usWqNGjezWAdsqW1RdzpgxQ71/SkqKqsa0zRg6g6pYcg8DOMMMI2L5ohERUVH+EjblqyrTKbRn88tE6sayPQ9AezRUF+I2Z84cKVeunAqc8Nix/ReyYzoEnLaP9ecckwrOlnHcDujroQ3Z888/L++//760bdtWtct79913VbWkKzdThRoTE5Mj8Dxz5oz1tdzWsQ06AcE4eqYaqb2bMwzgDIIZOCIicgbZMQRLW7dulapVq6rnEhMT5cCBA9KhQwf1eN++fXLhwgVVtVmlShX13LZt2zx2QNFe77bbbpMRI0ZYn0M1bV5upgq1bdu28tZbb6mATK8WRc9SrGObgXRc5/Llyyp72aJFC/Xczz//rK7J6NRgZAzgDIID+RIRkTPIXg0YMECNaYa2WQhOXnvtNdXQXs98IbALCgpSnQ+eeuop2b17t+rQ4Cl16tSR2bNnqx6vaP+GHrIIQPFzYVWh3nXXXSpQQ2eKiRMnqjZsaLcXFxenOmkAMnQYWmT16tVSqVIl1fO2e/fuqocpMnuoqh45cqQ8/PDDdm0M0bsVmUxk5q5cuaICTRz7vKqECxOHEfEwU3YlKjNwRESUm0mTJqls0d133616SaITAoKPkJAQ9TqqTGfNmqV6fSKIQSbuvffe89gBHTZsmPTu3Vv69u2rMlnIDtpm4wqDv7+/LFmyRN3jWKG6FcHa66+/bl0Gw6Ps379fBWo6VDnXr19fdRDp2bOnGkpk+vTpdtvG8+jF+sMPP8jatWvl1ltvddqrtSiZNLS+JKcQZaOhKFLVrtK2N6PrzKZyxs8sHc93kyn/8tyXraghYNXT3Pgr0hf4YpmB5fad8230c40R9RMSElQWSA98CsINdWIogDZvyCChjRl6gnqCJ8rtaVoBldnVZ9Hd2INVqB6mn37NzDiaiIic2759u2rnhp6ouLDrWaX77ruPh8xHMYAzCPZCJSIiV1Aliuo/tHVDg/v169ersdjINzGA8zSHrtdERESO0N4KPSWJdMZrpOCr48AxgCMiIiI3MYAzCA4jQkRERO5iAGcQGjNwRERE5CYGcIaZSktT3ZOJiIiI8sIAzjgDiUgmhxIhIiIiNzCAM1AYl5HFnqhERESUNwZwHqYP5OwnmmRksQqViIiI8sYAzjAQwDEDR0REdKOOHTsmvXr1krCwMDWl2wsvvKCmvnIFE9Q/+uijatqqUqVKqanJkpKSrK8fOXJETZuFG6aIw0DKuP/tt988eqI4kK/HWVJwJpMmmczAERER3ZCsrCwVvMXExMjGjRvl1KlTajL7wMBAefvtt3NdD8Ebll25cqWa5H7QoEEydOhQ+frrr+2WW7VqlTRs2NA6F6qnZ8FgBs4gXRjYBo6IqOih939yRvJN31IyU/K1fH5HHbh69aoKNEqUKCEVKlSQDz74QDp16iTPPfecdZkvv/xSWrZsKRERESqIeeSRR+Ts2bPW13/55ReVRVqxYoWa2SE0NFQ6d+6sllm2bJk0aNBAZaGwXnJysnU9vM/TTz+t3qt06dISHR0tn332mVy7dk2efPJJtU7t2rXVNmyDKWSyMFk73qdevXry4YcfSmH66aefJD4+Xr766itp1qyZ9OjRQ9544w2ZOnWqpKenO11n7969snz5cvn888+ldevW0r59e/n4449l7ty5cvLkSbtly5Qpo46rfkNg6EnMwBkGq1CJiIoaAq/WX7cu8vfd/MhmCQsMc3v5UaNGyYYNG+T7779XAdTYsWPljz/+UIGKDtkjBCwIlhCUYZ2BAwfK0qVL7bY1btw4mTJliqpm7NOnj7oFBwerjBOqDh944AEVxLz44ovWdb744gsZPXq0bNmyRebNmyfDhw+XRYsWyb333isvv/yyTJ48WR5//HFVhYntYnahypUry4IFC1Tgg4wYsloIPvF+uQkPD3d5HB577DH55JNPnL62adMmady4sTo+um7duql93bNnjwpana2DalMEvrquXbuqKtLNmzerY6FDWVNTU6VOnTrqWNx3333iSQzgDMKSgWMnBiIiypl9QwCFAKtLly7quZkzZ0rFihXtlhs8eLD155o1a8pHH30ksbGxKiizDYzefPNNadeunfoZWbIxY8bIoUOH1Drw0EMPyZo1a+wCuKZNm8orr7yifsby77zzjqpCxPqoTkRAOW3aNNm1a5e0adNGZafGjx9vXR+ZOARL8+fPdxnA7dixw+XpR7YvN6dPn7YL3kB/jNdyWwdt5WyhPFFRUdZ1cOzef/99dcyQwURQisBu8eLFKqjzFAZwHmbS28CJJumZ7MRARFSUQgNCVTbsZqA6FFWG/v7+6gLv7vu66/Dhwyq71qpVK+tzkZGRKtNmC5PdI7u2c+dOuXTpknWObWTF0HZL16RJE7sABxkzPXjTn0OmzZbtOignsmqNGjWyWwdsq2xRdTljxgz1/ikpKaoa0zZj6AyqYo2mbNmyKpupn2tk8s6cOSPvvvsuAzhSHwtJZy9UIqIihYArP1WZzuCinmmyNGx3N4AraGiPhupC3ObMmSPlypVTgRMeO7b/sm27hf11bMuF5/Tgz9k6ztbTy62vhzZkzz//vMpctW3bVrXLQ8CDaklXbqYKNSYmJkfgiUBLfy23dWyDTkAnBfRMzW0dQDCNTg+exAych+lfdYwDxwwcERE5QnYMwdLWrVulatWq6rnExEQ5cOCAdOjQQT3et2+fXLhwQVVtVqlSRT23bds2jx1MtNe77bbbZMSIEdbnUE2bl5upQm3btq289dZbKiDTq0URZGEd2wyk4zqXL19W2csWLVqo537++WcViKJTg6v9RHs+T2IAZ4QITsuuQmUGjoiIHCB7NWDAADWmGdpmITh57bXXVEN7PfOFwA7jk6HzwVNPPSW7d+9WHRo8BQ39Z8+erXq8ov0besgiAMXPhVWFetddd6lADZ0pJk6cqNqwod1eXFyc6qQByNBhaJHVq1dLpUqVVM/b7t27y5AhQ1RmD1XVI0eOlIcfftjaxhDtD3FsUXWKbOv//vc/1QYRPVc9icOIGAS6LzADR0REzkyaNElli+6++27VSxIN6hF8hISEqNdRZTpr1izVwB5BDDJx7733nscO5rBhw6R3797St29flclCdtA2G1cY/P39ZcmSJeoexwrVrQjWXn/9desyGB5l//79KlDTocq5fv36qoNIz5491VAi06dPt9s2gmFk6NBB44cfflBVxBgvzpNMWn4Ho/EhV65cUQ1Fkap2lba9GXfPbilHtTS5/dit0uu+d6RXE8+mZIsK0tN6mht/RfoCXywzsNy+c76Nfq4xBERCQoLKAumBT0FQbeAyi7YNHNq8IYOENmboCeoJnii3p2kFVGZXn0V3Yw9WoRqEagOXleXp3SAiIgPavn27aueGxvO4sOtZJU+PRUaewwDOMDTJyGQylIiInEOVKKr/0B4L1Xnr16/3+HRO5DkM4AwCidg0dmIgIiIn0IAePSWJdMZrpOBzsuvQTRxGhIiIiNzjEwEcprzABLyYHsSoOBMDEVHRYN89Kg6fQZ8I4J599lk1Ho0RmWzuOYwIEVHh0WcOwFASRJ6kfwYdZ7jID59oA9epUyf55ZdfxMgD+Vqm0mIvVCKiwoLxwUqVKmWdOglzgBbE8Be+OJyGr5Zbu8kyY30Eb/gM4rOIz6TXBnDr1q1T86OhceapU6dk0aJFcv/999stgwlxsQxGVW7atKkaadp2Ul/vxsnsiYiKij6/peP8lzcDF2WMgWc7M4Iv8MVyawVUZgRvruZa9YoADoMRIigbPHiwGrXZ0bx582TUqFFqiguM5jx58mQ1OS+6UutznTVr1kxFxI5++ukn61QYRnW9ClWTjCwOI0JEVKi/c00mNYclrh+2o/HfDFzQMdNAmTJlDDmAcWHxxXKbC6DMqDa9mcybYQK4Hj16qJur6UMwR5k+ZQUCuR9//FFmzJghL730kluT37orLS1N3WxHQ9ZPGG6FPoxIRlahv49RoJz6XzK+whfLDCy375xvbzrXCOQwnlpBQHlRpYbt+Uog46vlNhdQmV19R9z9/ng8gHMlPT1dVa2OGTPG+hwOGOaB27RpU4G/34QJE2T8+PE5nj937pya9qIwmM2aNQOXmGSpF/cF+IBiNHH8svelL76vlRlYbt853zzXvnOuffV8m4ugzFevXvX+AO78+fOSlZUl0dHRds/jMaYUcRcCvp07d6rq2sqVK6vJfjHRrSMEiqiutc3AValSRU0SXFhzofrjA5AdbPsFBlmrhX3hS4C/gHFsfemL72tlBpbbd843z7XvnGtfPd/mIiizu/P0GjqAKyirVq1ya7ng4GB1c4STVNgfTksbOEvDSF+BL0FRHFsj8cUyA8vtO+eb59p3zrWvnm9TIZfZ3e0a+ohjjjc09Dtz5ozd83h8s703jIczMRAREZF4fwCnT9i7evVqu/QlHjurAvVGejdk/M9eqEREROQVVahJSUly8OBB6+OEhATVqzQqKkqqVq2q2qQNGDBAWrZsqcZ+wzAiaMum90otLjiVFhEREXlNALdt2za54447rI/1TgQI2mbNmiV9+/ZVvUDHjh2rBvLFmG/Lly/P0bHB25lMmqRlGb/rPREREXlegBGmucprUteRI0eqW3HHuVCJiIjI69vA+QaTTScGzoVKREREeWMA5wTmXm3YsKHExsZKUYZx6axCJSIiIjcwgHMiLi5O4uPjZevWrVJU+Tc/MUtGJudCJSIiorwxgDMQZuCIiIjIHQzgPE4fB44D+RIREZF7GMB5WPY4vgp7oRIREZE7GMAZhMrAZZnzHFKFiIiIiAGcwbAdHBEREeWFAZyHmWzawAGrUYmIiCgvDOCMwmQJ4DihPREREeWFAZzHWTJw/szAERERkZsYwHl4JgbrQL7Z3VFZhUpERER5YQDn4ZkY9AjOL/tMpGdxPlQiIiJyjQGcUapQs9vApWWaPbw/REREZHQM4DzsehWq5Z5VqERERJQXBnAGoQdw7IVKREREeWEA52Fadg6OGTgiIiJyFwM4g1Sh+utVqOzEQERERHlgAOdpei/U7Pu0DHZiICIiItcYwBmmF6rlEXuhEhERUV4YwBmlF6qfPowIx4EjIiIi1xjAGYR/9n0qq1CJiIgoDwzgPC67F2r2mWAGjoiIiPLCAM4oc6Fm37MTAxEREeWFAZzH50K1z8Clsg0cERER5YEBnEFwGBEiIiJyFwM4D3OsQmUGjoiIiPLCAM7j7KfSYhs4IiIiygsDOIOwBnCZnImBiIiIXGMAZ5gMnGUg39QMDuRLRERErjGAM9pcqMzAERERUR4YwBmEHsAxA0dERER5YQDnYSa9ClX0uVDZBo6IiIhcYwBnEHobOAZwRERElBcGcJ6WPRODPh5cGjsxEBERUR4YwBmlCpUZOCIiInITAziD0DNw7MRAREREeWEA58TUqVOlYcOGEhsbK0U2mb1Yxn9jGzgiIiLKCwM4J+Li4iQ+Pl62bt0qRVWFam0Dl8mBfImIiMg1BnAGYcoeRiQjS5Mss+VnIiIiImcYwBksgANm4YiIiMgVBnAeZrIOI3I9gEvN4GC+RERElDsGcB53PYAL9Lf8zAwcERERucIAzjA0CQ7wVz+lMQNHRERELjCAM0gVqqaZJSTQcjpS2ROViIiIXGAA53HZA4howgwcERERuYUBnGGYJTggOwPH+VCJiIjIBQZwnmatQtUkODC7DVwme6ESERFR7hjAGYV2PQPHAI6IiIhcYQDncfokWtr1TgysQiUiIiIXGMAZpheqzTAirEIlIiIiFxjAGaYXqs0wIszAERERkQsM4DzMlB3AYSItPQPHAI6IiIhcYQDnxNSpU6Vhw4YSGxsrRUYzS2h2L1QGcEREROQKAzgn4uLiJD4+XrZu3SpF14dBk9AgSwCXwipUIiIicoEBnIeZrKfAbA3gktOzPLpPREREZGwM4IxC0ySMVahERETkBgZwnpY9jIhqA8cMHBERERV0ADdx4kRJSUmxPt6wYYOkpaVZH1+9elVGjBiRn036vOu9UG3awLEKlYiIiAoqgBszZowK0nQ9evSQEydOWB8nJyfLp59+mp9Nkth0YsiuQmUnBiIiIiqwAA6zBbh6TDfRC1U0CWMGjoiIiNzANnAGOQUIhkODAtTP7IVKRERErjCAMwqbgXxZhUpERESuWFI++fD5559LeHi4+jkzM1NmzZolZcuWVY9t28dRPnuhsgqViIiICiOAq1q1qnz22WfWxzExMfLll1/mWIZuoBeqpklIdgYuOT2Th5CIiIgKJoA7cuRIfhYnd1gTcGZrJ4bUDDOPHREREeWKbeCM0w3V2gYuPcssmVkM4oiIiKgAArhNmzbJkiVL7J6bPXu21KhRQ8qXLy9Dhw61G9iX8mbKbgOn2czEAOzIQERERAUSwL3++uuyZ88e6+M///xTnnjiCenatau89NJL8sMPP8iECRPys0my0iQ4wM/ap4GzMRAREVGBBHA7duyQLl26WB/PnTtXWrdurTo2jBo1Sj766COZP39+fjZJehWqpqlsnD6hPTNwREREVCAB3KVLlyQ6Otr6eO3atWo6LV1sbKwcP348P5v0ederUC2zWnBCeyIiIirQAA7BW0JCgvo5PT1d/vjjD2nTpo31dYwDFxgYmJ9NkrUTg6XTAgM4IiIiKtAArmfPnqqt2/r169XE9mFhYXL77bdbX9+1a5fUqlUrP5ukbKbsDFxYoGVkl9SMLB4bIiIiuvlx4N544w3p3bu3dOzYUc3GgFkYgoKCrK/PmDFD7rrrLvF2U6dOVbesrMIPokym7LlQ8Z/ZLCHZPVE5HyoREREVSACHKbPWrVsniYmJKoDz978+7AUsWLBAIiIixNvFxcWp25UrVyQyMrLo3hiD+bITAxERERVkADd48GC3lkMmjvI/kK+a0D47A5fC6bSIiIioIAI4VJlWq1ZNmjdvbu01STdJ74WaI4BjGzgiIiIqgABu+PDh8s0336ieqIMGDZLHHntMoqKi8rMJcgUBnD6hPTsxEBERUUH0QkXD/lOnTsno0aPVrAtVqlSRPn36yIoVK5iRK6Aq1PBgS0x9LS3zxjdJRERExVq+J7MPDg6Wfv36ycqVKyU+Pl5uueUWGTFihFSvXl2SkpIKZy99YCBfFcfZBXCsQiUiIqICCuDsVvbD3J0mlX0riiE3fCEDVyI7gLuaygwcERERFVAAl5aWptrB3XnnnVK3bl01of2UKVPk2LFjamgRuvH4TWXgQliFSkRERAXYiQFVpZjAHm3fMKQIAjmMDUc3zpQdQ2uI5DRNwoMtnRiS2AaOiIiICiKA++STT6Rq1apSs2ZNNZE9bs4sXLgwP5v1bY4ZuGDLXLIM4IiIiKhAArj+/ftfb3RPBUofB64EM3BERERU0AP5UsEyOXRiiAgOUT9yGBEiIiIqlF6oVMBsM3DshUpERES5YABnsCpUvRdqUnomB0cmIiIipxjAGawKVR/IF1PNJnM+VCIiInKCAZwB50L1y47p2A6OiIiInGEAZ7AqVPTytc7GwLHgiIiIyAkGcB5mNyyL2TIdWQQntCciIiIXGMAZiWYJ4PQMHHuiEhERkTMM4IzEbJnA3toTlVWoRERE5AQDOCMGcHoGjgEcEREROcEAzkiy28AxgCMiIiJXGMAZqReq3okhuwr1KmdjICIiIicYwBlpIN/sKtRSYUHq/nJyuqd2i4iIiAyMAZwTU6dOlYYNG0psbGzRno3sAC4yNFDdJ6ZkFO37ExERkVdgAOdEXFycxMfHy9atW4u4CtU+gLuczACOiIiIcmIAZ6SBfDWzumMGjoiIiFxhAGck1jZwrEIlIiKi3DGAMwgNiTi2gSMiIiI3MIAzYi/UUL0XKtvAERERUU4M4Iwkexw4vQ1cSkaWpGVaniMiIiLSMYAz6EC+et8GDiVCREREjhjAGbAK1c/PJCVDLFm4KxwLjoiIiBwwgDOS7ADOticq28ERERGRIwZwBqEhE2cTwHEwXyIiIsoNAzhPs6lB1dvA2c6HeonzoRIREZEDBnBGol0P4MqGWwK480mc0J6IiIjsMYAzEpsq1HLhwer+fFKaB3eIiIiIjIgBnAF7oULZ7ADu3FUGcERERGSPAZyR2AZwEXoVKgM4IiIisscAzsNM2SP2mtV/6n+7DBwDOCIiInLEAM7D/LJPgZZLFSo7MRAREZEjBnBGycCZHDoxRARbhxHJyLqemSMiIiJiAOdhfiY/pwP5lg4LEj+TiKaJXLzGoUSIiIjoOgZwBqlCtbSBuz4OnL+fSaJKsCcqERER5cQAzsP0KlRLG7gMu9eiS1oCuDNXUj2xa0RERGRQDOAMUoWq2sBl2g8ZUrFUqLo/cTnFI/tGRERExsQAzkjDiGTZt3WrxACOiIiInGAAZ5hhREw5MnDWAO4SM3BERER0HQM4w/RCzVmFWqk0q1CJiIgoJwZwRhoHLst5Bu4k28ARERGRDQZwRunE4CIDd/ZqmqRncjBfIiIismAA52EmtH3LpQ1cmRJBEhrorwbzPX4p2UN7SEREREbDAM5IvVAzU3O8VrNcCfXzobNJntg9IiIiMiAGcB4/ATbjwDkMIwK1yoWr+0PnrhX5vhEREZExMYAzcC9UqF3eEsAdZAaOiIiIsjGAM1QVau4B3KFzrEIlIiIiCwZwhsnAmXIMI2JbhYoMnNms8nRERETk4xjAefwE5D4XKqATQ3CAnySlZcqRC2wHR0RERAzgDN0LFQL9/eSWiiXVz7v+Tizq3SMiIiIDYgbOialTp0rDhg0lNja2aDsxmDOdZuGaVC6l7nf+fbnQ94eIiIiMjwGcE3FxcRIfHy9bt24tghNg0wYO0q7mWKZplUh1v+M4AzgiIiJiAGecKlT/QMsTaVdyLNOiapS6//PvRNUWjoiIiHwbM3BGmQtVD+BScwZwVcuESbUyYZJp1uS3QxeKeheJiIjIYBjAGSYDF5RrFSq0r11W3f968HzR7RwREREZEgM4j5+A7DZw1ipU5wHc7XXKqftVe8+IhtntiYiIyGcxgDNaFaqTNnDQsW45CQvyl78vpch2dmYgIiLyaQzgDFKFqrloAwehQf5yZ8No9fP3O04W3Q4SERGR4TCAM0wGLtjyRHLubdzua1ZR3X+344SkZmQVzQ4SERGR4TCA8/gJsJyCrIDsAC7pTK7LdqhTTiqVCpVLyRmyePuJotpFIiIiMhgGcB4W4Beg7jMCsnuhJp3NfVl/Pxl4W3X1839/TeDk9kRERD6KAZyHBWUPH5LuH5BnBg76xFaRiOAA+etskvywi23hiIiIfBEDOIMEcBl+/nlm4CAyNFCGdqipfn7/pwOSnmku/J0kIiIiQ2EA52GBfpbep+nZvVHl6mkRs+sOCoPb15Cy4cFy7GKyzN50pCh2k4iIiAyEAZyHBfllZ+DQGxXt4cwZIldPuVynRHCAvNCtrvp50soDcuJySpHsKxERERkDAzgPC8we/y0dgVupqpYnLybkud4/WlSRVtWjJDk9S8Yu3s3ZGYiIiHwIAzijZOAQwJWuYXnyUt7Von5+JnnrgUYS6G+S1fvOyv9+/7uwd5WIiIgMggGcUXqhZqWLlLYMESKX8s7AQZ3oCPnnnZaq1HHf75Ej568V3o4SERGRYTCAM0onBrsAzv2OCcM61JJWNaLkWnqWPDdvh2RksVcqERFRcccAziBt4FQValR2FerFw26v7+9nkg/6NpOIkADZcfyyvLdif2HtKhERERkEAzgjtYEra6kOlXMHRDTN7W1geq3/PNhE/fzpusOy7E/XvViJiIjIuzGAM1IbuKialqFEMq6JJOavU0LPxhVkyO2WDN4L/9slB88mFcr+EhERkecxgDNIBi7dnC6C6tSoWpYXzue/KvTF7vWlTc0oSUrLlKe++l3dExERUfHDAM4gbeAyzZli1swi5epZXjiX/wAOk91/3O9WiS4ZrDJwz83dIVlm96tiiYiIyDswgPOwEP8Q68+pmak2Ady+G9peuYhgmfZYCwkK8JNVe8/IO8v2FtSuEhERkUEwgPOwkACbAC4LAVz96x0ZbtCtVUvLe/9oqn7+bH2CfLPl2M3vKBERERkGAzhPnwCTn7UdXI4MXD56ojq6t2lF+WdXS6/WVxfvlg0HzxfMDhMREZHHMYAzgGD/4OsBXJnaImISSb0sknTmprb7TJfacn+zipJp1lSnhv2nrxbQHhMREZEnMYAzUDu4lKwUkcBQkTLZPVHP7L6p7ZpMJnnnwSYSW720XE3NlP4zNsvxi8kFsctERETkQQzgDECvQk3LTLM8EdPYcn9q101vOyTQXz7vHyv1oiPkzJU06T9ji5xPyn4fIiIi8koM4IxWhQoxllkV5PSfBbL9yLBA+WJwKzVjQ8L5azJw5ha5mppRINsmIiKioscAzmhVqFBBD+BuPgOni4kMkS+faCVlSgTJ7hNXZOjs3yU1I6vAtk9ERERFhwGcAQT75ZKBu3BIJK3gpsSqWS5cZg1qJSWC/GXT4QsyZPY2BnFEREReiAGcEatQw8uLhMeIiCZyZk+BvlfjypEyc1ArCQvyl/V/nZehXzITR0RE5G0YwBmoClUN5KsrhGpUXasaUTJzYKyEBvrLugPn1BAjaZmsTiUiIvIWDOAMlIFLycxuA2fXE3Vnobxn65plZOYgSxD3y/5zMuzL3yUlnUEcERGRN2AAZwB2MzHoKja33J/4o9Det03NMvLfgS0lJNBPBXEDZmyRK+ydSkREZHgM4AxUhZqWZTM+W+VYy/3ZeJHUK4X23rfVKiuzB7eWiOAA2XLkovSb/hvHiSMiIjI4BnBGrUKNiBGJrGrpyHDi90J9f7SJ+2ZoGykbHiR7Tl6RPp9skhOXbfaFiIiIDIUBnBGHEdFVyc7C/b210PehUaVImT+srRrs9/D5a9L7/zbI7hOJhf6+RERElH8M4IzaCxUqtyqyAE4fJ+5/w9tKnfLhatqtPp9uklXxZ4rkvYmIiMh9DOAMFMAlZyTnnoHTtCLZlwqRofK/4bfJ7XXKSnJ6lgz5cpv899cE0Yro/YmIiChvDOAMICwgTN1fy7hm/0J0Y5GAEJGUSyLnDxTZ/kSGBsqMgbHSr1VVFTe+sSReXlm8W9IzzUW2D0RERJQ7BnBGDuACgq73Rk1YV6T7FOjvJ28/0Ehe7tlATCaROZuPSd/pm+R0okM1LxERERU5BnAGCuCSMpzMe1qzo+U+YW0R75WIyWSSIR1qyn8HtJSSIQGy/dhlufvj9bLp0IUi3xciIiK6jgGcAZQIKJF7AFejk+U+Yb2I2TMzJXSuHy0/PN1eGlQoKeeT0uWx/26WT9ceErOZ7eKIiIg8gQGcgQK4a+nXcnYWwIwMwSVFUi8Xyryo7qpWpoQsHH6b9G5eSbLMmkxYtk/6z9giZ66wSpWIiKioMYAzUBVqppZpPxsD+AeIVGtn+flw0Vej2goN8pf3+zSVCb0bq+m3fj14XrpPXicr9pz26H4RERH5GgZwBhlGxCQmF+3gsqtRD64ST0O7OPROXfL07dKoUkm5lJwhw778XV76dhfnUSUiIioiDOAMwM/kJyUCs9vBpTsJ4Op2s9wf3SiSfFGMoHb5cFk4vJ0M61hT9VKdu/W43DVpnazey4F/iYiIChsDOIMIDwx3PpQIRNUQKd9QRMsS+WulGEVQgJ+M6dFAvhnSRqqXCZPTV1LliS+2yTPfbJcLSQ5VwURERFRgGMAZhDUD56wKFer1tNzv/1GMpk3NMrLs2Q4yrENN8TOJfL/zpHSdtFbmbD6qOjwQERFRwWIAZxDhQeGuA7j6vSz3B1eLZBiv5yc6OIzp2UAWx7WT+jERqm3cy4t2y71TfpVtR4xR7UtERFRcMIAziIjACHV/Je2K8wUwnEhkFRG0kTuwTIyqSeVSsuTp9jLunoZq8N89J6/IQ59skmfnbpe/LznM9UpEREQ3hAGcQZQKKaXuL6Vdcr4Aego06WP5eedcMbIAfz8Z2K6GrHm+k/RrVUXt+nc7Tkrn99bKuO/3yHm2jyMiIropxT6AO378uHTq1EkaNmwoTZo0kQULFogRlQ4ure4vpeYSwEGThy336MiQdFaMrkx4sEzo3US+j2svt9UqI+lZZpm18Yh0mLhGJq08IElpnplZgoiIyNsV+wAuICBAJk+eLPHx8fLTTz/Jc889J9euOenp6WGlQywB3MVUF+3FytUVqdTC0ht113zxFo0rR8rXQ9rIV0+0lqaVIyU5PUumrDkk9/13l0xcsV/OXWWPVSIiovwo9gFchQoVpFmzZurnmJgYKVu2rFy8eNE7M3DQ/DHL/ZbpHpsb9Ua1r1NWdXL45LEWUrd8uFxLN8snaw9L+//8LK8u3i3HL7KNHBERkVcEcOvWrZN77rlHKlasqEb5X7x4cY5lpk6dKtWrV5eQkBBp3bq1bNmy5Ybe6/fff5esrCypUqWKGE1USJR7ARyqUdFe7vJRkf1LxdvgHHdvFCNLn2kvE++pJc2qREpaplm+/O2odHx3jQydvU02HDyfc05YIiIiMk4Ah+rMpk2bqiDNmXnz5smoUaPktddekz/++EMt261bNzl79nobMGTYGjVqlON28uRJ6zLIuvXv31+mT58uRqRXoebaiUEXFCbScpDl501TRbw00PHzM0mHWqXk26faqoGAb69TVjBk3E/xZ+TRzzerceRmbUjg9FxEREROBIiH9ejRQ91yM2nSJBkyZIgMGmQJWj755BP58ccfZcaMGfLSSy+p53bs2OHyPdLS0uT+++9Xy992220ul8NNd+WKZUgPs9msboUB20W2KTI40toGLs/3avmkmDZNFdOxTWI++LNIrTvE2+jlxq11jdLSukasHDhzVb767Zgs2n5CDp27JuN+iJd3lu+TbrfESO/mFeW2WmXFHyMFeym9zIX1WTIqltt3zjfPte+ca1893+YiKLO72/Z4AOdKenq6qvYcM2aM9Tk/Pz/p2rWrbNq0ya1t4EAPHDhQOnfuLI8//rjLZSdMmCDjx4/P8fy5c+ckNTW10E5UYmKiBIcHq8cpmSly+MRh69RazgVIRMOHpcSfsyXrp7Fyoff/LMOMeBG93Dg/OKdQyiQysm05GXRrlCzbd0G+3XlOEi6mqiFIcCtXIlC61Y9St9plQ1V1rLeX2Rew3L5zvnmufedc++r5NhdBma9ever9Adz58+dVm7Xo6Gi75/F43759bm1jw4YNqhoWQ4jo7eu+/PJLady4cY5lESiiutY2A4f2cuXKlZOSJUtKYX0YEIjgPdCRAVWoGaEZUj6qvOsV73pFtH3fSuC53VL+7FqRxtljxHkJ23I7+xKMqFJBhnfVZNffibJw+wn5YecpOXctQ776/Yy6VSsTJt1uiZYet8RIk8qRXhHM5VXm4orl9p3zzXPtO+faV8+3uQjKjPb+Xh/AFYT27du7nY4MDg5WN0c4SYX54cSHAduvGF5RBXCnkk9Jg7INXK8UES1y+z9Ffn5T/Fb8W6T2nSIlyog30cvt6tg2rxalbq/efYv8vO+sLPzjb1l74JwcvZAs09clqFuFyBC5s2G0dKxbTs3LWiI4wKvLXByx3L5zvnmufedc++r5NhVymd3drnGvdCJqyA9/f385c+aM3fN4jCFBihsEcHsu7JGTSdc7X7h027MiuxeKnI0XWfq8yEMzvK4q1V1BAX6q9ypu19Iy5Zf952TZ7lOyZt9ZOZWYKrM3HVW3IH8/aVm9tHSoW0461Cmn5mVFhwkiIqLixNABXFBQkLRo0UJWr16tOiEAsml4PHLkSCluKoVXUvcnkk64t0JAkMi9H4v89y6RPQtFqrYVaT1Uijtk2Ho1qaBuqRlZ8utf52XN/rMqM/f3pRTZeOiCur2zbJ+UCguUltWipFWN0hJbPUoaVYqUQH/f+UuRiIiKJ48HcElJSXLw4EHr44SEBNWrNCoqSqpWrarapA0YMEBatmwprVq1UrMqYOgRvVdqcVKtZDV1f+jyIfdXqtxS5K43RFCNumKMSFQNkTp3iq8ICfSXrg2j1Q2NShPOX1OB3LoD5+S3wxflcnKGrNp7Rt0gNNBfmlUpJU2qREqTSqWkcaVIqRLlfR0iiIjIt3k8gNu2bZvcccf1YTD0TgQI2mbNmiV9+/ZVvUDHjh0rp0+fVmO+LV++PEfHhuKgflR9db/v4j4VjLgdVLQZIXJyh8if80XmPSbS7xuRWp3F1+B41SwXrm6D2tWQjCyz7D6RKFuPXJQtCZfUfWJKhmw6fEHddJGhgSqQQ3YOVa61y4erG4JDIiIiI/J4AIeJ5vMadR/VpcWxytRRndJ1xN/krzoynEk+IzEl3Gznh0Dv/v8TSU+yzM4w5x8iPd+7PuCvj0JVafOqpdVtaAdUv2vy19kk2X7skvx5IlHd9p26qoK6Xw+eVzcdms1VjQqTOtERUjfaEtBVjSqher+WKRHEjB0REfl2AEfXBfsHS93SdWXvxb2y5fQWubfWve4fHv9AkX/MElk8QmT3/0SWPCdy6GeRXu+LhOcxJImPQGeGejER6vZw9nPpmWY1gDCGK9lzMlH+OpMkB85eVVWvRy4kq9vKePtONCWC/KVKVJgK5qqVKaECvYqlQiSmZKjqEYt2d6ySJSKiwsQAzglM64UbxqArau0rtVcB3Nrja/MXwEFAsMiDn4tENxRZ87bI3u8tQVyb4SKth3vdMCNF1bsVVae46ZARPpeUJgcRzJ25KgfOJsnhc0ly7EKynLqSKtfSs2Tf6avq5kxwgJ/ERCKgC1EBXXRkiJQLD5aosEDxz0yRmpnBUjYiRKJKBElwAKtpiYgo/0waZw3PFQbyjYyMVKMuF+ZAvpjXtXz58mrsFwwj8vCShyXAL0BWPLhCyofdYPbs1C6RH54RObnd8tg/SKReT5Fb7hep2Ukk1DL3qqc4lttboNcreroeu3hNjUWH2/GLyXL6SqqcTkyVC9fS87W9iOAAKRMepIK50mFBUjI0UCJCAqRkSKCUDNXvA9W9ej779RJBARIS6OcVmT5vPdc3yxfL7YtlBpbbd863uQg+4+7GHszAGcwtZW6RW8vfKn+c/UPGbRwnH9zxgapazbcKTUSGrBHZt0Rk3Xsip3aIxC+23Ex+IjGNRWKaiFRoKhJVU6RUVZHIyiKBoYVRrGIDHRv0Tg7OpGVmydkraWpsulOJKSqoQ3B3PildLiSlydnEZElMM8ula+mSadbkalqmuqGqNr8Qu4UF+ktoUICEBfmrW2iQvwrucG99LhCP/STI31+CA3HvpzKPyBTq98gE4mf757Of8/eTQH+Tmoc2wM9PAvxxb/KK4JGIqLhiBs5gGTi9F+ojPz4iGeYMqVCignSo3EEN8hsZFKkyc/5+/hJgClAXUPxzy6WjIsc2WjJzVyzjzDldM6iESFC4SFCYSCB+LmHJ3qGNnV+A5R6P/QLE5OdvCQbFzxJNmPR7bNnv+mO8k/Xn7HJrmqSkJEtoWJj4Ydm8mNx80u2gwtlyhRuQmEWT5ORkCQsLs7a/S8nIktQM3GdKWoZZPZeWhfssScvQLPdZZsnA85mW19OzjDFxNA61n8lPdfjAz/74PKqbpROIGqk8u0MIEv0B/ljW8rq+nPoEq/vsW/YDy8/qXSzbl+vLiuN62e9v+S7o27fsoz6Gs3ote3mnZbHZXvYTDq9fL7PjevbP2y6gSUZGhgQFZreJtFtX31v7N7F9nF38m+R6A3lt3q23tyuyJmnp6RIcFGRzUPLYh5ssY2H/GZHn9tXnG3N3p0lQULCTz0jxhXKnpadJsJNyF/cyd2zWW5rWu61Q3oMZOC8fTmRql6ny71//LaeunZJ5++cV3MaRYAstl8dCCBCSLLf81QhSQQrMvlH+oFO7647tRavom9J6nq/+3kgV3+SD5fb/UwotgHMXq1ANqm3FtrKs9zJZf2K97D6/W86nnJfEtETJ1DIly5wlWVqWui84mog5UyQrw3KPbav7TBHNbPmzQ91bbhrusY7mcMW0HRLG+rPzKyqG9UBGxumfqAVyATbSVfx6xtVb24pYT3X2oxxHV/84OFkPnxc9W+XuWbFbLpeVrj/tYqta0X8ybA6TeykY431Ubxg+GW7XDORY15u5e7KLG18styalDTCdJwM4AwsJCJE7q92pbsWNLzb69cUyA8vtO+eb59p3zrWvnm+zTZk9zTeOOBEREVExwgCOiIiIyMswgCMiIiLyMgzgiIiIiLwMAzgnMI1Ww4YNJTY2tujPCBEREVEeGMA5ERcXJ/Hx8bJ169a8jh8RERFRkWMAR0RERORlGMAREREReRkGcERERERehgEcERERkZdhAEdERETkZTgXqgta9mTsV65cKdR51a5evSohISE+M5ecr5bbF8sMLLfvnG+ea9851756vs1FUGY95tBjkNwwgHMBJwmqVKlSkOeGiIiIKM8YJDIyMtfXTVpeIZ4PQ6R98uRJiYiIEJPJVGiRNgLE48ePS8mSJcVX+GK5fbHMwHL7zvnmufadc+2r5/tKEZQZYRmCt4oVK7rM8jED5wIOXOXKlaUo4IPgK18AXy+3L5YZWG7fwXPtW3zxfJcs5DK7yrzpfKPSmoiIiKgYYQBHRERE5GUYwHlYcHCwvPbaa+rel/hiuX2xzMBy+8755rn2nXPtq+c72EBlZicGIiIiIi/DDBwRERGRl2EAR0RERORlGMAREREReRkGcERERERehgGch02dOlWqV6+u5lVr3bq1bNmyRbzBhAkTJDY2Vs1SUb58ebn//vtl//79dst06tRJzWBhe3vqqafsljl27Jj06tVLwsLC1HZeeOEFyczMtFvml19+kVtvvVX1+qldu7bMmjVLPGXcuHE5ylS/fn3r66mpqRIXFydlypSR8PBwefDBB+XMmTNeXWbAZ9Sx3LihrMXlXK9bt07uueceNfo59n/x4sU5RkcfO3asVKhQQUJDQ6Vr167y119/2S1z8eJFefTRR9UAn6VKlZInnnhCkpKS7JbZtWuX3H777eo7jxHdJ06cmGNfFixYoD5XWKZx48aydOlSj5Q7IyNDXnzxRbUPJUqUUMv0799fzVCT1+fjnXfe8dpyw8CBA3OUqXv37l59vvMqs7PvOG7vvvuu157rCW5cq4ry93aBXvMxlRZ5xty5c7WgoCBtxowZ2p49e7QhQ4ZopUqV0s6cOWP4U9KtWzdt5syZ2u7du7UdO3ZoPXv21KpWraolJSVZl+nYsaMq06lTp6y3xMRE6+uZmZlao0aNtK5du2rbt2/Xli5dqpUtW1YbM2aMdZnDhw9rYWFh2qhRo7T4+Hjt448/1vz9/bXly5drnvDaa69pt9xyi12Zzp07Z339qaee0qpUqaKtXr1a27Ztm9amTRvttttu8+oyw9mzZ+3KvHLlSkzBp61Zs6bYnGvs08svv6wtXLhQlW3RokV2r7/zzjtaZGSktnjxYm3nzp3avffeq9WoUUNLSUmxLtO9e3etadOm2m+//aatX79eq127ttavXz/r6zgm0dHR2qOPPqq+O998840WGhqqffrpp9ZlNmzYoMo9ceJEdRxeeeUVLTAwUPvzzz+LvNyXL19W52zevHnavn37tE2bNmmtWrXSWrRoYbeNatWqaa+//rrd+bf9XeBt5YYBAwao82lbposXL9ot423nO68y25YVN1ybTCaTdujQIa89193cuFYV1e/tgr7mM4DzIPwijIuLsz7OysrSKlasqE2YMEHzNrjA4xfC2rVrrc/hov7ss8/mug6+BH5+ftrp06etz02bNk0rWbKklpaWph6PHj1aBUy2+vbtq76Ungrg8AvbGVzs8EtowYIF1uf27t2rjgsufN5aZmdwXmvVqqWZzeZiea4dL24oZ0xMjPbuu+/ane/g4GB1gQL80sZ6W7dutS6zbNkydQE8ceKEevx///d/WunSpa1lhhdffFGrV6+e9XGfPn20Xr162e1P69attWHDhmmFzdlF3dGWLVvUckePHrW7qH/wwQe5ruON5UYAd9999+W6jrefb3fONcrfuXNnu+e8/VyfdbhWFeXv7YK+5rMK1UPS09Pl999/V9UwtnOv4vGmTZvE2yQmJqr7qKgou+fnzJkjZcuWlUaNGsmYMWMkOTnZ+hrKidR5dHS09blu3bqpyYL37NljXcb2GOnLePIYodoMVRA1a9ZU1SdIrQPOJ6qcbPcXVQRVq1a17q+3ltnxs/vVV1/J4MGDVfVJcT7XuoSEBDl9+rTd/mGuQlSB2J5bVKO1bNnSugyWx/d68+bN1mU6dOggQUFBdmVElc6lS5cMfxz07zrOO8pqC9VoqIJq3ry5qnKzrV7y1nKjSgzVZfXq1ZPhw4fLhQsXrK8V9/ONKsQff/xRVQs78uZznehwrSqq39uFcc3nZPYecv78ecnKyrL7QAAe79u3T7yJ2WyW5557Ttq1a6cu3rpHHnlEqlWrpoIdtIlAWxp8iRcuXKhexwXRWfn111wtgy9OSkqKaotUlHDBRrsG/EI/deqUjB8/XrX12L17t9pX/NJyvLBhf/Mqj/6aEcvsCO1mLl++rNoIFedzbUvfR2f7Z7v/uNjbCggIUBcK22Vq1KiRYxv6a6VLl871OOjb8CS0FcK57devn91E3s8884xq+4Oybty4UQXw+H5MmjTJa8uN9m69e/dW+33o0CH597//LT169FAXW39//2J/vr/44gvVbgzHwJY3n2uzk2tVUf3eRvBa0Nd8BnB009D4EwHMr7/+avf80KFDrT/jrxc0/u7SpYv6ZVirVi2vPPL4Ba5r0qSJCugQuMyfP9/jgVVR+e9//6uOA4K14nyuyR6yFH369FGdOaZNm2b32qhRo+y+F7ggDhs2TDUgN8KUQzfi4YcftvtMo1z4LCMrh892cTdjxgxVw4DG9sXlXMflcq3yVqxC9RBUNeGvOMeeLngcExMj3mLkyJGyZMkSWbNmjVSuXNnlsgh24ODBg+oe5XRWfv01V8vgr38jBEz4q61u3bqqTNhXpMmRncrtnHp7mY8ePSqrVq2SJ5980qfOtb6Prr6vuD979qzd66haQk/Fgjj/nvy9oAdvOP8rV660y77ldv5R9iNHjnh1uW2hyQR+b9t+povr+V6/fr3KoOf1Pfemcz0yl2tVUf3eLoxrPgM4D8FfLS1atJDVq1fbpXfxuG3btmJ0+CscX4hFixbJzz//nCNl7syOHTvUPbIzgHL++eefdr8E9YtDw4YNrcvYHiN9GaMcIwwZgCwTyoTzGRgYaLe/+CWINnL6/np7mWfOnKmqjdCd3pfONT7f+CVru3+oGkFbJ9tzi4sA2rno8N3A91oPaLEMhnJAQGRbRlTJo2rJiMdBD97Q9hPBO9o+5QXnH+179CpGbyy3o7///lu1gbP9TBfH861n2fH7rGnTpl5/rrU8rlVF9Xu7UK75N9T1gQoEuhSjF9usWbNUj6ahQ4eqLsW2PV2Mavjw4WpIhV9++cWuO3lycrJ6/eDBg6qrObpkJyQkaN99951Ws2ZNrUOHDjm6Zt91112qeze6W5crV85p1+wXXnhB9QyaOnWqR4fU+Ne//qXKjDKhKzy6laM7OXo26d3R0UX9559/VmVv27atunlzmW17TKFs6FFmq7ic66tXr6ohAnDDr8ZJkyapn/XelhhGBN9PlG/Xrl2qh56zYUSaN2+ubd68Wfv111+1OnXq2A0rgR5vGGLh8ccfV8Ma4HcAyuw4xEJAQID23nvvqeOAns+FOZyGq3Knp6er4VIqV66szpvtd13vfbdx40bVKxGvY7iJr776Sp3b/v37e2258drzzz+veiHiM71q1Srt1ltvVeczNTXVa893Xp9xfRgQ7CN6WTryxnM9PI9rVVH+3i7oaz4DOA/DWDH44GBsGHQxxnhC3gBffmc3jLcDx44dUxfwqKgo9YHF+Ej4YNuODQZHjhzRevToocYJQiCEACkjI8NuGYw11qxZM3WMEBjo7+EJ6BZeoUIFtS+VKlVSjxHA6HAxHzFihOpGjy/zAw88oH5ZeHOZdStWrFDneP/+/XbPF5dzjfd29pnGcBL6UCKvvvqqujihnF26dMlxLC5cuKAu4OHh4WqIgUGDBqmLpi2MIde+fXu1DXyGEBg6mj9/vla3bl11HDA0wY8//uiRciN4ye27ro8B+Pvvv6shIHCRDAkJ0Ro0aKC9/fbbdoGOt5UbF3dcrHGRRmCBoTMwZpfjhdbbznden3FAoIXvKAIxR954riWPa1VR/94uyGu+KbuAREREROQl2AaOiIiIyMswgCMiIiLyMgzgiIiIiLwMAzgiIiIiL8MAjoiIiMjLMIAjIiIi8jIM4IiIiIi8DAM4IiIiIi/DAI6IfEanTp3kueeeEyMxmUyyePFiT+8GEXkZzsRARD7j4sWLauLqiIgIqV69ugrmiiqgGzdunArUMPm3rdOnT6tJvoODg4tkP4ioeAjw9A4QERWVqKioAt9menq6BAUF3fD6MTExBbo/ROQbWIVKRD5XhYr7o0ePyj//+U9VhYmb7tdff5Xbb79dQkNDpUqVKvLMM8/ItWvXrK8jc/fGG29I//79pWTJkjJ06FD1/Isvvih169aVsLAwqVmzprz66quSkZGhXps1a5aMHz9edu7caX0/POesCvXPP/+Uzp07q/cvU6aM2n5SUpL19YEDB8r9998v7733nlSoUEEtExcXZ30vIvINDOCIyOcsXLhQKleuLK+//rqcOnVK3eDQoUPSvXt3efDBB2XXrl0yb948FdCNHDnSbn0ET02bNpXt27erQA1QLYugLD4+Xj788EP57LPP5IMPPlCv9e3bV/71r3/JLbfcYn0/POcIgWK3bt1UlerWrVtlwYIFsmrVqhzvv2bNGrWvuP/iiy/U++oBIRH5BlahEpFPVqX6+/uroMu2CnPChAny6KOPWtvF1alTRz766CPp2LGjTJs2TUJCQtTzyJAhILP1yiuv2GXpnn/+eZk7d66MHj1aZdPCw8MlICDAZZXp119/LampqTJ79mwpUaKEem7KlClyzz33yH/+8x+Jjo5WzyHAw/MoQ/369aVXr16yevVqGTJkSAEfKSIyKgZwRETZUMWJzNucOXOsx0TTNDGbzZKQkCANGjRQz7Vs2TLHMUO2DsEeMmOo8szMzFRVrPmxd+9eldnTgzdo166dev/9+/dbAzhk8hC86VCViqpXIvIdDOCIiLIh8Bo2bJhq9+aoatWq1p9tAyzYtGmTytyhnRuqQCMjI1X27f333y+UY4uetLbQjg5BHhH5DgZwROST0HM0KyvL7rlbb71VtWGrXbt2vra1ceNGqVatmrz88svW59BJIq/3c4QMH9qyoS2cHiRu2LBB/Pz8pF69evnaJyIq3tiJgYh8EtqprVu3Tk6cOCHnz5+39iRFMIZOAxiv7a+//pLvvvsuRycCR2grd+zYMZV1QxUqqlIXLVqU4/1QDYvt4v3S0tJybAdZPLSzGzBggOzevVt1Unj66afl8ccft1afEhEBAzgi8knogXrkyBGpVauWlCtXTj3XpEkTWbt2rRw4cEANJdK8eXMZO3asVKxY0eW27r33XjUkCQK9Zs2aqSBQ752qQ89W9HC944471Pt98803ObaDIUhWrFihBhyOjY2Vhx56SLp06aI6LBAR2eJMDERERERehhk4IiIiIi/DAI6IiIjIyzCAIyIiIvIyDOCIiIiIvAwDOCIiIiIvwwCOiIiIyMswgCMiIiLyMgzgiIiIiLwMAzgiIiIiL8MAjoiIiMjLMIAjIiIiEu/y/zIwXSbY+uFoAAAAAElFTkSuQmCC", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "p_start = np.array([3.0, 0.5, 3.9, 0.0]) # [A, b, omega, phi]\n", "\n", "plt.figure(figsize=(7, 4))\n", "for gamma in [0.002, 0.01, 0.05]:\n", " # YOUR CODE HERE (1 line): run gradient descent from p_start with this gamma for\n", " # 20000 iterations, storing the results in p_fit and hist, like this:\n", " # p_fit, hist = gradient_descent(, , , ,\n", " # , , )\n", " p_fit, hist = gradient_descent(model_loss, model_grad, xs, ys, p_start, gamma, 20000)\n", "\n", " print(f\"gamma = {gamma:<7} -> p = {np.round(p_fit, 4)}, final MSE = {hist[-1]:.4f}\")\n", " plt.semilogy(hist, label=f\"gamma = {gamma}\")\n", "\n", "plt.xlabel(\"iteration\"); plt.ylabel(\"MSE\"); plt.legend(); plt.grid(alpha=.3)\n", "plt.title(\"3(c) three converging learning rates\"); plt.show()" ] }, { "cell_type": "markdown", "id": "d4519753", "metadata": {}, "source": [ "### 3(d) Plot the fit on top of the data\n", "\n", "One line to write: evaluate your fitted model on the smooth grid `x_grid` so it can be\n", "plotted as a curve. The numbers printed above the plot let you compare your four estimates\n", "against the values the data was actually generated from." ] }, { "cell_type": "code", "execution_count": 19, "id": "e11ffdca", "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "your estimate: [3.001 0.5843 4.0032 0.4779]\n", "true values : [3. 0.6 4. 0.5]\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAmQAAAGJCAYAAAAkIy99AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjAsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvlHJYcgAAAAlwSFlzAAAPYQAAD2EBqD+naQAAZONJREFUeJzt3Qd4FFUXBuBvUwi9h947CALSBES6dCkqVaSqICjlBxULiCBYUEBFUARREVFQQAHpHaRIExQRMBSl95oQkvmfcydbEhJIwu7Ozsz3Ps+S2dnN5s7Mhj2599xzHZqmaSAiIiIiwwQZ96OJiIiISDAgIyIiIjIYAzIiIiIigzEgIyIiIjIYAzIiIiIigzEgIyIiIjIYAzIiIiIigzEgIyIiIjIYAzIiIiIigzEgIwowsbGxKF++PN566627PveNN96Aw+Fw3Y+OjkbBggXxySefJPvnnTp1Co8//jhy5MihXmvChAlYs2aN2pavRkp4fClVr149dbOyw4cPq3M0Y8YMw8+3EaS90m4rHhvZCwMyIh/7448/8MQTT6BYsWJInz49cubMiYcffhg///xzos//9ttvcezYMfTv3z/FPys0NBSDBw9WwVxkZGSyvmfQoEFYunQphg0bhq+//hpNmzZN9HmzZs1SwVpCx48fVx92u3btSnF7iYhIx4CMyMeOHDmCK1euoFu3bpg4cSJef/11tf/RRx/FZ599dtvz33vvPXTs2BFZsmRJ1c/r0aMHzp49qwKo5Fi1ahVat26NIUOG4Mknn0SZMmVUwHjjxg31NTkB2ciRIxmQWcBrr72mrruZSHul3URmF2J0A4isrnnz5urmSXq/qlSpgg8++ADPPPOMa//OnTuxe/duvP/++6n+eVmzZsUjjzyihrB69ux51+efPn1afY+noKAgpE2bNtVtIHMKCQlRNzPh+5Ssgj1kRAYIDg5WuV4XL16Mt3/+/PlIkyZNvJ4ppw0bNqBatWrqA6h48eL49NNPk3z9xo0bq+efP38+yedIwCY5NZqmYdKkSWrbmWOTMIdM8rAWLVqkevuczytSpIh6XNrk7JlzPuaZz7RlyxY1DCo9fjJkW7duXWzcuPGeji8x0tso35cuXTpUr14d69evv+05N2/exPDhw1UwLO3JkCED6tSpg9WrVyealzVu3Dh1bpzDzRLoynCynLNRo0ahQIEC6udJD2PCcy3np2XLlli2bBkqVaqkjqtcuXL48ccfb2uXvA8GDhyo3hNhYWEoUaIE3nnnHZVPmPB53bt3V22XIFp6XRO+h5Ii+YXSk1myZEnVFskZfOihh7B8+fI75lnJffkDQt6bktso7bvvvvuwZMmSeM+TdskxJ5TYa8rPlJ8tx5AxY0aULl0ar7zyym1/KPTq1Qu5c+dW7a1YsSK+/PLLZOWQ3et7icgI5vpTiMjErl27poZXLl26hJ9++gm//PILOnToEO85mzZtUh96kgvmac+ePSoYCA8PVx8+t27dwogRI9SHVWIk4JCgQV5PgoLESNAnOWNdu3ZVAdxTTz2VZNtfffVV1e5///0X48ePV/vkg7Rs2bJ48803VZAjPX0S3IhatWq5hkObNWum2iPtlZ63L774Ag0aNFABkwROqTm+hKZNm4Znn31W/VwJbP755x81JJw9e3YV5DhdvnwZn3/+OTp16oSnn35aDSXL9zZp0gRbt25VgZOnb775RgVxzz//vAq43n33XbRv3161X4LRl156CQcPHsRHH32khnynT58e7/sPHDigrnGfPn1U8CTHLvmEEszIORfXr19XQep///2njqFQoULquklO34kTJ1zDxHI9JfCTYENeT879vHnz1Osmh5zXsWPHonfv3uq8y7n47bffsGPHDldbkiI/UwLJ5557DpkyZcKHH36Ixx57DEePHlWBXUpzKuU9ef/996v3jgR4cg49g3T5PZE/AmS/BINFixbFnDlzVNAnAeiAAQOSfP17fS8RGUYjIr949tlnNfmVk1tQUJD2+OOPa+fPn4/3nAIFCmiPPfbYbd/bpk0bLW3atNqRI0dc+/78808tODhYvV5Cx48fV/vfeeedu7ZLntevX794+1avXq32y1enFi1aaIULF77t+7dt26ae+8UXX8TbHxsbq5UsWVJr0qSJ2na6fv26VrRoUa1x48apPj5PN2/e1HLlyqVVqlRJi4qKcu3/7LPP1PfWrVvXte/WrVvxniMuXLig5c6dW+vZs6drX0REhPre8PBw7eLFi679w4YNU/srVqyoRUdHu/Z36tRJS5MmjRYZGenaJ+dKnvvDDz+49l26dEnLmzevVrlyZde+UaNGaRkyZND+/vvveO16+eWX1fEfPXpU3Z8/f756vXfffTfe8dSpUyfR85+QtFmu4Z2MGDHitvMt9+XYDh486Nq3e/dutf+jjz5y7evWrVui74+Erzl+/Hh1/8yZM0m2Y8KECeo5M2fOjHeda9asqWXMmFG7fPlyvPbJz/DGe4nISByyJPIT6bmRoRoZdpFeo5iYGNX74uncuXPIli1bvH3yPJkF2aZNG9V74iQ9JNKzkxjna0hyv1Fk1qX0EHXu3Fkdl7RFbtJT2LBhQ6xbt04NyaXm+DxJL48Mb0mvkQz3OjmH9hIOFTufIz9ber2kB6Vq1aqqpygh6c3yfI0aNWqorzL5wTPXSvbLtZReLk/58uVD27ZtXfczZ86seiIlV/DkyZNqn/T8SM+iXDPnOZJbo0aN1LmR8yQWL16sfmbfvn3jHY/03iWHDA9K75Rck5SStsjQn5P0bsmxSE9kSjnzFRcsWHDbkKyTHGuePHlUT6aT9Bq/8MILuHr1KtauXZvo993re4nISAzIiPxEZi/KB5t8IC9cuFB9sLRq1UoNRXlKeP/MmTNqCEdyfxKS3JvEOF/DyLpLzg9+GVKT4SPPmwwbRkVFqWHQ1ByfJ8lrEwm/Xz7AJfcrIQmIJaBw5lFJeyQ/TtqSkOeHunAGZ57DoJ77L1y4EG+/5IIlvAalSpVy5ak5z5MMYSY8R/JeERJsOo8zb968aqg4pedIyPCgDPfJz69QoQKGDh2K33//PVnfm/A8CAkgEx5vcsgQbu3atdXQqQwjyozi77//Pl5wJscq11OGuD1JYOV8PDH3+l4iMhJzyIgMIsVYJWfo77//dn1YSICQmg+5hJyvITXPjOL8gJUyHglzs5wkuJDAzF9mzpypes6kB0UCkly5cqleJsmtOnTo0G3Pl8cSk9T+hMF0cs+T5HC9+OKLiT7uDODuleQMyjFKz5RMNJCgWPIBp0yZooKjO0nO8SYV/EuvlSeZBCG9fjKRQgJhCUa/++47lZcn7UrqZxFZHQMyIoM46z159sxIL1pERES850lviXyIJTbUtH///kRf2/kazh4Fb0jqAzep/c4hLhnacvb2JCY1x+epcOHC6qt8v3yoe84qlPMgs/Oc5s6dq3rNJEHds92S9O0LkpQuQYvnz5IAXDhnJMp5kt7SO50j53GuXLlSPdezlyw558hJJjnIbFi5yetIkCaJ73cLyJJDeswSm/GZWG+W9HzJsLXcpPTLmDFj1MQRCdLkPMixSu+dBKuevWR//fVXvGvu7fcSkZE4ZEnkY84hJ08SLHz11Vfqw0NKITjVrFkTe/fujddrJD0Gkv8iZQdkVpvTvn37VL5MYrZv366CAHk9b5ESEYkN68l+kfDDWGZWSrAhpSPkwz+x4aXUHp8nyf+SD2Lp6fHMyZPSGwnb5Ox98ezZkbIcv/76K3xBiubKTEgnmdko1116DCVHSsisTfn5iR2rtF9y3ITUspPtyZMnx+t9khmeySF5fJ4kqJMhVW/1UMq1lveH5zCozBL1PH6RWCkWZw+qsy1yrJJjJz1nTnLscqzSbpmVmph7fS8RGYk9ZEQ+JsOS8kEsvRH58+dXHzRSTkH+2pcCsJ69HVLWQOpbSdKyTN13kvpRMrQjyd9SesD54ST1oBLLA5LJA5Knk9KSBHciAZZ8QMrSTFLjSdotOXDyQSyJ2hIQSUkECdAkyV1KFciwmExgkHZKr4wcvyS+S0+I9Jw5l49K6fElzBUbPXq0Os/SQyY5StIzJiUmEuaQSbkF6R2TRPsWLVqo50m7JShOLGi8VzLcKLW0tm3bpvKlpCyGrB0qbXOSoVMpgyJtk+FUOc8y8UHKN0iPnuSaydCznGu5pi+//LLa56xplliQnBh5vpSSkNeXnjKZDCGvn5oluhIjuWBSBkTOrSTfSzkPCR7lHHhOmJBcNhmylPMvPV3yB4usvSo13aQ2mZASKlI7TM6H/HEhvYnSVimNIWVA5H2WlHt5LxEZytA5nkQ28O2332qNGjVSpRVCQkK0bNmyqfsLFixI9Pn333+/1qtXr9v2r127VqtSpYoqQVCsWDFtypQpiZYpkDIN8pzPP/88We1LbtmLq1evap07d9ayZs2qHvMscSDHUq5cOXV8CUsw7Ny5U2vXrp2WI0cOLSwsTH1f+/bttZUrV6bq+JLyySefqHIa8jOqVq2qrVu3TpW88Cx7IeU3xowZo9ogz5PyEwsXLrytZIOz7MV7772X6HmZM2dOvP1yvLJfSoA4yetJmYmlS5eqayo/r0yZMrd9r7hy5YoqqVGiRAl1/Dlz5tRq1aqljRs3TpV7cDp37pzWtWtXLXPmzFqWLFnUtpzf5JS9GD16tFa9enV1/dKlS6fa8tZbb8V7/aTKXiR8fziPT86bp2XLlmnly5dXx1C6dGlVtiLha8p1b926tZYvXz71PPkqZUMSlv04deqU1qNHD3Uu5HkVKlRI9BgTlr3wxnuJyAgO+cfYkJCIPEmx1n79+qkhl4RLGiWH9CBIAVNJ4JYhUTKG9OpIkV+ZUUtEdDfMISMKMF26dFFlBmTJnpSS3DRJkpbFlhmMERGZB3PIiAKMzCqTxP7UkHwqz2RmIiIyB/aQERERERmMOWREREREBmMPGREREZHBGJARERERGcxWSf2yDIdUzpaigkYuukxERETWp2karly5gnz58sVbBgx2D8gkGCtYsKDRzSAiIiIbOXbsmFqN4k5sFZA5l9uQEyPLtviiB07W55N19e4WCVsJj9s+15vX2j7X2q7X247HLHjc4T653rJsnnQE3Wm5L1sGZM5hSgnGfBWQRUZGqte22y8yj9se15vXmr/bVsf3ON/jvpCcNCl7fIoQERERBTAGZEREREQGY0BGREREZDBb5ZARERHZvQzDrVu3EBMTc1vuXHR0tMoHtlsOdPQ9HresIRwcHHzPbWFARkREZAM3b97EiRMncP369UQDNQlOpGaWnep0al44bvk+KWmRMWPGe2oLAzIiIiKLk6AjIiJC9eRIkdI0adLEC0CcPWchISG2C8hu3cNxy/dLmZR///0XJUuWvKeeMgZkRERENugdk6BMamKlT5/+tscZkIWkOhCVmnWHDx9WQ5/3EpDZZ6DYhiKjY3Ds/HX1lYiIyE75Yf7irR5F9pBZUEyshvHL/8a0DRG4ER2DdKHB6PVQUQxqXArBQfbpiiYiIjILBmQWJMHYx6sPuu5LUOa8P6RJaQNbRkRERIlh36XFyPCk9IwlZvrGCA5fEhGR6dWrVw8DBw6ElTAgs5gzV6JUj1hirt+MUY8TERHZxZo1a1Se18WLFxHIGJBZTHimMJUzlpj0aYLV40RERBRYGJBZbCZl2rgE/sT0rF1UPU5EROTNzx5funbtGp566ilVeDVv3rx4//334z3+9ddfo2rVqsiUKRPy5MmDzp074/Tp0+oxKUdRv359tZ0tWzbVU9a9e3d1f8mSJahTp44qW5EzZ060bNkShw4dglGY1G/BmZRyc+aMyTCl9IxJMObcT0REZJZZ/EOHDsXatWuxYMEC5MqVC6+88gp27NiBSpUqqcel/teoUaNQunRpFYgNHjxYBV2LFy9Wddd++OEHPPbYY9i/fz8yZ86MdOnSuQK9QYMGoVy5cmrppBEjRqBt27bYtWuXIeVBGJBZdCal3Po3KKFyxmSYkj1jRETk688eb7t69SqmTZuGmTNnomHDhmrfl19+qZYqcurZs6dru1ixYvjwww9RrVo19b3Sq5Y9e3b1mARzWbNmdT1XgjTPgrjTp09XvWV//vknypcvD3/jkKWFZ1JKEFYwe3oGY0RE5LfPHm86dOiQWmWgRo0arn0SYElvmNP27dvRqlUrFCpUSA1b1q1bV+0/evToHV/7wIEDanhTXitLliwoUqRIsr7PVxiQmRBnUhIRET97oIYdmzRpooYiv/nmG2zbtg3z5s1Tp0YCuTuRIO78+fOYPHkyNm/ejC1btiTr+3yFAZkJcSYlERHZ4bOnePHiCA0NdQVL4sKFC/j777/V9l9//YVz587h7bffVgn6ZcqUcSX0O8lC6iImxt2DJ98jOWWvvvoqGjRogLJly6rXNRIDMhPynEnp0GKR/uYNhMZEq/ucSUlERL7+7EnIV589GTNmRK9evVRi/6pVq7B3716VsO9MupdhSgm4PvroI/zzzz/46aefVIK/p8KFC6vZlQsXLsSZM2dUbpnMuMyRIwemTp2KgwcPqteWyQBGYlK/GcXEYPCFXXh8+YfItWc70kdHITooGOeLlkJ4tm5A9Wdkfq/RrSQiIosxYhb/e++9p4IoGWKUHLH//e9/uHTpknpMkvBnzJihZl5KMv8DDzyAcePG4dFHH3V9f/78+TFy5Ei8/PLL6NGjhyqhId8ze/ZsvPDCC6hcubLKI5PvlxUAjOLQZIqBTVy+fFkl7smFlPFmb4uNjVVdpTKTw2dTZvftA7p2lSzGpJ+TMycwYQLQpQv8wS/HHYDseNx2PGbB47bP9bbqtZayDhEREShatCjSpk172+Oesw2lN+murxcdY4lZ/FoKjzul5zYlcYd13m12sHQpULVq/GBMpv42aAB4TtE9exZ48klgwADVm0ZERORNnMXvfQzIzGLJEpkSAly/rt8vWxZYvlzm5wIrVwJ79sgcXqB9e/f3fPgh0Lu3/MlnWLOJiIjo7hiQmcHu3cATT0g5Yv1+u3Z6L1mjRoBnF2uJEsB33wGffw6ExKUHzpgBvPiiMe0mIiKiZGFAFuiuXAEef1zKFbuDMQm64pZ+SFSvXvpzguPG9WXdr6+/9k97iYiIKMUYkAW6F14ADsYtUyH5YzNnunq/7ri4qwRuH33kvt+njz6kSURERAGHZS8C2YoV+pCjyJQJmD1b9Ywle3FXCcJ++w2YPl3PPZPZmRs3unvOiIiIKCCwhyxQRUUBffu678uwY/Hi8RZ3lWDMc3FX2R+P5JdNnKjnlgmpdPzJJ/47BiIiIkoWBmSBSgIn51Bl7dp6XlhqFnfNmNHdyyZefRU4ccJ37SYiIqIUY0AWiC5eBEaPdvdySS5YXIHCVC0sLgFdz57uSQJvvum7thMREVGKMSALRFJl//x5fVuq7VeufO+Lu779tt5bJpWJP/8cUX8lGN4kIiIK0Gr6zzzzDLJnz66q6WfNmhUDBw7028+XZZbkZ/oaA7JAc/mynvclJPk+QW9Wahd3jcmRE5taP6W2HbduYekTfTFu6X41QYCIiChQLVmyRAVFsjj4iRMn8Pfff8dbQLxIkSKYIB0ZBgRR3sRZloFm8mR9yFLIrMiiRb2yuKsk/H8ZXhfr0n6NbJFX0HLvajT7foV6bEiT0r46GiIionty6NAh5M2bF7Vq1bL0mWQPWSCRdScnTXLnjg0blujTpLSFBFE7Xm+M9S/WV1/lfrySFx6cEwGuhGXAJw8+ofYFQcPg9TMTnwhAREQUALp3747nn38eR48eVcOV0htWr14915ClbB85cgSDBg1Sj8ttzZo16NGjh1rQ27nvjTfeUM+PiorCkCFDkD9/fmTIkAEPPvgg1q5de1vvWqFChZA+fXq0bdsW586d88uxsocskCxaBBw7pm83bw6USrrHy3Nx17vxnAjw1QMt0Ou3+chz9TwaH9iCXCePqseT8zpERGQxUnD85En/BgR58ug1MpNh4sSJKF68OD777DNs27YNwcHBeEKWEozz448/omLFiirH7Omnn1b7JNdMhjCHDx+O/fv3q30Z43Ko+/fvjz///BOzZ89Gvnz51Pe3bNkSv//+O0qVKoUtW7agV69eGDt2LNq0aaOGS0eMGAF/YEAWSDxrhPXr57WXdU4EkKAsKjQMX1R9FMPWzFC9ZE/vXIjwTHpJDSIishkJxv77D4mPrxgvS5YsyJQpkwrE8kggl4AEX/KYPMfzcfk+6Rnz3Ce9bF988YX6KsGYkN4yCbpkvwRhEgA2bdoUL8atAS1B2qZNm9RzfI0BWaCQZY2WLtW3JW+sSROvvbRzIoAUjxXfVmyKARu/RfroKLTfswKhVy7Ju9prP4+IiEwiLmDxnN7l8+AskcDKH/bs2YOYmBgVZHmSYcycOXOq7X379qlhSk81a9ZkQGYrU6a4t6VCf1zdMW/xnAhwGRkxr9Ij6LLtZ4RG3QCmTgVeesmrP4+IiEzAOXSoabh16xZCZK1kyWG2oKtXr6retO3bt6uvzpIactyBMCOTPWSBIDIS+OILfTssDOjRw+s/wjkRoH+DEipnLFe3EsB9C9UvoSo8O3gwEBrq9Z9LRETkS2nSpFE9X3fbV7lyZbXv9OnTqFOnTryATAWiAMqWLavyyDxt3rzZLxeQsywDJZn/wgV9u317IK7r1BecEwHCypYGHn1U3/nff8C8eT77mURERL5SpEgRrFu3Dv/99x/Onj3r2ic9YitXrlT7rl+/roYqu3Tpgqeeekol80dERGDr1q145513sEg+hwG88MILanhy3LhxOHDgAD7++GO/DFcKBmSBYNYs9/ZTevFWv3j+eff2tGn++7lERERe8uabb+Lw4cNqNmZ4eLjaJzXL+vTpgw4dOqh97777rtovyfsSkP3vf/9D6dKlVb7Yb7/9pspcCCmDMXXqVJXcL7M3ly1bhtdee80v18qhSX+dTVy+fFnNvJDaJJkzZ/b668fGxqqu0Fy5ciEouTlgly4BuXNLVqH+VXqr4sa2fS42FihRAoiI0HMG5Gvhwv45bguw43Hb8ZgFj9s+19uq1zoyMlL1CBUtWhRp06a97XHPoTuZnWgXmheO+07nNiVxh3XebWYlQ4USjIkOHfwXjAn5z8aZryZx+Zdf+u9nExERkQsDskAaruzUyf8/v3t394wamVggvWZERETkVwzIjHTqFLBypbv2WI0a/m9DwYLummeHDwOrVvm/DURERDbHgMxI33/v7pGS3jGjxu17eVTqZ3I/ERGR3zEgM5JnqQkjhiudpPxFjhz69oIFUj3PuLYQEZHP2Ggen+nOKQMyo0jdsXXr9O3ixYH77jOsKUiTBnj8cX37xg3gp5+MawsREXldaFzhb6nHRd518+ZN9dVZ/T+1WKnfKFJozllFWHqojJ5mLD10n36qb8+eDXTubGx7iIjIayRYkOWBpKSHSJ8+fbwyDyx7EZKqshdSJuXMmTPqfDqr/acWAzKjePZCtWoFwz30EJAvH3D8uB4snj/PBceJiCwkT9yi3s6gzJMEZBJcSO01u9Uhi73H45bvlcKy93reGJAZIToa+OUXfVsWNJVgyGjS1Sp10MaP19sn+W2eyf5ERGRqEjDkzZtXFb2Nlv/nPUhQcu7cOeTIkcNSBXHvxhvHLetmeuOcMSAzwvr1eoV+0axZ4Czq3bGjHpCJb79lQEZEZNHhy4T5ThKYSJ6ZVJq3W0AWGiDHbZ+zHkh+/tm97VzgOxBUqwYUK6Zvr14NnDxpdIuIiIhsgQGZv8n0WGf+mCQANm2KgCHj39JLJqQ+2vz5RreIiIjIFkwTkI0dOxbVqlVDpkyZ1Ph3mzZtsH//fpjOgQPAP//o23Xq6DlkgeSxx9zbDMiIiIj8wjQB2dq1a9GvXz9s3rwZy5cvVwmJjzzyCK5duwZTWb7cvR1IvWNOlSsDhQrp27KMkjPXjYiIiHzGNAHZkiVL0L17d9x3332oWLEiZsyYgaNHj2L79u0wlRUr3NuNGiHgyLBlmzb6tszCWbzY6BYRERFZnmlnWV6K67nJnj17ks+JiopSN6fLly+7ZlXIzdvkNZ01TRJ16xYcq1ZBKpVoOXNCu/9+91qWgaR1awR9+KHa1H78EZqUw7iX47YoOx63HY9Z8Ljtc715re1zrf1xvVPyuqYMyOQABw4ciNq1a6N8+fJ3zDsbOXLkbfulqm5kZKRP2iWBolzcxKbPhm7fjhxxQWFk7dq4dPYsAlKpUsiVLRuCLlyA9ssvOH30KJA2baqP26rseNx2PGbB47bP9ea1ts+19sf1vnLlirUDMskl27t3LzZs2HDH5w0bNgyDBw+O10NWsGBBhIeHI3PmzD65sFJ4T14/0QvrMbwa1ry5mpwQqBxSjuPLLxF07Rpy7dkDtGiR+uO2KDsetx2PWfC47XO9ea3tc639cb2lvpllA7L+/ftj4cKFWLduHQoUKHDH54aFhalbQnLSffWGkwub5OuvXOluQ5Mm0hAErHbtVEAmghYsuOvyTnc8bguz43Hb8ZgFj9s+15vX2j7X2tfXOyWvaZqzLt2JEozNmzcPq1atQtGiRWEqV68Cv/6qb5coARQujIDWuLGsPqtvL1oUmLluREREFhFkpmHKmTNnYtasWaoW2cmTJ9Xtxo0bMIW1a/VZi85gJ9ClSwc0aKBvS8X+nTuNbhEREZFlmSYgmzx5skq8q1evnloc1Xn77rvvYApS0yuQy10kxjNvjOUviIiIfCbETEOWprZunXu7bl2YQvPm7m0Ztnz9dSNbQ0REZFmm6SEzNZn2umOHvi1lOnLkgClIxX5nWZGtW6VeiNEtIiIisiQGZP6waZM7Kf7hh2EqzmFL6aFcssTo1hAREVkSAzJ/WL/evW3WgMw5bElERERex4DM3/ljderAVGrWBLJm1beXLlXLPxEREZF3MSDzNVmiacsWfbt4cSBfPphKSAggRWzFxYvA5s1Gt4iIiMhyGJD5miTD37xpzuFKp6ZN3dvLlxvZEiIiIktiQObP/DGzDVc6eRayZUBGRETkdQzI/Jk/ZtYesvz5gbJl9W0ZfpWhSyIiIvIaBmS+JAnwUvJCSO5YsWIwLWcvmZTvWL3a6NYQERFZCgMyX/rjD31RcVG7tiwpD9PisCUREZHPMCDzpV9/jV8+wsxkuSeZcSmYR0ZERORVDMh8ybNExIMPwtQyZQJq1dK3Dx4EDh82ukVERESWwYDMHwFZmjTAAw/AzCKjY3Cpdj33DvaSEREReQ0DMl85fx7Yv1/frlwZCAuDGcXEahi3dD8qv7kc3f7N4tofu4z1yIiIiLyFAZmvOKvzm3y4cvzyv/Hx6oO4ER2D3/OUwKWwDGp/1JKlOHbmiuo5IyIionvDgMxXLJA/JsHWtA0RrvuxQcHYVLii2k539TL6Dflc9Zx9sOxvxMZqBraUiIjI3BiQ+SMgM+kMyzNXolTPmKcNRSq5th86vEs9/smag1iw6z8DWkhERGQNDMh8QYqnOocs8+QBChWCGYVnCkO60OB4+9YXqezarnN4p2t7+b5THL4kIiJKJQZkvvDXX8ClS+7hSpMWhE0bGoxeDxWNt+9otrw4miW32n7gv30Ii45S21G3YnH2qr5NREREKcOAzBcskD/mNKhxKfSvXwLp07h7ypx5ZGExt/DA8b/07ZAg5MxozpmkRERERmNA5gOO335z36lRA2YWHOTAkCalseP1xlj/Yn30qVsMvxaq4Hr8waN71NfGZXOrHjUiIiJKOQZkvrB9u/5VhipNXhDWSYKtgtnTY2iTMijboZVr/0PH9uC5eiXQulJ+Q9tHRERkZgzIvC0qCti9W98uXRrInBlWIj1mfZ6si9gSJdT9B07+jcEPFUBQkDnz5IiIiAIBAzIvC/nrLziio/U7VavCqoLq11df1bFu2mR0c4iIiEyNAZmXhTp7xywekKGee11Lx9q1hjaFiIjI7BiQeZkdAzKsWWNkS4iIiEyPAZmXhf7+e9yZDQIquavaW06+fECpUvr21q1wXL9udIuIiIhMiwGZN924oXLIlHLlgAz6QtyW5cwju3ULodu2Gd0aIiIi02JA5k2//66CE8sPVyYybJlm40ZDm0JERGRmDMh8UX/MLgFZ3bquzTS//mpoU4iIiMyMAZmvKvTbISDLmxcoU0Zthu7aBVy9anSLiIiITIkBmQ96yLSQEOD++2ELccOWaqiWw5ZERESpwoDMW65dA/78U98uXx5Ilw62EJfYLxwsf0FERJQqIan7NrpNdDS0119H1KZNCLNyuYs75JFh/XojW0JERGRaDMi8JWtWYPhwXDx9Grly5YJtVnbMnRtayZJwHDgASA7djRv26R0kIiLyEg5Z0r176CH3upZbt/KMEhERpRADMrpnWlxApnDYkoiIKMUYkNG9q1PHvc2AjIiIKMUYkNG9K1YMMblz69ubNgHO1QqIiIgoWRiQ0b1zOHCzRg19W4rD7t7Ns0pERJQCDMjIK6KdAZngsCUREVGKMCAjr3D1kIkNG3hWiYiIUoABGXnFrTJloGXJ4u4h0zSeWSIiomRiQEbeERwM1Kypb58+DUihWCIiIkoWBmTkNRrLXxAREaUKAzLyHhaIJSIiShUGZOQ91aoBYWH6NmdaEhERJRsDMvIeCcaqV9e3//kHOH6cZ5eIiCgZGJCRd3nmkbH8BRkkMjoGx85fV1+JiMwgxOgGkMUkTOxv397I1pDNxMRqGL/8b0zbEIEb0TFIFxqMXg8VxaDGpRAc5Ljt+RKwnbkShfBMYUgbGmxIm4mIBAMy8q5atYCgICA2lnlk5HcSjH28+qDrvgRlzvtDmpROdeBGRORrHLIk78qcGahYUd/+/Xfg4kWeYfIL6e2SACsx0zdGxBu+dAZuEox5Bm6yn4jICAzIyHflL6Ra/6ZNPMPkFzL06AywErp+Ux+aTGngRkTkL6YKyNatW4dWrVohX758cDgcmD9/vtFNosSwQCz5UkwMsGIF8MorQNOmQIECQLZsKFAoF3Z81BlzZr6It3/5EG33rkLmyKvqW9KnCVZ5YikJ3IiI/MlUOWTXrl1DxYoV0bNnT7Rr187o5lByCsSyh4y85cwZYOJE4MsvgX//ve1hyfzKLrfrf6Laf3+i4+/LcDMoBOuKVsbZPi+4kvYlMJOcsaSCsm+2HMHQJmWYS0ZEfmWqgKxZs2bqRgEub16gaFEgIgLYuhW4eRNIk8boVpFZ3bgBfPghMGYMcPny7Y/nyAHkzAmEhEC7eBGO//5zPZQm9hYaHdoGDO0K/DwVGDsWaWvVUgn8nsn/nqas/QchQUHxJgEQEfmaqQKylIqKilI3p8tx/5nHxsaqm7fJa2qa5pPXDmSJHbejVi04JCCLjETsjh3ugrEWYsfr7fdj3rEDjs6d4fBYrF6TheybNoXWpYs+q1eGLB3umZHalSu4uW07bs79ERkXzkeQM0Bbtw6oXRta374YMPot3LoVg8/W/5Poj52x8R88V6+Yq1fNjtfarsdtx2MWPO5Yn53X5LJ0QDZ27FiMHDnytv1nzpxBZGSkT078pUuX1C9zkJR+sInEjjtdhQrIEvf41aVLcb1IEViNHa+3345Z05B++nRkevNNOKSHVXYFBeFG5864OngwYqUX1nMoM6Hy5dTt+msvI+28ecg4cSJCDh1SDzkmT0bQggV4YvxHWJ8tNIkG3MKho8ddeWd2vNZ2PW47HrPgcWs+ud5XrlxJ9nMtHZANGzYMgwcPjtdDVrBgQYSHhyOzlGfwwRtaJhvI69vtF/m2427SBHj5ZbWZac8eZMyVC1Zjx+vtl2O+dQuO556DY9o01y6talVo06YhbfnySJvS1+vXD+jTB7GTJsHx6qtwXL+O4OPHUezJjqjW6Fl8VeGReD1szkkAxQvli9dDZrdrbdfjtuMxCx53uE+ud9q0yf8fy9IBWVhYmLolJCfdV79o8ovsy9cPVLcdd4UKek2yy5fh2LRJPZ7wQ88K7Hi9fXrM0nMtQ5E//ujeN3gwHGPHwnEveYjS1oEDgTZtgG7d1PClIzoab/zyMYqf/AcjGj2L2CB3pf7utYshfVj83jM7Xmu7Hrcdj1nwuIO8fk5T8h6y17uN/EfyfB58UN8+cQI4coRnn+5M8j0ffdQdjIWGArNnA++/771JITJ0LiUzJDiL03XnYkxa9D5CY6JVz1j/+iVUxX4iIn8yVQ/Z1atXcfCge2ZUREQEdu3ahezZs6NQoUKGto0SIQnXy5bp2xs36h+GREnVFuvaFVi+XL+fPr0emMnQt7dJoDd+PPDAA0DPnmqItNmf6/B73jA4fvwRaTNn5DUiIr8zVQ/Zb7/9hsqVK6ubkPww2R4+fLjRTaPE1K7t3mY9MkqKrOgwYAAwZ447GJNA3hfBmCcJABcskCQPdTfdyuVI27mjXqaFiMjPTBWQ1atXT818SXibMWOG0U2jxNSooefuOHvIiBIzYQIwaZJ7qHvu3PjBvC81b64HfxnjesUWLdIDNemxIyLyI1MFZGQymTIB99+vb+/Zk3hRT7K3tWuhDR3qvj99ulSA9v9SXwsXunrK8P33+sxMTVPrWh47f53rWxKRz5kqh4xMmke2a5fMqQa2bAEaNza6RRQgYo79i8g2jyFDXG/UlNodcTV3DQyK1fy/bFHdusAPPwCtW6ucMnz6KVZr2fBcrrpqiaUMoUHoXzMcvR+RqfH+bRoR2QP/ayHf8hx64rAlOcXE4ESzNshw8Zy6u65IZbxbq5Nazmj88r+NOU8yfOmR/lD3s3dQa98mtS1B2cI9JzBxhXvFACIib2JARr7vIXNiYj/FiX77HRT4Y7va/jdzOAa0GuKqAzZ9Y4RxQ4RduuDWq6+pzSBo+PCn91DyjLtky4xNBraNiCyNARn5VuHCQL58+vbmzUyWJmD3boSMfEOdiRhHEAa0GooL6Z0LbQHXb8bgzBX3GrT+dmLQy/i5TB21nSE6ElPmj0HGqOsB0TYisi4GZORbUp3f2Usma3rt3cszbvfir127qir54tMa7bC9QLl4T5HirM41JI0QnjktXms9GH/kKqbuFz//H95ZPFFfX9PgthGRdTEgI99jHhk5jRmjz7gFcLpoaUyo3eW2c9OzdlHXGpL+FhOr4eNVB3EtKA36tH0Fl8IyqP3N/t6ENhsWoHst49pGRNbGgIx8j3lkJPbvB95+Wz8XISHI8cNsPNO4rOp1EoGwbJFMKJCJBbdiNRzLmgeDWw52PdZz6VcYkOOqYW0jImtj2QvyPVlZIV064MYNzrS0czX+555zV8EfMgTBlSthCID+DUqovCwZCjSy90mS9adtiIi3b2WJGphSvR36bP0RwbduIajbU7JkiL6aABGRF7GHjHxP1g6sVk3fPnwYOH6cZ91uvv0WWLXKPdHj9dddD0kQVjB7esOHAiUolPIWCb3/cFfsza3nkzn27QNefNGA1hGR1TEgI//gupb2JZM5/vc/9/2PPw7IHibpoUuXSFAYHRyKV9q+iNiwuEr+sszT4sX+byARWRoDMvIPBmT29e67wMmT+rZUwm/ZEoFIeuh6PVQ00ccebl0XV94Y4d7Rowdw+rT/GkdElseAjPyjZk33Niv228exY8C4ce6ha+d2gJIJBTKxIOFEgwGNSuJGt27QpJq/kGCsf39jG0tElsKkfvKP7NmBsmUBycHZsUNP8JdEf7K2V18FIiP1bQlgSpRAIJM1NIc0KX3bRINYWYvV4YD2+edwVKgAnDsHzJkDzJsHtG1rdLOJyALYQ0b+L38hizdv28Yzb3USeH/9tTsg90jkD3RJTjTInRv48EP3fZk5euGC39tHRNbDgIz8h3lk9usdcxo+HMiWDZbQqZM7D05y4zwnLBARpRIDMjKmQCzzyKxtwwZgyRK1eatgIUT2fgaWWg5syhQgc2b9/hdfAMuWGd0qIjI5BmTkP6VKATly6NubNunFQsl6NA3aa6+57g4r3xaV316LcUv3q6WJLCF//vgTFJ55BrjKKv5E5MeArFu3bli3bt09/EiyLc+Fxs+f15fSIetZtQqOtWvV5j/Z8uHH8g1UwVVZkkiWJrKM3r2BBg307SNHgFGjjG4RUapWqDh2/rr6SiYLyC5duoRGjRqhZMmSGDNmDP777z/ftIysiXlk1qZpiPXoHZvwUBfEBLkT46dvjLDOf/zyB8annwJhYfr9Dz4A/vzT6FYRJYv0VkuvdeU3l6POu6tRbfQKzNvxr3V6se0QkM2fP18FYX379sV3332HIkWKoFmzZpg7dy6io6N900qyDuaRWdvq1QjavFlt/pWzMH4uWyfew9dvxqhyEpYhZTxeftk9e1hmXXIonkxAequl19q5XJh8XbjnBCauOGB002wrVTlk4eHhGDx4MHbv3o0tW7agRIkS6Nq1K/Lly4dBgwbhwAFeUEpC1ap6gVBnHhlZy+jRrs1JNdtDc8T/L0YKrUptL0t56SWgeHF9W4Zqv/nG6BYR3ZH0Uk/bEJHoYzM2WagX205J/SdOnMDy5cvVLTg4GM2bN8eePXtQrlw5jB8/3nutJOuQYrAPPKBv//WXXmCTrOHXX1UPmTifvzAWlXnotqf0rF3U8EXEffKe/ugj930pg3HxYrynME+HAon0Ujt7xhKyXC+2lQMyGZb84Ycf0LJlSxQuXBhz5szBwIEDcfz4cXz55ZdYsWIFvv/+e7z55pu+aTGZH/PIrOmtt1ybWd94Hc81LH3bEkSyNJElNWsGtGvnXlYpLo8uYZ6OfLXUbFMyJemlTpfEH0aW7MW26tJJefPmVcuIdOrUCVu3bkWlSpVue079+vWRNWtWb7WRrJhHJgnQzmHLVq2MbhHdq927gUWL9O2CBRH0VFcMSZPmtiWILG3CBGDpUuDaNeCTT9QC5OPPZlR5Ok7O2aZClmgiMoL8LvZ6qGi896ZT91oW7MW2ag+ZDEVKb9ikSZMSDcaEBGMREYmPTxMxsd+C3n/fvT10KJAmzZ2XILKiggX1FQmcs0379MH09YcSfaqlZpuSKUlvtfRae/Zit6yQFwMalTS6abaV4h4ySd4nuid58wJFiwIStMualjdvuj7AyYSk9M2337rXrOzZE7Y1cCDw5Zeq/EXQb7+hRfhSzLn/kSTzdCRYJTJCcJBD9dI6e7FzZAjF5Qvn1H4yBiv1k7F5ZJGRwM6dvApmNmmSXvJB9OkDZMgA25I/LD7+2HX3pXVfIWPU9duexjwdChS26sW2Wg8ZkdfyyGbOdK9rWaMGT6wZSb6UrOsopJxJv35Gt8h49esDjz0G/PADcl67iP6bZuPt+j2tP9uU/CsqCvj3Xyl3AJw6pf9xK/uCg/U/ijJlAvLlAwoUkDwivZAxBTQGZGSMhzxKIkhANngwr4QZyfDchQv6dqdO+gcAAe+9ByxcqD4gn97xMxZUbY59mfKonjEJxiw725R8Q3qgd+0CZNlCmQi1dy9w8CAQk8w8xGzZgAoVgPvvB+rUAerVA3Ll4tUKMAzIyBj33QdkySJrcekBmVQ3519w5iIfBp71BgcNMrI1gUVyJIcMUaVAgm9FY+GR+Tj+1Xf2mG1K3iG9XUuWAD/+CPz8s/sPn9SQ75VgTm7OIfXy5YFGjYD27YEHH+RVCwAMyMgYQUFAzZr6fzjS3X7okL4MDZmH9ADJX+lCFtlOYta1bcmSSl98ARw/juBFC5H71zU4VbMegzK6s337gKlTga++Srpwdtq0QNmyQMmSeq90njxA+vT6uqryh5KkEkhxYplwc+yY/prHj8d/Dellk5uUaylUCI727REi9fTYc2YYBmRkbGK/BGRCeskYkJmLs5ac4JDz7TJmBN55R6amq7tHu/dB0+4fITRtmKoBJcOWnNFGLhs26O8X+UMnsfdS06Z6fuLDD+vBmOSKpYQEdzKrXVbTWLUK2LEDiI3VHzt6FI5x45Bz3DhoMqTZvz/Qtq17mTvyC86ypMDJIyNzFYKV4Q9RurReqZ5u17kzjpepqDZLnD2GJ3cudhWHlcWdibB1K9CwoZ7b5RmMSW+X5GXKvrNngTlz9MXrZagxpcGYyJFDD+ok6JPATF5zxgx9n8frOdavBzp0AIoUAd59F7h61fUYlwDzLQZkZJzq1YGQuE5aBmSm4nDOrBTPP68PQdNtImM0DKjZ3XV/0IZvkO36JbXN4rA2JzMkJX9LZphLj5VngWEptCxDjLNmAS1a6MGZt0mif7duwC+/ACdPIvbDD3FLhkCd5Oe/9JIKzGLHjMGHP27nEmA+xv9FyTiS81C5sr7955/A+fO8GibguHwZ+OYb91AKi0UnStarHLNoH7blKom55RuqfVmirmHwBv3ccRFnG8+YlLwtGXaUXi+nYsX0WcuSTyspAFJk2V9y5lQla86uXYvY5cuBNm3ck6zOnUPQq6/iqS710GXjHKS5Fc1eXh9hQEbG4kLjppNu7lw4JGlYSDCWObPRTQpIMiT51eYjavudut1wNU06td151xKUOR3B4rB2JHlbMjIgM5KdQ4Hh4XpxZUm8f+opY/O2JAiTCTrz5ul/JD/5JLS43u+skVfx2urpWPl5H7Tct07NjGcvr3cxIKPACcg4bBn4NA3pJe/EqW9fI1sTsCTXZtoG93q+ZzJmx6Sa7dV2sBaL4SunometIiyBYRcy81Fyt2R40nNlkmefBfbv13PDAm35uDJlgK+/xsnNO/FD+QaIhd5jVvDSKXz807uY9/UQFD26Xy27JJhfdu84y5KMxYDMXNauRciBA/q2JCFLsUm6jXxISfK+p+lVW6PTriUodOkUah39HTWu/Smfejx7dsgVk57kNWvc+6RAq+RhSumfAJet0n14rc1QfF6tDV5ZNR11juxS+yuf2I+fvhqM2AwH8GHD7pi886x6z6cLDeYs4lRiDxkZv9C45E4ImfkjxRApYDkmT3bfkb/qKVFSAFY+mDxFhaTBWw16ue4HvzhUX+6GTOuuvUKLF+vBlzMYkyHBV14BfvvNFMGYkELGUqZlX65i6NphFLo9MRIHchR09faGfjIJHZ96BI12r1I96JxFnHoMyCiwFhqXHAsKTDLrav58talJ8ch27YxuUcB/iCW0tGRNHK0Yt27rP//oyd1kygkb45bud806rDRyGYbP34trUbf0J8jKI2PGAC1buivsy+xJqQH21lumq+8lNfP61y+B9GEhWFusCh5/dhLW9fwftHR6XmSuaxfw0c/v4avvh6PApVNqH/PLUo4BGRmPw5bm8PnncMgMMdG7d+DlvATqh1gavadMvvZvUBL5Z3zqLhMiH86yODSZbsKG1JJzDktH3opVEzgqjlyGifN3QHuiPfDqq3pgJqTIqtTuq1sXZiQFjIc0KY0drzfG+hfrY8sbzfHwtHE48esOLC8R9wcGgIcP78SS6f3RRertRUW78ssoeRiQkfEYkJkjKVmWc5E//oOCoD39tNEtMt2HmHyV+8GVKgLPPKM/SWbayQc3mXbChqdcF0/hkWceg+OHue4hytGjgR9+0Ot+WaDnt2D29K7JKNnLlcQLHd9A73av479M4Wpfxps38NayT/Dt968j1zn+sZESDMjIeOXKAVmz6tvOhcYpsEhtIklOllwomRZfqJDRLTLth5jy5ptAliz6tqx3KTlFZNoJG6Ls6X/w49dDUPbMYXVfk3IwP/2kB9zOml4WHZpfUbIGmvSahFkVm7gee/DwboRVrghI3qlziSa6IwZkZDwZvqlVS98+cwZwzuKjwDFtmmvzhiznQvdGak+NGOG+P3Ag/xAx8YSNWod34ftvXkKeq3px64hseXFy2Vo9h8wmQ/OxmTLhlabPo3fn0bgcnld/UOoVyuSfRo2Aw3qgSkljQEaBgcOWgUuC5AULXMn8UY0bG90ia+jXT18H1Nkz/N13RreIUjFh49E/12DGnDeQ6eYNdX9n3tJ4sscHyPZABVsOzX8842VkPviXe1heyGQGmW0qf9hxBCRJDMgoMDAgC1xffw1ER+vbRlcStxKZFPHBB+77Q4cC168b2SJK5gzLWE1DSJBD1ZWb8PP7SBOrT3ZZXqI6OnV6C20fqWy7or/xhuZluPbTT4Fly9zpDVeu6JOBHn1UrZ1Jt2NARoGhWjX3Bz0r9gcO+WvWY7hS69HD0OZYTvPmQNOm+rbk6L33ntEtomTMsPxkzSF02fYTxi79GEHQc16/qdQUgzsMR+9HyqthPCtJdRV+6U3//XfA8/+NhQuB++4Dvv/e6+00OwZkFDgLjT/wgL7911/A2bNGt4jEli36mnbOXkxZToW8S3rJQuIWTZHldY4d4xkO8BmWz26Zi5ErPnXt/7R6O4xq8TzWvxI3kzbIYcl6a/JV7sv+ZJPJK9On62kPuXPr+86fBzp0ACQfVbZJYUBGgYMLjQcej94x9HJXmScvKltWzycTN24AL73E0xugzlyOxNNrZmLYGvd6rhNrdcTYej0QeUvDlci4On0Wrbd2T1X4Zahy717g8cfd+2bPBsqXB375xYutNi8GZBQ4mEcWWKRGlvyHKTJlAp54wugWWZfMuMyRQ9/+9ltg0yajW0QJaRryvjcagzd849r17sNPYXydJ1VZCyn8KzMw7VBvLdVV+HPm1Icqv/nGXepICiPL0P0zz+h5ZjbGgIwCBwOywCL/cUpQJjp2BDJmNLpF1iVFQ0eNct8fMIC1mwLN6NEIeedt191R9Xvhk5rtXfd71i5qqUT+pOqties3Y1JfhV9qsnXurPeWNXHXLYMUnq5YEVi/HnZluoBs0qRJKFKkCNKmTYsaNWpg69atRjeJvEXyC0qU0LelUCYXGjcWhyv9S1Y/kOEb5/v/q6/83ABK0rhxwPDhrrsrn3sN3z70uHtJrPolLJfIn1i9NSev9Abmz68PVU6ZAmTIoO+LiNCXlxoyRF/b2BcTDQKYqQKy7777DoMHD8aIESOwY8cOVKxYEU2aNMHp06eNbhp5u5dMgrHt23lejbJvn3vYTGZEVa/Oa+Frktjvudj4sGG2H8IJCJ98opckcXr/fTScNOr2JbEsksifVL01T17rDZTesmef1df5fOgh98zu998HqlRJ9DPAKxMNApSpArIPPvgATz/9NHr06IFy5cphypQpSJ8+PabLDA6y3rCljbuuDef5OyW1gyy69EvAadgQaNNG35ZaTWPHWrInwDRkWSvnhAshw8qDBye9JJZFq/BLj5hPewOLFwfWrNHLvkh9PiGzux98EBg50l0H0dsTDQKMQ9PMUTb35s2bKviaO3cu2jj/wwLQrVs3XLx4EQviKol7ioqKUjeny5cvo2DBgrhw4QIyS+E6L4uNjcWZM2cQHh6OIFkOyCa8etz79yNI1raUP5SaNYMmNWsClGWv961bcBQuDMfJk9BCQ6FJfSxJxrXyMd+FX4/70CE4ypeH4+ZN3AoJRYtnp+BAxlxq+KhH7aIY0Kik33pj7Hi9nceca9UqBHXtKh+Sar/28svQZKFwi/5xcqdrLX8MnL0ahZwZw3wfgP7xBxzdu8OxY4drl1alCrQZMxBZsjSqjV6RaG6bBItbX22U4vb5+j0ucUe2bNlw6dKlu8YdccVvAt/Zs2cRExOD3M46JnHk/l9StyoRY8eOxUiJrhOQkx95l/Hp1JALKyddYly7/Ofl9ePOmhXhOXMi+OxZaBs24LTMwAkOzL9ArXi9b96KRcyyFSgaV0k7qlEjXJSFgePSAqx4zMnh1+POlAkZn34aGSdNQsitaIxaORkju70ufz5j9e4DyBB7FW0fKAB/sOP1lmO+tWgRgp57zhWMXXv6aVx54QV9GTGLutu1ln6ryxeu4rI/1nmdPx8ZJ0xAhokT4YiJgUOGLqtWxaVBQ1E0c1Voib4Xb+HQ0eMpzm3z9Xv8SgpmjpomIEuNYcOGqZyzhD1kEgn7qofM4XDY6q9JXxy3Q5I6f/gBQVeuIJcEBpUrIxBZ6XpL/sXEFQfwxcYIjPlxOpyZIyE9eyFXrlyWPOaU8PdxR77xJk59NRu5r5xD9b9+Q4GtW7C8VE312JHNZ9CtYUW/DJXZ8XrHbt6MoIED4bil1xTTnn4a6SZPRjqL9owF7LV+7z1o7dsD0lv2119wREUh79ujMaJQefyv2QD8mzXPbT1kxQvlS1UPmS+PWyYgWi4gyym9JsHBOHXqVLz9cj9PnvgXxiksLEzdEpKT7qs3nFxYX75+oPLqcccFZCJowwY9uTNAWeV6f7B8Pz5ecwgZoq7jkb83q30X02bEF2GlMCjBsVnlmFPKn8d9Ligt3q7fG5N+ekfdH75iKtYVeQA30qTF1ZuxOHctGgWz+2dNUVtdb/ngf/RROJxrij72GByTJ8MRoL30lr/WNWoAMnT56qv6hBdNQ7Wje7H4ixcwun4vzK7YxDWE3L12MaQPCw24407JawbIWb+7NGnSoEqVKli5cmW8yFbu16yp/+VIFvHww+7tdeuMbIkteBaAbPr3r0h3S8+7XFimDqZu+4/J5AaQYZdVFepiXRG9dzj/lTN4YZNepNdqBUgDhuRKNmkCx7lz6q5Wrx4wc2bApkzYRrp0+vJiq1cDRYqoXRlv3sDbSz/G97NeQqULRy1TdsQ0AZmQ4cepU6fiyy+/xL59+9C3b19cu3ZNzbokC5FaTM4qzhKQmWPeiSUKQLb9Y5Vr/4/3Nbi3ApB0byUH6hTD8MZ9EBWsD2T03jYPJc8csVwB0oAg6ynKIu9Hj6q70eXLQ5s3T8abjG4ZeY6cyELlUq8vTvV//8S8ac9jyC+TEXzF59ltPmeqgKxDhw4YN24chg8fjkqVKmHXrl1YsmTJbYn+ZHLyF6mzJo0sMp7EpA3ybgHIPJfPotaR39W+w1nzYkf+MuyNMZD8xd/ysbqYVkuvBh8aG4Pp277AoEYljWyW9cjwZKtWanaf0IoVwwVZ2scHecZ0jzJlAj77DFi2DCil94hJ0j8mTgRKl9aLKcskJJMyVUAm+vfvjyNHjqhyFlu2bFHV+smCOGzp9wKQrfetQRD03sh599VXuRnsjTGOlLaQgqM9F36KW0X0aRYF9/yG4FnutRQp5eLVdZP6VpI4HlcEOSZXLkT+vAixHhNZKAA1bqz3lo0Zow9pCskv79ZNzcbE8uUwI9MFZGSj7mkn5pH5nPS6PB2x0XV/SaWGlsnLMLu0mTMi5JNJ7h3/+x9w4YKRTTKl2yq8j1yGvc3bA4sWqcevpkmHR5u/iqrfRmDejn8tUfnd0sLC9NUsZFWRdu3c+3fuBB55RA/aTLbaCwMyCkxS6sK5vtnatcwj87Hg33cj55EDajuq+oNYML6bJZeDMa1mzdSMP0VqYXku5UPJkrDC+4Dln6P8ivlqW/L0nm73Ov7IXVw9vnDPCVUGhkygcGF9Vr4MY3qWSFqxQu8t69gR2LNH7Qr0VS8YkFFgCg0FatXSt//7T190lnzn669dm2E9ujFpPBDJtH/JoXEu/C4fOJTimcSi99Yf0Wfrj2o7Fg4MaDUUvxa+P973zNgUEbAf3JQI6RH77TdA8v/iZmMq330H3H8/DtZsiG7PfhTQ618yIKPAxTwy/5ACmLNm6duyjpzk1FDgKVAAePdd932ZbXbtmpEtMudM4r2r8Npq91qtrzV5DktKe6yhG4czjE0oKAjo3FmfCCaJ/nFLvokSm1fhuy8G4btvXkLD31djyoq/Am79SwZkFLgYkPmH9LQ4Cy63bAlkz+6nH0wp9swz7vzKw4f1gpmU7JnE9Q9tw3uLJ7j2f/BQF8yq1CzR72G9N5Pnl73wgvodiR73Pk5myuF6qMa/f+Djn97Fpsk9kO6tNxF1KHBGXxiQUeCqXl3vsRFM7PfLcCW6dvXhDyKv9ABMnequj/Xhh8Cvv/LEJmMm8evZL+CT+W8jRNPLInxVuQU+rNURVQtnS/R7utdivTfTy5ABJ3v0wcPPfI4Xm76Ag9nda8DmunYB/dbPQljpksj2xBPAn3/CaAzIKHDJh46zrMmhQ3ouGXmXLHwrBTCF9Iw1b84zHOhKlgRGjdK3pWhyz57AjRtGtyqw7d2LTm8+F28Vineb90X/BiUx6+kH1Yxi6RET8rVlhbwYwHpvlukdDU6XFt9XfASNek9Gp45v4ZdStXDLoYc/QZqG0E2/AjncvWhGMc1almTjYcv16/Vt+SozZsh7ZHaS88O8Qwd3jyQFtoED9WRlSWKWfBmZ/i9J/3S7I0f0JZEuXlR3Yxo2RMWv5+C3nJldk1dkRnH/BiVUrlmODKG4fOEcZxhbrM7ix6sPqtqKvxauqG5SCPuxvSvxxN4VOJ2vEMpnz4n0BreVPWQU2JhH5r/hyqee8vEPI68JCQFmzNBzZYQkMJu0GKZPSYkQqUl1/Lh+v2pVBM+bh4J5s902k1juF8yenjOMLWhQ41J46sHC8fadzJwTk2p1QIOnP8W4xwfg7FXjl4hjQEaBTRaOdy7uyzwy7y+mLAv2OofBuOqFudx3H/DOO+773bvrazKSezhehuD/jptJJ0vtLF7sLh1CthEc5MArLcoibWgiIY/DgVuZMiNnxrg/bgzEgIwCm/znWaWKvi1rzcnaluQdUq/HuXD7k0+q/5jIZJ5/HmjUSN+WXqC+fVlEWURFAW3b6kO6Il8+vXBoeLiBF4uMlDY0GL0fKpboY43L5g6InlEGZBT4OGzpfRKIyUK8ThKQkTlnXcrQZba4mYLff68H2nYmi03LbOGVK/X7cm4kGJOK7gS7D132TzCB47l6JdC6Un4EAgZkZK51LZ1DbHRvZL035zTv2rWBYon/5UgmkD8/8Omn7vvSSyaJ/nYUGwv06gXMmaPfl4WnFy7Uh3fJ9oKDHGoCx47XG2P9i/XV18GPlEJQgCwRx4CMzNFD5swjW7XK6NZYA5P5rUXqKEkOmbh6VV/30m5V/KXXt18/4Msv3RMf5s51L8FGFOATOBiQUeDLnFlfJFZIr46zqjylfqmkb7/Vt6XMhXyYk/lNmgRUqOD+PXn22RTlkwX6wst3JMc5aBAwZYp+X/6Amz2bdfXIVBiQkTk0aODe5rDlvZHyCM6gtlUrd/4RmVv69HqPkHMWoeSSeQ5lJkEWWJaFlmXB5UBeePmOwdgrr+ilP4RMTpH8SOklJDIRBmRkvoCMw5b3hkslWZeUdpg2zXVXGzAA2Lr1jt8iCyxL0Uzn4tvyVe4H2sLLSXrzTeDtt9335fhlgWkik2FARuYgeSDOKvIMyFLv8mX3UkmyVEizxBdWJnNSvV2Z78dX1Vur+46bN3G1WUvEHD2W6PNleHLahsQXV56+MSKwhy+lZ+z114E33nDv++QToEcPI1tFlGoMyMg8wzFSJNa5rqUsh0KpWyopMlLf5lJJluPs7Rr1cHdsKaDPLMx4/gzONmiaaJK/LBXk7BlL6PrNmICoXp5kMDZ0KDB6tHvfBx/oM0yJTIoBGZkH88juHWdXWpZnb1d0cCj6tH0FR7PkVvdzH/oTVx/vgMgbUbctvJwuiZlmUqMpEKqXJ1raon9/4P333fs++khP6icyMQZkZB7MI7s3x44Ba9a4l0qqXt0bV4UCRMLergvps6Dn4yNwOY2+ZHLGJYuwsFZrjFvylyth37nwcmJ61i4acGUBEB2tD0nK0KQzgX/qVD1AIzI5BmRkHhJAyNClM48sBVP6KcFSSVLJnEslWUpivV0HcxZC37av4GZQiLr/+K6lSD/y9XgJ+4lVL5f7sj/g1qZs0cK9woSsUiDbvXsb3TIir2BARuYhSf116ujb//0HHDhgdIvMQwIxz+FKLpVkOUn1dm0sUgmDWw5GLPRq5M9tnovQt8e4EvYTq14u92V/wDhxQi8QLSVbRFiYXo2f72OyEAZkZC4ctkydHTvcSyU99BBQNPFhKjK3hL1dTgvLPozhjfu47g9Y8xWiXh9hiurl2LtXn9Cza5d+X+rmyTqV7doZ3TIir2JARubCgCx1mMxvC569XSsGPYy0oe7/4mc+0AJv1evpup/lnbeAV18N7KH/774DatRwz6qWBcI3bdLXXyWyGAZkZC6VKwNZsrgr9suMK0r+Ukky1MOlkixPerlK5M6E3g/FXzR+ao12GNXAI+dqzBigZ089Wd5LvLIEk7xnhwwBOnYErl/X91WpAvz6K1CmjNfaShRIGJCRucgadfXq6dtnz7qHMShpy5YBp0+7l0rKmpVnyyYSS9hP9+IQxEqZCOekjhkzgJYtgYsX7+lnpXYJpoQBXGTEEUTWbxi/rEW3bsD69UDevPfURqJApk+9ITKTRx4BFixwBxsPPGB0iwLbl1+6t596ysiWkEFDmP0blFBlMWQmpsoRa1IayJNHT4qPitJ/j6QHStbClF7oeyhK6+RcgklIGxKSQE2+R2qnyXPThgThmRPb0GvWe8gSeVV/TnAIHBMnIOi55zgrmCyPPWRkPk2auLeXLjWyJYFPej2cwWt4ONC0qdEtIgMkmrD/+OPAihX6Elrin3/05Hmp8ZXCVIDULMHkuYZm7itn8cGc0Rg8fYQrGDuRMQfadxyDD0o0YjBGtsCAjMyneHH9JjZuBK7q/4FTIqQ0gPSACFlwOTSUp4ncZMbt9u1AtWr6fXmv9OsHNGyoL1GWTHdbgkkeTyyAC42JRs9tC7Di875o/vcm1+M/l6mDpj0/xvYC5QJ/TU0iL2FARuYdthSSjOysPk+3cxbRFByupMTIzEXJz5JAzEl+p8qXh2PYMDguXbrrebvbEkzyuKczF6+jya4VWDW1D4avmopMN2+o/WfTZ8ELrYbg+UdfxKV0mZIM6IisiAEZmROHLe9Oejg2bNC377sv1blBZAMy+/bjj/XCq0WK6PsiI+F4912EP/ggHCNH6sVZk3CnJZjK5c2M0OC4j5rLl4EJE5C/+v2YsPB9FLx0yvW8WRWboGHvKfipXL14Q5SJBXREVsSAjMypfn0gJG5OCvPIEjdzZvzeMS6VRHfTqBGwZ49eckJWxpAPiYsX4XjzTb0nTUqmSAmVRHrNZEZn1cLZbtt/aN9hLH59op6zlju3WgQ86PBh1+Nriz6AFt0m4JWmz7t6xQJ+TU0iH+AsSzKnzJmBWrWAdev0JZQiIlh93pMU+3QOV0og1qWLMdeJzCdjRuC999SC3drrrwOzZsERE6OnB8gsTLnJH0PS6yoznEuUAHLlQkyGTCiybAeKRUYi35UzKHzhBMqd/gelzx5N9MdoTZri+wYdMfJqbjUsmS40CPfly4I/TlzGjZsxqmdMgrGAW1OTyEcYkJG588gkIBMybf/ZZ41uUeCQauYya87Z65E/v9EtIrMpXBjajBk4O2AAwn/4AY6pU/Xaf87Crbt367c40p827m6vmSsX0L490LcvHOXKoQOA1tEx8UpySAJ/vBIdRDbBIUsyL+aRJY21x8hLYvPnhzZ6tJ5DtmoVMGCASvhXRZrv4pYjCLvylsQXD7bDzV+WAP/9B0hR2nLlkizJEbBrahL5GHvIyLxkuCRnTv2vdllsWIZUWNYBuHED+P579/BT27ZGXymyAhmmlNxNuQlZ0kgW/pYg68wZlbC/8p+LWPHPJZzJkA1HsubB0ax5EBUaplYLSJNIcVgicmNARuYVFAQ0bqwnGcvsra1bueiw+Plnd9K1JFJnyGDsdSLTcQ4b5shwh7p16dMD1avH21UvVsPO5X9jwcYIlRcmeWD9mQdGlCwMyMj8eWTOhbNltmXt2ka3yHisPUaplHA5owyhQehfMxy9HwlXf/+keqkmIror5pCRNQrEil9+MbIlgeHUKWDJEn27YEGgbl2jW0Qm4rmckZCvC/ecwMQVB1L0OswDI0o5BmRkbvnyAZUq6du//QacPAlbk95CKVEgunbVh3WJ7nE9yhmbuHwRka/xf2syv5Yt3duLF8PWPIcrJSAj8tF6lETkXQzIyFoB2cKFsC2psL5zp74tydZlyhjdIjKRlK5HSUTexYCMzK9aNSA83F0gNsqmf8lPm+be7tbNyJaQCd1pPcrutbh8EZGvMSAj85M8qebN9e1r14C1a2E7EoQ6166UhaI7dTK6RWRCskyR1AyTHjEhX1tWyIsBjUoa3TQiy2PZC7LOsKWzOr0MW3rOvrSDn34Czp3Ttx97DMh2+yLPRCktWyF1yC5fOKf2E5FvsYeMrEECMKkk7gzIZHFtuw5X9uxpZEvIAli2gsj/GJCRNWTO7K65FREB/PEHbOPoUT13ThQt6l7ahoiITIMBGVlHq1bu7fnzYRsyVOvsEezRg7XHiIhMiAEZWUebNu7tefNgC7GxwPTp+rbDwdmVREQmxYCMrKNwYeCBB/TtHTuAw4dheatXu49T8ugKFTK6RURElAoMyMha2rWz17ClZzJ/r15GtoSIiO4BAzKylrZt3ds//ghLkzIXzmPMkQN49FGjW0RERFYPyN566y3UqlUL6dOnR9asWY1uDgWqsmWBUqX07Q0bgNOnYVlffOFeleCpp/SCsEREZEqmCchu3ryJJ554An379jW6KRTIJLHdOWwpMw+lYKpVk/mnTHHf79PHyNYQEZFdArKRI0di0KBBqFChgtFNITMNW86dC0tauRI4dEjfbtjQ3StIRESmZOmlk6KiotTN6fLly+prbGysunmbvKamaT557UAWcMddpQochQrBcfQotBUroMmwZc6cljpuxyefwLmYTeyzz+o9Zna81n7C47bP9ea1ts+19sf1TsnrWjogGzt2rOpZS+jMmTOIjIz0yYm/dOmSurhBsuC1TQTicWds2RIZJWiJicHlGTNwQ3KsLHLcQSdOIPznn9V2TO7cOPPgg37LlQvEa+0PPG77XG9ea/tca39c7ytXrpgjIHv55Zfxzjvv3PE5+/btQ5kyZVL1+sOGDcPgwYPj9ZAVLFgQ4eHhyCxL7fjgwjocDvX6dntDB9xxS8X6Tz5Rm5l/+QWZhgyxzHE7Jk9WgaYIevpp5Mqf397X2g943Pa53rzW9rnW/rjeadOmNUdA9r///Q/du3e/43OKFSuW6tcPCwtTt4TkpPvqDScX1pevH6gC7rirVAFKlgQOHIBj7Vo4Tp4E8uUz/3HfugV8/rm+HRQExzPPwOHncx5w19pPeNz2ud681va51r6+3il5TUMDMolI5Ubkk9mWHTsCo0bpsy0luf+FF8x/omWo8vhx99qdBQsa3SIiIvIC04TBR48exa5du9TXmJgYtS23q1evGt00ClQSkDnNng1L+Phj9zZLwBARWYZpkvqHDx+OL7/80nW/cuXK6uvq1atRr149A1tGAatcOUDKpOzZA/z6KxARARQtCtPavRtYtUrfLlECaNzY6BYREZHdeshmzJihZkEkvDEYozvq3Nm9/dVX5j5Z48e7twcNUjlkRERkDfwfnayta1d34CI9rCarsRMZHYNj568j8ui/wKxZ+s5s2YBu3YxuGhEReREDMrI2KQnhHNqTIct162AGMbEaxi3dj8pvLkedd1fjiy4vAtHR+oNSCDZDBqObSEREXsSAjKzPs7TKjBkwg/HL/8bHqw/iRnQMwqKj0GH7IrU/JjgE6N/f6OYREZGXMSAj62vdGsiSRd+eM0dKJyPQhymnbYhw3W/3x2pkv6Ev+7W4bB1E5spjYOuIiMgXGJCR9aVLB3TqpG9fv64HZQHszJUo1TMmHFosem2b73rs0yqt1eNERGQtDMjIfsOWU6cikIVnCkO60GC1Xe+f7Shx/l+1vaVgefxTqLR6nIiIrIUBGdlD9ep6TTKxeTOwYwcCVdrQYPR6qKhaYaD/pu9c+z+v1gY9axdVjxMRkbUwICP7LKXUr5/7/qRJ8ctKxA0RBopBjUvhnRznUOX4X+r+gfDCKN27s9pPRETWY5pK/UT3rEsX4MUXgcuXoc2ahUnNnsGk3RdUvla6uF4pCXiCgxyGn2xpQ4el7kK2hd9/C0OalTW0TURE5DvsISP7yJjRlUvmiIzExU8+cyXPy1cpMyHlJgLCpk3uZZJKlkSazh7rchIRkeUwICN78ViQ+8mdv6hZjJ6mb4wIjOHLUaPc28OGAcHMGyMisjIGZGQvZcog8mF9MfoiF0/gkQOb4z18/WaM8WUl1q8HlizRtwsXBp580tj2EBGRzzEgI9sJ+t9g13bfzXPUbEan9GmCjS0rIW2RHjGnESOA0FDj2kNERH7BgIxsJ02rljhdtLTarnTiAGoe/d31mOFlJRYvBjZu1LfLlNEXRyciIstjQEb243Ag56jhrrt9N89VPWP965cwtqxEbCzw6qvu+6NHAyGcCE1EZAcMyMiWgjq0B4oVU9sPH96JnS2yY0iT0saWvJg5E9i9W9+uUgVo1864thARkV8xICN7kp4nqUkWJ2zUm4Y2R2qj4aWX3PffflsvZktERLbAgIzsq1s3oGBBfXvRImDtWuPaIsOTJ0/q223aAI0aGdcWIiLyOwZkZF9p0wJvevSMSQ+Vx4xLv9m/H5gwQd8OCwPef9//bSAiIkMxICN7k1mM5cvr21u2AD/84N+fLwHg888D0dH6/aFDXbltRERkHwzIyN6kAr7kazlJDbAoPxaGnTYNWL5c3y5QAHj5Zf/9bCIiChgMyIiaNwfq1tXPw8GDwJgx/jknR48Cg91FavHpp0CGDLweREQ2xICMSGYzfvSRu+bX2LHAH3/4fqjymWeAK1f0+7LouQSGRERkSwzIiESFCu7hQsnn6t0biPHhIuMffAAsXapv58sHjB/P60BEZGMMyIicpEp+aX1JJWze7LvZjuvWxa859vnnQNasvA5ERDbGgIzIswzG1KnxE/xXrfLu+Tl+HGjf3t379sorQLNmvAZERDbHgIzIU506wGuvudeW7NABOHbMO+fo4kWgZUvg1Cn9vhR/9ayDRkREtsWAjCihN94AmjTRt8+eBdq2dSffp9a1a0CLFsDOnfp9WSFg1iy97AYREdkeAzKihCRIkmCpSBH9/vbtQKtWwNWrqTtX8n2tWwObNun3c+bUE/rDw3nuiYhIYUBGlJjs2YGff9a/ClnnskED93qTyXXkCPDww8DKlfr9LFmAZcuAsmV53omIyIUBGVFSZEmlJUvcMyC3bQMqVwZ++SV5dca++UZ/vnOYUoIx+V7ZR0RE5IEBGdGdVKsGrF+vL2skpIdMCrhKcv6aNXrivydZdmnePKB2beDJJ4ELF/T9RYvqQ5Y1a/J8ExHRbeJKkxPRHXvKduzQAywZbhSLFiFo0SLkypYNjkqVgEyZ9AkAu3YB16/H//4nngA++4y1xoiIKEkMyIiSQxLwZfjyq6+A1193lcIIkh6w1asT/5777tMXLpfeNCIiojvgkCVRSta87NZNX4D8m2+gPfooYp1J/06FCgFdu+q5Yr//zmCMiIiShT1kRCmVJg3QuTO0jh1x+tQp5EqXDkE3buhDkunS8XwSEVGKMSAjutdes8yZmR9GRET3hEOWRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMFstnaRpmvp6+fJln7x+bGwsrly5grRp0yIoyD6xLo/bPteb19o+19qu19uOxyx43Gl9cr2d8YYz/rgTWwVk8ksmChYsaHRTiIiIyEbxR5YsWe74HIeWnLDNQn8BHD9+HJkyZYJDFoX2QSQswd6xY8eQWRactgket32uN6+1fa61Xa+3HY9Z8LiP+eR6S4glwVi+fPnu2gNnqx4yORkFChTw+c+Ri2qnX2QnHrd98Frbix2vtx2PWfC4ve9uPWNO9hkgJyIiIgpQDMiIiIiIDMaAzIvCwsIwYsQI9dVOeNz2ud681va51na93nY8ZsHjDjP6EtgrqZ+IiIgoELGHjIiIiMhgDMiIiIiIDMaAjIiIiMhgDMiIiIiIDMaALIUmTZqEIkWKqHXOatSoga1bt97x+XPmzEGZMmXU8ytUqIDFixfD6sc9Y8YMtRKC502+z0zWrVuHVq1aqerK0v758+ff9XvWrFmDBx54QM1WKlGihDoPZpPS45ZjTnit5Xby5EmYxdixY1GtWjW1gkeuXLnQpk0b7N+//67fZ/bf7dQct9l/tydPnoz777/fVfy0Zs2a+OWXXyx9nVNz3Ga/zol5++231XEMHDgQgXq9GZClwHfffYfBgwerKdE7duxAxYoV0aRJE5w+fTrR52/atAmdOnVCr169sHPnTvUfntz27t0LKx+3kF/6EydOuG5HjhyBmVy7dk0dpwSiyREREYEWLVqgfv362LVrl/ql7927N5YuXQorH7eTfJB7Xm/5gDeLtWvXol+/fti8eTOWL1+O6OhoPPLII+pcJMUKv9upOW6z/27LSi3ywbx9+3b89ttvaNCgAVq3bo0//vjDstc5Ncdt9uuc0LZt2/Dpp5+qoPRODL/eUvaCkqd69epav379XPdjYmK0fPnyaWPHjk30+e3bt9datGgRb1+NGjW0Z5991tLH/cUXX2hZsmTRrEJ+TebNm3fH57z44ovafffdF29fhw4dtCZNmmhWPu7Vq1er5124cEGzitOnT6tjWrt2bZLPscrvdkqP22q/2yJbtmza559/bpvrnJzjttJ1vnLlilayZElt+fLlWt26dbUBAwYk+Vyjrzd7yJLp5s2b6q+LRo0axVsbU+7/+uuviX6P7Pd8vpCepaSeb5XjFlevXkXhwoXVIr13+0vMCqxwre9FpUqVkDdvXjRu3BgbN26EmV26dEl9zZ49u62ud3KO20q/2zExMZg9e7bqEZQhPLtc5+Qct5Wuc79+/dToRcLrGIjXmwFZMp09e1a9kXPnzh1vv9xPKl9G9qfk+VY57tKlS2P69OlYsGABZs6cidjYWNSqVQv//vsvrCqpa3358mXcuHEDViVB2JQpU/DDDz+om/znXa9ePTW0bUbyXpXh5tq1a6N8+fJJPs8Kv9upOW4r/G7v2bMHGTNmVLmeffr0wbx581CuXDnLX+eUHLcVrrOQwFP+L5J8yeQw+nqH+OWnkK3IX12ef3nJL3LZsmXVGP6oUaMMbRt5l/zHLTfPa33o0CGMHz8eX3/9tSn/mpZ8kQ0bNsBOknvcVvjdlver5HlKj+DcuXPRrVs3lU+XVHBiFSk5bitc52PHjmHAgAEqP9IsExIYkCVTzpw5ERwcjFOnTsXbL/fz5MmT6PfI/pQ83yrHnVBoaCgqV66MgwcPwqqSutaSGJsuXTrYSfXq1U0Z0PTv3x8LFy5UM00lCfpOrPC7nZrjtsLvdpo0adQsaFGlShWV8D1x4kQVbFj5OqfkuK1wnbdv364mnsnMdycZ7ZH3+ccff4yoqCj12RZI15tDlil4M8ubeOXKla590o0r95Mah5f9ns8XEq3fadzeCsedkPwSSHe5DG9ZlRWutbfIX+FmutYyf0GCEhnCWbVqFYoWLWqL652a47bi77b8fyYfzla9zqk5bitc54YNG6o2y/9HzlvVqlXRpUsXtZ0wGAuI6+2XqQMWMXv2bC0sLEybMWOG9ueff2rPPPOMljVrVu3kyZPq8a5du2ovv/yy6/kbN27UQkJCtHHjxmn79u3TRowYoYWGhmp79uzRrHzcI0eO1JYuXaodOnRI2759u9axY0ctbdq02h9//KGZaWbOzp071U1+TT744AO1feTIEfW4HK8ct9M///yjpU+fXhs6dKi61pMmTdKCg4O1JUuWaGaS0uMeP368Nn/+fO3AgQPqfS0zmIKCgrQVK1ZoZtG3b181o2zNmjXaiRMnXLfr16+7nmPF3+3UHLfZf7flWGQWaUREhPb777+r+w6HQ1u2bJllr3Nqjtvs1zkpCWdZBtr1ZkCWQh999JFWqFAhLU2aNKocxObNm+Nd7G7dusV7/vfff6+VKlVKPV/KIixatEiz+nEPHDjQ9dzcuXNrzZs313bs2KGZibOcQ8Kb8zjlqxx3wu+pVKmSOu5ixYqpqeNmk9Ljfuedd7TixYur/6yzZ8+u1atXT1u1apVmJokdr9w8r58Vf7dTc9xm/93u2bOnVrhwYdX+8PBwrWHDhq6gxKrXOTXHbfbrnNyALNCut0P+8U9fHBERERElhjlkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZERERkMAZkRERERAZjQEZElIgzZ84gT548GDNmjGvfpk2bkCZNGqxcuZLnjIi8iouLExElYfHixWjTpo0KxEqXLo1KlSqhdevW+OCDD3jOiMirGJAREd1Bv379sGLFClStWhV79uzBtm3bEBYWxnNGRF7FgIyI6A5u3LiB8uXL49ixY9i+fTsqVKjA80VEXsccMiKiOzh06BCOHz+O2NhYHD58mOeKiHyCPWREREm4efMmqlevrnLHJIdswoQJatgyV65cPGdE5FUMyIiIkjB06FDMnTsXu3fvRsaMGVG3bl1kyZIFCxcu5DkjIq/ikCURUSLWrFmjesS+/vprZM6cGUFBQWp7/fr1mDx5Ms8ZEXkVe8iIiIiIDMYeMiIiIiKDMSAjIiIiMhgDMiIiIiKDMSAjIiIiMhgDMiIiIiKDMSAjIiIiMhgDMiIiIiKDMSAjIiIiMhgDMiIiIiKDMSAjIiIiMhgDMiIiIiIY6/+CkBz//Uf6PQAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "p_best, hist_best = gradient_descent(model_loss, model_grad, xs, ys, p_start, 0.05, 20000)\n", "print(\"your estimate:\", np.round(p_best, 4))\n", "print(\"true values :\", np.round(p_true, 4))\n", "\n", "x_grid = np.linspace(0, 4, 400) # 400 evenly spaced x values, for a smooth curve\n", "\n", "# YOUR CODE HERE (1 line): evaluate your fitted model on x_grid, storing it in y_curve.\n", "# Your forward function accepts a whole array of x values at once.\n", "y_curve = forward(x_grid, p_best)\n", "\n", "plt.figure(figsize=(7, 4))\n", "plt.scatter(xs, ys, s=18, label=\"data\")\n", "plt.plot(x_grid, y_curve, \"r-\", lw=2, label=\"fitted\")\n", "plt.xlabel(\"x\"); plt.ylabel(\"y\"); plt.legend(); plt.grid(alpha=.3)\n", "plt.title(\"3(d) fitted damped sinusoid\"); plt.show()" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "name": "python", "version": "3.13.5" } }, "nbformat": 4, "nbformat_minor": 5 }