{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Training: Hybrid case" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "In this tutorial we train a pure quantum :term:`PennyLane` :term:`model` to solve a toy problem: classifying whether a given sentence is about cooking or computing. We also train a hybrid model that determines whether a given pair of sentences are talking about different topics.\n", "\n", "We use an :py:class:`.IQPAnsatz` to convert :term:`string diagrams ` into :term:`quantum circuits `. When passing these circuits to the :py:class:`PennyLaneModel`, they are automatically converted into :term:`PennyLane` circuits.\n", "\n", ":download:`Download code <../_code/trainer-hybrid.ipynb>`" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Preparation\n", "\n", "We start by specifying some training hyperparameters and importing NumPy and PyTorch." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "BATCH_SIZE = 10\n", "EPOCHS = 15\n", "LEARNING_RATE = 0.1\n", "SEED = 42" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import torch\n", "import random\n", "import numpy as np\n", "\n", "torch.manual_seed(SEED)\n", "random.seed(SEED)\n", "np.random.seed(SEED)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Input data\n", "\n", "Let's read the data and print some example sentences." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "def read_data(filename):\n", " labels, sentences = [], []\n", " with open(filename) as f:\n", " for line in f:\n", " t = float(line[0])\n", " labels.append([t, 1-t])\n", " sentences.append(line[1:].strip())\n", " return labels, sentences\n", "\n", "\n", "train_labels, train_data = read_data('../examples/datasets/mc_train_data.txt')\n", "dev_labels, dev_data = read_data('../examples/datasets/mc_dev_data.txt')\n", "test_labels, test_data = read_data('../examples/datasets/mc_test_data.txt')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "import os\n", "\n", "TESTING = int(os.environ.get('TEST_NOTEBOOKS', '0'))\n", "\n", "if TESTING:\n", " train_labels, train_data = train_labels[:2], train_data[:2]\n", " dev_labels, dev_data = dev_labels[:2], dev_data[:2]\n", " test_labels, test_data = test_labels[:2], test_data[:2]\n", " EPOCHS = 1" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['skillful man prepares sauce .',\n", " 'skillful man bakes dinner .',\n", " 'woman cooks tasty meal .',\n", " 'man prepares meal .',\n", " 'skillful woman debugs program .']" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_data[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Targets are represented as 2-dimensional arrays:" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[[1.0, 0.0], [1.0, 0.0], [1.0, 0.0], [1.0, 0.0], [0.0, 1.0]]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "train_labels[:5]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Creating and parameterising diagrams" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "The first step is to convert the sentences into :term:`string diagrams `." ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Tagging sentences.\n", "Parsing tagged sentences.\n", "Turning parse trees to diagrams.\n", "Tagging sentences.\n", "Parsing tagged sentences.\n", "Turning parse trees to diagrams.\n", "Tagging sentences.\n", "Parsing tagged sentences.\n", "Turning parse trees to diagrams.\n" ] } ], "source": [ "from lambeq import BobcatParser\n", "\n", "reader = BobcatParser(verbose='text')\n", "\n", "raw_train_diagrams = reader.sentences2diagrams(train_data)\n", "raw_dev_diagrams = reader.sentences2diagrams(dev_data)\n", "raw_test_diagrams = reader.sentences2diagrams(test_data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Simplify diagrams" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "We simplify the diagrams by removing cups with :py:class:`~.RemoveCupsRewriter`; this reduces the number of :term:`post-selections ` in a diagram, allowing them to be evaluated more efficiently." ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [], "source": [ "from lambeq import RemoveCupsRewriter\n", "\n", "remove_cups = RemoveCupsRewriter()\n", "\n", "train_diagrams = [remove_cups(diagram) for diagram in raw_train_diagrams]\n", "dev_diagrams = [remove_cups(diagram) for diagram in raw_dev_diagrams]\n", "test_diagrams = [remove_cups(diagram) for diagram in raw_test_diagrams]" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "We can visualise these diagrams using :py:meth:`~lambeq.backend.grammar.Diagram.draw`." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAApQAAADCCAYAAAD3h1L/AAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAAYX0lEQVR4nO3deVAWhxnH8d8riuU2gneROPEo0VfjMZ6l0mgUre1YHGptKmI0JkYEJrFRWw9odDwiVoumjdAIYzM1mtHUNsYj4oEa8UCrUSciaphkmHjEjAKpCGz/sLyT14MoC+57fD8zzmTfd9/d58U3+/z22ZXXZhiGIQAAAKCOGlldAAAAANwbgRIAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAAAKYQKAEAAGAKgRIAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAAAKYQKAEAAGAKgRIAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAAAKYQKAEAAGAKgRIAAACmECgBAABgCoESAAAApjS2uoCGUlJSouTkZP33v/+1uhS3lpqaql69elldhtvasGGD/v73v1tdBgDAzdlsNr399ttq3bq11aXcl8dOKNevX69NmzZZXYbbKisr07/+9S+99957Vpfi1t544w19+umnVpcBAHBjX331lbZs2aKtW7daXcoDeeyEUpL8/f21ZcsWq8twS4sXL1ZeXp4SExOtLsXt/eIXv9CKFSusLgMA4KYmTpyo4uJijRs3zupSHsijAyXq5ubNm1q2bJkmT56s8PBwq8sBAMBrnT9/XuvWrVN6err8/PysLueBPPaSN+pu9erVunHjhmbPnm11KQAAeLWFCxeqRYsWmjJlitWl1IoJJZwwnQQAwDW4y3RSYkKJuzCdBADANbjLdFJiQonvYDoJAIBrcKfppMSEEt/BdBIAANfgTtNJiQkl/o/pJAAArsHdppMSE0r8H9NJAABcg7tNJyUmlBDTSQAAXIU7TiclJpQQ00kAAFyFO04nJQKl12M6CaA2FRUVlu27qqpK1dXVlu0feNxqppOzZs1yq+mkRKB8KNHR0Zo+fbpSUlL0xBNPqFWrVsrMzFRZWZkmTpyooKAgdezYUR999JGkOwfBSZMmqUOHDvLz81OXLl20cuVKp20mJCRo9OjRWrZsmdq0aaPQ0FBNmzZNt2/ffqzvjekk4F2io6OVmJioxMREhYSEKCwsTHPnzpVhGJKkJ598Um+88Ybi4+MVHBzsmJLs379fUVFR8vPzU3h4uJKSklRWVubYbs3rxo0bp4CAALVr106rV6922vfy5ctlt9sVEBCg8PBwvfLKKyotLXU8n52drWbNmmnLli16+umn1bRpUxUXF+vWrVuaMWOG2rVrp4CAAPXr10979uxxvO7zzz/Xz3/+cz3xxBMKCAhQ165dtXXr1gb8KQINw12nkxKB8qHl5OQoLCxMhw8f1vTp0zV16lTFxcVp4MCBKigo0LBhwzR+/HiVl5erurpaP/zhD7Vx40adOXNG8+bN0+9//3tt2LDBaZu7d+9WUVGRdu/erZycHGVnZys7O/uxvSemk4B3ysnJUePGjXX48GGtXLlSy5cvV1ZWluP5ZcuWqUePHjp+/Ljmzp2roqIixcTEaMyYMTp58qTee+897d+/X4mJiU7bffPNNx2vmzVrlpKTk7Vz507H840aNdKf//xnnT59Wjk5OcrNzdXrr7/utI3y8nItWbJEWVlZOn36tFq2bKnExER98sknWr9+vU6ePKm4uDjFxMSosLBQkjRt2jTdunVL+/bt06lTp7RkyRIFBgY24E8QqH/uPJ2UJBkeavny5UZQUFC9bGvw4MHGj3/8Y8dyZWWlERAQYIwfP97xWElJiSHJ+OSTT+67jWnTphljxoxxLE+YMMGIiIgwKisrHY/FxcUZY8eOrZeaH8aiRYuMJk2aGMXFxY9tn96mW7duRnJystVlAA6DBw82IiMjjerqasdjM2fONCIjIw3DMIyIiAhj9OjRTq+ZNGmSMWXKFKfH8vLyjEaNGhnffvut43UxMTFO64wdO9YYMWLEA2vZuHGjERoa6lheu3atIck4ceKE47HPP//c8PHxMb788kun1w4ZMsSYPXu2YRiGYbfbjdTU1O9974ArS0hIMFq3bm2Ul5dbXUqdMKF8SN27d3f8t4+Pj0JDQ2W32x2PtWrVSpJ0+fJlSXcuJffu3VstWrRQYGCg1qxZo+LiYqdtdu3aVT4+Po7lNm3aOF7f0JhOAt6rf//+stlsjuUBAwaosLBQVVVVkqQ+ffo4rf+f//xH2dnZCgwMdPwZPny4qqurdfHiRaftfNeAAQN09uxZx/LHH3+sIUOGqF27dgoKCtL48eN17do1lZeXO9bx9fV1Ot6eOnVKVVVV6ty5s9P+9+7dq6KiIklSUlKSFixYoEGDBmn+/Pk6efJkPfyUgMfH7aeT4tcGPbQmTZo4LdtsNqfHag7O1dXVWr9+vWbMmKH09HQNGDBAQUFBevPNN5Wfn/+923xcN6Bz7ySABwkICHBaLi0t1UsvvaSkpKR71m3fvv1DbfPSpUsaNWqUpk6dqoULF6p58+bav3+/Jk2apIqKCvn7+0uS/Pz8nMJuaWmpfHx8dOzYMacTcEmOy9qTJ0/W8OHD9eGHH2rHjh1atGiR0tPTNX369Ed634BV3PneyRoEygZw4MABDRw4UK+88orjsZozaVfAdBLwbnef3B46dEidOnW6J7DV6NWrl86cOaOOHTvWut1Dhw7dsxwZGSlJOnbsmKqrq5Wenq5Gje5cHLv7vvL76dmzp6qqqnT58mVFRUU9cL3w8HC9/PLLevnllzV79mxlZmYSKOEW3PX3Tt6NS94NoFOnTjp69Ki2b9+uc+fOae7cuTpy5IjVZTkwnQS8W3FxsV599VV99tln+sc//qGMjAwlJyc/cP2ZM2fq4MGDSkxM1IkTJ1RYWKh//vOf9/yjnAMHDmjp0qU6d+6cVq9erY0bNzq227FjR92+fVsZGRm6cOGC1q1bp7/+9a/fW2vnzp31/PPPKz4+Xps2bdLFixd1+PBhLVq0SB9++KEkKSUlRdu3b9fFixdVUFCg3bt3O4Is4Oo8YTopESgbxEsvvaTY2FiNHTtW/fr107Vr15ymlVZiOgkgPj5e3377rfr27atp06YpOTm51mbWvXt37d27V+fOnVNUVJR69uypefPmqW3btk7rvfbaazp69Kh69uypBQsWaPny5Ro+fLgkqUePHlq+fLmWLFmibt266d1339WiRYseqt61a9cqPj5er732mrp06aLRo0fryJEjjsvtVVVVmjZtmiIjIxUTE6POnTvrrbfequNPB3h8POHeyRo2w/j/Lx/zMH/60580f/583bhxw+pSXMrixYs1b948FRUVESgfA7vdriFDhmjFihVWlwJIuvN7KJ955pl6/0w++eSTSklJUUpKSr1uF/BkEydO1LZt23ThwgW3D5TcQ+lFmE4CAOAaPOXeyRpc8vYi3DsJAIBr8JR7J2swofQSTCcBSHL6ysL6dOnSpQbZLuCJPG06KTGh9BpMJwEAcA2eNp2UmFB6BaaTAAC4Bk+cTkpMKL0C00kAAFyDJ04nJSaUHo/pJAAArsFTp5MSE0qPx3QSAADX4KnTSYkJpUdjOgkAgGvw5Omk5OGBsqKiQjk5OVaXYZm8vDymky7gzJkzXv05BABIGzdu9NjppOTBgbJ3794yDEMJCQlWl2KpqVOnMp200MCBA7VmzRrt3LnT6lIAABZbsWKFR04nJQ/+Lm9JqqyslFVvr7i4WB07dtTWrVs1dOhQS2qQJB8fHzVqxK2yVjEMQ5WVlZbtPy0tTdnZ2bp48aJlNQAdOnRQQkKC5s+fb3Up8FIff/yxRo4cqfPnz6t9+/aW1dG4cWPZbDbL9t+QPHZCKd35i7NKkyZNHDXU/De8j81ms/Tv38fHx/IaAJvNJh8fHz6HsExNHmjSpAmfwwbC6AoAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAA4BElJCRo9OjRVpfhMgiUAAAAMIVACcCUiooKq0sAAFiMQGmh6OhoJSUl6fXXX1fz5s3VunVrpaamWl0WvMyjfg5rLvMsXLhQbdu2VZcuXR5fsfBY77//vux2u/z8/BQaGqqhQ4eqrKzM6rLgRejJ5hAoLZaTk6OAgADl5+dr6dKl+uMf/8jX9OGxe9TP4a5du/TZZ59p586d+ve///0YK4UnKikp0bhx4/TCCy/o7Nmz2rNnj2JjYy37pjN4L3py3Xn0N+W4g+7duzu+jqxTp05atWqVdu3apeeee87iyuBNHvVzGBAQoKysLPn6+j7OMuGhSkpKVFlZqdjYWEVEREiS7Ha7xVXBG9GT644JpcW6d+/utNymTRtdvnzZomrgrR71c2i32wmTqDc9evTQkCFDZLfbFRcXp8zMTF2/ft3qsuCF6Ml1R6C02N3fKWqz2VRdXW1RNfBWj/o5DAgIaOiS4EV8fHy0c+dOffTRR3r66aeVkZGhLl266OLFi1aXBi9DT647AiUAwHI2m02DBg1SWlqajh8/Ll9fX23evNnqsgA8JO6hBFCr+Ph4tWvXTosWLbK6FHio/Px87dq1S8OGDVPLli2Vn5+vK1euKDIy0urSAAeOhbUjUAKoVXFxsRo14mIGGk5wcLD27dunFStW6MaNG4qIiFB6erpGjBhhdWmAA8fC2hEoLbRnz557Hvvggw8eex3wbt/3Obz7+ezs7AatB94nMjJS27Zts7oMeDmOheYQtQEAAGAKgRIAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAAAKYQKAEAAGAKgRIAAACmECgBAABgCoESAAAAphAoAQAAYAqBEgAAAKYQKAEAAGBKY6sLaCiGYejEiRMyDMOS/V+/fl0xMTH6+uuvVVBQYEkNQEBAgAYOHGj5ZzAsLEzt27e3tAYA1rG6J3/99deKiYlRYWGhrl69akkNkmcfCz02UO7YsUMxMTFWl6Ft27ZZXQKgDRs2WLp/f39/nT171mMPpABqR0++w5OPhR4ZKA3DUGpqqvr27au//OUvVpcDeLWzZ8/qt7/9ra5eveqRB1EAtaMn3+Hpx0KPDJQ7duzQoUOHtG3bNvXq1cvqcgAA8Fr0ZO/gcf8op+ZMqH///ho2bJjV5QAA4LXoyd7D4yaU3z0TstlsVpcDAIDXoid7D4+aUHImBACAa6AnexePmlByJgQAgGugJ3sXj5lQciYEAIBroCd7H4+ZUHImBACAa6Anex+PmFByJuR6EhISNHr06Ac+n5qaqmeeeeaB60dHRyslJcWxXF5erjFjxig4OFg2m03ffPPN99Zw6dIl2Ww2nThx4pHrBwDUDT3ZO3nEhJIzIfczY8YMTZ8+/aHXz8nJUV5eng4ePKiwsDCFhITUed8JCQmSpOzs7DpvAwBwf97Qk1NTU7Vnzx7t2bPH6lJchtsHSs6E3FNgYKACAwMfev2ioiJFRkaqW7duDVgVAMAMerL3cvtL3jVnQqmpqR57JuTK3n//fdntdvn5+Sk0NFRDhw5VWVnZPesdOXJELVq00JIlSyTde8m7NtHR0UpPT9e+fftks9kUHR0tSbLZbPrggw+c1m3WrNl9J481l79tNptycnKUk5PjWAYA1I/66MkP6itHjhzRc88957hKNXjwYBUUFDhed7/bnL755hvZbDanSeLp06c1atQoBQcHKygoSFFRUSoqKnI8n5WVpcjISP3gBz/Qj370I7311luO52r6Rlpamvbu3etYvnTpUp3eqydx6wklZ0LWKikp0bhx47R06VL98pe/1M2bN5WXlyfDMJzWy83NVWxsrJYuXaopU6Y88n42bdqkWbNm6dNPP9WmTZvk6+v7yNsIDw9XSUmJJCk5OVmStHLlykfeDgDg/uqjJ9fWV27evKkJEyYoIyNDhmEoPT1dI0eOVGFhoYKCgh5q+19++aV+8pOfKDo6Wrm5uQoODtaBAwdUWVkpSXr33Xc1b948rVq1Sj179tTx48f14osvKiAgQBMmTHD0kWXLlungwYPatGmTJKlFixZ1er+exK0DpTfcp+HKSkpKVFlZqdjYWEVEREiS7Ha70zqbN29WfHy8srKyNHbs2Drtp3nz5vL395evr69at25dp234+Pg4Xuvn5ydJdd4WAOBe9dGTa+srzz77rNO6a9asUbNmzbR3716NGjXqoba/evVqhYSEaP369WrSpIkkqXPnzo7n58+fr/T0dMXGxkqSOnTooDNnzujtt9/WhAkTHH0jMDDQVE/yRG57yZvppPV69OihIUOGyG63Ky4uTpmZmbp+/brj+fz8fMXFxWndunV1DpMAANdXXz25tr7y1Vdf6cUXX1SnTp0UEhKi4OBglZaWqri4+KG3f+LECUVFRTnC5HeVlZWpqKhIkyZNctznHxgYqAULFjhdEsf9ue2Ekumk9Xx8fLRz504dPHhQO3bsUEZGhv7whz8oPz9fkvTUU08pNDRU77zzjn72s5/d939gM2w22z2X12/fvl2v+wAAfL/66sm19ZWpU6fq2rVrWrlypSIiItS0aVMNGDBAFRUVkqRGje7MyL7bF+7uCTVXqO6ntLRUkpSZmal+/frdUxdq55YTSqaTrsNms2nQoEFKS0vT8ePH5evrq82bN0uSwsLClJubq/Pnz+tXv/pVvYe9Fi1aOO5nkaTCwkKVl5fX6z4AALWr7578oL5y4MABJSUlaeTIkeratauaNm2qq1evOl5Xcx/jd/vC3b+HuHv37srLy7tvP2rVqpXatm2rCxcuqGPHjk5/OnToYPp9eTq3nFAynXQN+fn52rVrl4YNG6aWLVsqPz9fV65cUWRkpE6ePClJatmypXJzc/XTn/5U48aN0/r169W4cf187J599lmtWrVKAwYMUFVVlWbOnFnvU1AAQO3qsyfX1lc6deqkdevWqU+fPrpx44Z+97vfOU0c/fz81L9/fy1evFgdOnTQ5cuXNWfOHKftJyYmKiMjQ7/+9a81e/ZshYSE6NChQ+rbt6+6dOmitLQ0JSUlKSQkRDExMbp165aOHj2q69ev69VXXzX13jyd200omU66juDgYO3bt08jR45U586dNWfOHKWnp2vEiBFO67Vu3Vq5ubk6deqUnn/+eVVVVdXL/tPT0xUeHq6oqCj95je/0YwZM+Tv718v2wYAfL/67sm19ZW//e1vun79unr16qXx48crKSlJLVu2dHr9O++8o8rKSvXu3VspKSlasGCB0/OhoaHKzc1VaWmpBg8erN69eyszM9MxjJg8ebKysrK0du1a2e12DR48WNnZ2UwoH4LNuPsmNBe3fft2xcTEaNu2bRo+fLjV5cAN8U05j1dBQYF69+6tY8eOqVevXlaXAwuEh4frhRdeUFpamtWloJ55a0+uyzflePqx0K0ueTOdRH3g908CgHne3JNnzJihpKQkq8twKW4VKLl3EvXBzPeAAwDu8Oae/ChfHewt3OYeSm8+EwIAwJXQk3E3t5lQevOZEAAAroSejLu5xYSSMyEAAFwDPRn34xYTSs6EAABwDfRk3I/LTyg5EwIAwDXQk/EgLj+h5EwIAADXQE/Gg7j0hJIzIQAAXAM9GbVx6QklZ0IAALgGejJq47KBsuZM6KmnnlJYWJgKCgqsLglAHZw9e9bqEgCYRE82z9OPhS4bKCsqKvTFF1/oiy++UJ8+fawuB4AJ/v7+CgsLs7oMAHVET64fnnwsdNlA2bRpUx08eFBXrlyxuhQAJoWFhal9+/ZWlwGgjujJ9cOTj4UuGyglKTw8XOHh4VaXAQCA16MnozYu/a+8AQAA4PoIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTCJQAAAAwhUAJAAAAUwiUAAAAMIVACQAAAFMIlAAAADCFQAkAAABTGltdAADAs82ZM0fdunWzugwADchmGIZhdREAAABwX1zyBgAAgCkESgAAAJhCoAQAAIApBEoAAACYQqAEAACAKQRKAAAAmEKgBAAAgCkESgAAAJhCoAQAAIApBEoAAACYQqAEAACAKQRKAAAAmEKgBAAAgCkESgAAAJhCoAQAAIApBEoAAACYQqAEAACAKQRKAAAAmEKgBAAAgCkESgAAAJhCoAQAAIApBEoAAACYQqAEAACAKf8DzqvGmn6/F1IAAAAASUVORK5CYII=", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "train_diagrams[0].draw()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create circuits" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "In order to run the experiments on a quantum computer, we apply a quantum :term:`ansatz ` to the string diagrams. For this experiment, we will use an :py:class:`.IQPAnsatz`, where noun wires (``n``) and sentence wires (``s``) are represented by one-qubit systems." ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA0MAAAMzCAYAAABk4skuAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAAC4AElEQVR4nOzde1zO9/8/8MfVSV1FoVQOCRmXUyqiyJIsp4zFDCGzjY01I7PNRuNjMxPmy7CPtozmsI0ktsmhJoeiNEyOQ5tdbFGmouPr94df18clhw7X1bur9+N+u3XT9T683s/3W8/r/X6+D6+3QgghQEREREREJDNGUgdAREREREQkBRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDelZSUoKcnBypwyCqUwoKCpCbmyt1GER1Sm5uLgoKCqQOg6hOycnJQUlJidRh0BOwGNKj3NxcDB8+HO7u7jhz5ozU4RDVCVlZWfD390fv3r2RmZkpdThEdUJmZiZ69eqF/v37IysrS+pwiOqEM2fOwN3dHS+88AJP4NViLIb0JDMzE71790ZCQgLMzMzg5eWFn3/+WeqwiAzamTNn4OnpiXPnziEnJweenp5ITk6WOiwig5acnAxPT0/cvn0bZ8+eRY8ePZCRkSF1WEQG7eeff4aXlxfq1auHAwcOoHfv3vjjjz+kDosegcWQHpTtWHJycnD48GGkpKSgd+/eGDx4MFatWiV1eEQGqWzHYmlpiZSUFKSkpKBNmzbw9fXFli1bpA6PyCBt3rwZzz77LNq0aaPJKwsLC3h5eWHPnj1Sh0dkkFauXIlBgwbBx8cHKSkpOHTokOYEXkpKitTh0cME6dTmzZuFubm58Pb2Fjdu3NAMLy4uFtOnTxcAxLRp00RRUZGEURIZlpUrVwpjY2MxaNAgcfv2bc3wu3fvirFjxwoA4qOPPhKlpaUSRklkOEpLS0V4eLgAIIKDg8Xdu3c1427fvi0GDhwojI2NxapVqySMksiwFBUVialTpwoA4u233xbFxcWacTdu3BBeXl7C3NxcbNmyRcIo6WEshnSktLRUfPTRRwKAGDt2rNaO5UFr1qwRxsbGIiAgQOTk5NRwlESGpaioSEybNk0AENOnT9fasZQpLS0VCxYsEADE6NGjH5t7RHRffn6+GD16tAAg/vOf/zzyJEJRUZF46623BADx5ptv8gQe0VPk5OSI5557TpiYmIi1a9c+cpq7d++KMWPGCABi/vz5PIFXS7AY0oG7d+9qdiwLFix46h93fHy8sLGxER06dBCXLl2qoSiJDEtOTo4ICAgQxsbGYs2aNU+dfuvWrcLc3Fz07NlTXL9+vQYiJDI8arVa9OjRQ1hYWIjvvvvuqdOvXr1aGBsbiwEDBvAEHtFjXLp0SahUKmFjYyP27t37xGkfPIE3ZswYnsCrBRRCCCHNDXp1w40bNzB8+HCcOHEC33zzDUaOHFmh+c6ePYshQ4bg9u3b2L59O3r37q3nSIkMx+XLlzFkyBBcu3YN33//Pfz9/Ss037FjxzB06FCYmZkhLi4OnTt31nOkRIbj5MmTCAwMRGFhIWJjY9G9e/cKzbd3716MGDECzZs3x86dO9GqVSs9R0pkOJKSkjB8+HDY2NggLi4O7dq1q9B8W7duxYQJE+Dm5oaYmBg0adJEz5HS47ADhWo4deoUPD09cfnyZSQmJla4EAKA9u3bIzk5GR06dEC/fv2wYcMGPUZKZDiSkpLg6emJgoICHD16tMKFEAB0794dKSkpaNiwIby9vbFr1y49RkpkOHbt2oVevXqhUaNGSElJqXAhBAD+/v44evQo7t27B09PTxw6dEiPkRIZjm+++Qb9+vVDx44dcfTo0QoXQgDw4osvIjExEb///js8PT1x+vRpPUZKT8JiqIp27doFb29vNGzYECkpKfD09Kx0G40bN0Z8fDyCg4Mxfvx4zJkzB6WlpXqIlsgwbNiwAf369UOHDh2QnJyM9u3bV7qNFi1aICkpCX5+fhg6dCiWL18OXgAnuRJCYNmyZRg6dCj69euHgwcPokWLFpVu58ETeH5+fti4caMeoiUyDKWlpXj//fcxYcIEBAcHY8+ePWjcuHGl2ynrXc7Gxgbe3t7YvXu3HqKlp5L2Lj3DU1paKpYtWyaMjIzE0KFDxZ07d3TS5uLFi4VCoRBBQUEiLy9PB5ESGY6SkhLx/vvvCwDi5ZdfFgUFBdVus7i4WMyaNUsAEJMnTxaFhYU6iJTIcBQWForXXntNABDvvPOOKCkpqXabBQUFYuLEiQKAmDNnjk7aJDIkubm54oUXXhAKhUJ89tlnOukE4c6dO2Lo0KHCyMhILF++nB0r1DAWQ5VQWFgoJk+eLACIWbNmPbJnq+qIiYkRSqVSeHh4iGvXrum0baLaKi8vTwQFBQmFQiEWL16s851AZGSkMDExEf369RO3bt3SadtEtdWtW7eEn5+fMDU1FV999ZVO2y4tLRWffvqpUCgUYsSIETyBR7Lx559/Cnd3d2FpaSl27Nih07aLi4tFWFiYACCmTJnCE3g1iMVQBd26dUv069dPmJiYiMjISL0tJy0tTTRv3lw0a9ZMpKam6m05RLXBtWvXhIeHh1AqlSImJkZvyzlw4IBo1KiReOaZZ8T58+f1thyi2uD8+fPimWeeEY0aNRIJCQl6W8727duFUqkU3bp14wk8qvNSU1NF06ZNRfPmzcWJEyf0tpx169bxBF4N4zNDFXDhwgX07NkTJ06cQHx8PF5++WW9LcvNzQ0pKSlo2rQpfHx8sH37dr0ti0hKaWlp8PT0xI0bN5CUlITnn39eb8vy9fVFcnIyFAoFevTogYSEBL0ti0hKCQkJ6NGjBxQKBZKTk/Hss8/qbVnDhg1DUlIS1Go1PD09ceLECb0ti0hK27Ztg4+PD5o1a4aUlBR07dpVb8uaNGkS4uPjceLECXh5eeHixYt6Wxbdx2LoKR7esfj6+up9mY6OjkhISMDgwYPxwgsv4NNPP+UD4FSnbN++HT4+PmjatClSUlLg5uam92W6uLjgyJEjcHd3R//+/REZGan3ZRLVpHXr1qF///7w8PDAkSNH4OLiovdllp3Ac3R0RO/evRETE6P3ZRLVFCEEFi1ahKCgIAwZMgSJiYlwdHTU+3J9fX1x9OhRCCHQo0cPJCYm6n2ZsibxlalaTReXKleuXClatmwp6tWrJzw9PUVycnKF5y0pKREffvihACAmTJgg7t27V6UYiGqL0tJSsWjRIgFAjBw5ssrPGlQnrx589i8sLEznz/4R1bTi4mIxc+bMaj1rUJ2cEuL+s38jRowQCoVCLFq0iA+Ak8G7d++eGD9+vAAg5s6dW6XOQqqbVw8++6fPRzTkjsXQI+iqF6rNmzcLMzMz8dVXX4nffvtNvPrqq8LGxkbcuHGjUu1s3LhRmJmZCR8fH/HPP/9UKRYiqRUUFIiQkBABQHz44YdV7oVKF3lVWloqli9frtNeIYmkcOfOHREYGCiMjIzE559/XqUiRFf7qpKSEvHBBx8IAGLixIk66RWSSAr//POP6N27t6hXr57YuHFjldrQVV492CvkrFmz2IOjHrAYesidO3fE888/r5PuDT09PcXUqVM1n0tKSkTTpk3FJ598Uum2Dh06JOzs7ETr1q3FmTNnqhwTkRT++ecf0adPH2FmZlblHUsZXebVrl27RP369YWrq6vIzMysVlxENe3q1avC1dVV1K9fX+zevbvK7egyp4QQYsOGDcLMzEz06dOHJ/DI4Jw5c0a0bt1aNGnSRBw+fLjK7egyrx58rcvzzz/PE3g6xmeGHvDHH3+gd+/e2L9/P3bu3Im33noLCoWiSm0VFhYiNTUV/v7+mmFGRkbw9/fHkSNHKt2et7c3UlJSYGFhAS8vL8THx1cpLqKalpGRgR49eiAjIwMHDhzA2LFjq9yWrvNq0KBBOHToEHJycjQvvyMyBMnJyfD09EROTg4OHz6MgQMHVqkdXecUAAQHB2P//v3IyMhAz549cfbs2Sq1Q1TT9uzZAy8vLyiVSiQnJ8PLy6tK7eg6rxQKBaZPn47Y2Fjs27cPPj4++PPPP6sUG5XHYuj/O3bsmGbHcujQIQwaNKha7WVlZaGkpAT29vZaw+3t7XH9+vUqtens7IzDhw/D29sbAwcOxOrVq6sVI5G+xcfHw8vLCxYWFkhJSYG3t3e12tNHXnXu3BkpKSlo1aoVnn32WWzdurVaMRLp25YtW+Dr64s2bdogJSUFnTp1qnJb+sgpAOjVqxdSUlJgbm6Onj17Yu/evVVui6gmfPHFFxg0aBB69eqFQ4cOwdnZucpt6SuvBg8ejMOHDyM7Oxuenp44duxYldui/2ExBGDr1q3o06cPnJ2dkZycjM6dO0sd0mM1aNAAsbGxmDZtGt544w2EhoaiuLhY6rCIylm9ejUGDhwIb29vHD58uFo7Fn1r0qQJ9u/fjxdeeAGjRo3CggUL2IMj1TpCCMyfPx8vvfQSgoKCsG/fPjRp0kTqsB6r7ASel5cXBgwYgDVr1kgdElE5xcXFCA0NxdSpU/Hmm28iNjYWDRo0kDqsx+rcuTOSk5PRsmVL9OnTB999953UIRk8WRdDQgj85z//wahRozB8+HAcOHCgXBVfVba2tjA2NsaNGze0ht+4cQMODg7VatvExATLly/HF198gS+++AKBgYG4fft2tdok0pWyHcsbb7yBqVOn6nTHos+8Mjc3x8aNGzF//nzMnTsXwcHBuHfvXrXaJNKVe/fuYezYsZg3bx4WLFiADRs2wNzcvNrt6jOngPsn8Hbu3ImpU6fi9ddfx1tvvcUTeFRr3L59G0OGDMEXX3yB1atXY9myZTA2Nq52u/rOK3t7exw4cADDhw/Hiy++iP/85z88gVcd0j6yJJ27d++KsWPHCgBi/vz5eukG1NPTU0ybNk3zuaSkRDRr1qzKD6U+yp49e4S1tbXo2LGj+P3333XWLlFV5OTkiAEDBghjY2PxxRdf6GUZNZFXW7ZsEebm5sLLy6vSPf8Q6dr169eFl5eXMDc3F1u2bNF5+zWRU0IIsWrVKmFsbCwGDhwobt++rdO2iSrr999/Fx06dBDW1tYiPj5e5+3XRF6VlpaKjz76SAAQY8eOFXfv3tVZ23Iiy2Loxo0bet2xlNm8ebOoV6+eiIqKEmfOnBGvvfaasLGxEdevX9fpcjIyMkSbNm2Era2tSEpK0mnbRBX1+++/i44dOwpra2uxZ88evS2npvIqOTlZ2Nvbi5YtW4pTp07ptG2iijp58qRwcnISDg4OlX5HSUXVVE4JwRN4VDscPHhQ2NraijZt2oiMjAy9LKMm82rz5s3C3NxceHt78wReFciuGDp16pRo2bKlsLe319uO5UH/93//J5ycnISZmZnw9PQUR48e1ctysrKyNF0Xb9iwQS/LIHqcpKQkve9YHlRTefVg18W7du3SyzKIHicuLk5YWVnVSNfvNZVTQvyv62I7Oztx6NAhvS2H6FG++eYbTdfvWVlZel1WTebV0aNHhb29vXB2duYJvEqSVTH04DtFrl69KnU4OldQUCAmTpwoAIg5c+bwxVxUIx58p4i+dyxSuHPnjhg6dKhO3j1GVBEPvlOkrr4U+J9//hE+Pj46efcYUUWUlJSIOXPmCADi5ZdfrpMvBb569aro0qVLtd89JjeyKIZKS0vF559/Xqd3LGVKS0vFp59+KhQKhRgxYoTIy8uTOiSqo+T0tvni4mIRFhYmAIgpU6aIwsJCqUOiOqqwsFBMnjxZ87b54uJiqUPSm3v37omQkBABQHz44Yc8gUd6k5eXJ0aMGCEUCoVYvHhxnT6p9e+//4rAwEBhZGQkVqxYUafXVVfqfDFUWFgopkyZIgCIsLCwOr1jedD27duFUqkU3bp1E3/99ZfU4VAd8+CO5dNPP5XNl+26deuEiYmJ6Nevn7h165bU4VAdc+vWLdGvXz9hYmIiIiMjpQ6nRjx4Am/kyJE8gUc6d+3aNdGtWzehVCpFTEyM1OHUiOLiYjFz5kwBQLz++us8gfcUdboYys7OFv7+/sLExESsW7dO6nBqXFpammjWrJlo3ry5SEtLkzocqiP++usvzY5l+/btUodT4w4cOCAaNWok2rVrJy5cuCB1OFRHnD9/XjzzzDOiUaNG4sCBA1KHU+O2bdsmlEql6N69O0/gkc7I/Tjov//9rzAxMRH9+/cX2dnZUodTa9XZ9wxdvHgRXl5eSEtLQ3x8PCZNmiR1SDXOzc0NKSkpcHBwQO/evRETEyN1SGTgTpw4AU9PT6jVaiQlJWHYsGFSh1TjfH19cfToUQgh0KNHDyQmJkodEhm4hIQE9OjRAwqFAsnJyfD19ZU6pBo3fPhwJCUl4a+//oKnpyfS09OlDokMXExMDHr37o2mTZsiJSUFbm5uUodU41555RXs2bMHx48fh5eXFy5duiR1SLVSnSyGEhMT0aNHD5SWluLo0aOy3LGUadq0KRITEzFo0CC88MILWLx4MV/MRVVStmNxcHCQ7Y6lTNu2bXH06FF07doV/fv3x1dffSV1SGSgIiMj0b9/f7i7u+PIkSNwcXGROiTJlJ3As7e3R+/evbFjxw6pQyIDJITAp59+ihdeeAGDBw9GQkICHB0dpQ5LMn379kVycjJKS0vRo0cP/PLLL1KHVPtIfGVK57766ithamoq/Pz8eE//A+T0sDvpFjvleLzCwkLx2muvyeJhd9KtBzvlmDx5Mu/pf4CcHnYn3SooKGCnHI9x69Yt4efnJ0xNTcXXX38tdTi1Sp0phkpKSsQ777wjAIjXXnuNO5bHeLAb5H/++UfqcKiWe7C79g8++IA7lkco6wZZoVCI559/vk73Vkm6we7an04O3SCTbv3zzz+a9y2yu/ZHKywsFK+++qoAIGbPns19+v9XJ4qhO3fuiOeff14oFAqxdOlS7lieoqZfkEmG6cEdyzfffCN1OLXezp07a+wFmWS4MjMzhaurq7CyshJxcXFSh1Prlb0g89lnn62T7zEj3XjwRb5JSUlSh1OrlZaWiqVLlwqFQiGGDRsmcnNzpQ5JcgohDPsBkj///BOBgYG4ePEiNm3ahCFDhkgdkkG4fPkyhgwZgmvXruH777+Hv7+/1CFRLXL27FkMGTIEt2/fRkxMDHr16iV1SAbh5MmTCAwMRGFhIWJjY9G9e3epQ6JaJCUlBc8//zzq1auHnTt3onPnzlKHZBCSkpIwfPhwWFtbY9euXWjXrp3UIVEtEh8fj5EjR6J58+aIi4uDs7Oz1CEZhJ07d2LMmDFo27YtYmNj0bx5c6lDkoxBd6Bw/PhxeHp64tatWzh06BALoUpo1aoVDh8+DC8vLwwYMABr1qyROiSqJfbu3YuePXuiXr16SElJYSFUCV26dEFKSgqcnZ3Rp08ffPfdd1KHRLXE1q1b8eyzz8LZ2RnJyckshCqhd+/eSE5ORr169dCzZ0/s27dP6pCollizZg0GDhwIb29vHD58mIVQJQQGBuLQoUO4efMmPD09cfz4calDko7Ul6aq6rvvvhMWFhaiZ8+e4vr161KHY7CKiopEaGioACDeeustPgAuc6tXrxbGxsZiwIABIicnR+pwDNbdu3fF6NGjBQCxYMEC3rorY6WlpWLBggUCgBg9erS4e/eu1CEZrJycHBEQECCMjY3FmjVrpA6HJPTgsUtoaKgoKiqSOiSDpVarRY8ePYSFhYX4/vvvpQ5HEgZ5m1xxcTGaNGmC7OxsAIBCoZA4IsNX9meQlJTEKwEylZOTA1tbW5SUlABgXlXXg1+tFy9eRJs2bSSMhqRy6dIlre6ymVfVU5ZXJiYmyMrKgrW1tcQRkRSSkpLg4+MDgDmlC2V51bBhQ/z9998wMTGROKKaZZBra2JigujoaPzxxx9Sh/JU3333HU6fPo2PPvpI6lCeSqFQwMvLS+owSCI2NjbYsmULbt68KXUoTxUZGYnbt29jxowZUofyVEqlkoWQjLVp0wbffPMN7t69K3UoT7V06VJYW1sbxEvKGzduzEJIxry9vfHll18axHsT582bh06dOmHkyJFSh/JULVq0kF0hBAAGeWXIkMycORO7d+9GRkaG1KEQ1Rljx46FWq3G/v37pQ6FqM7w8/ODo6MjoqOjpQ6FqM5QqVQYNGgQIiIipA6FHsOgO1AgIiIiIiKqKhZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiqA65cuUKFAoF0tPTHztNQkICFAoFcnJyaiwuIkPGvCLSPeYVke4xr6qGxZDMeHt7Q61Wa15WFxUVBRsbG2mDIjJwzCsi3WNeEeke86o8+b1mVubMzMzg4OAgdRhEdQrzikj3mFdEuse8Ko9XhmqRvLw8jB8/HlZWVnB0dERERAR8fX0xffp0AIBCoUBMTIzWPDY2NoiKitIadvbsWXh7e8Pc3BydOnVCYmKiZtyDl0cTEhIwceJE3L59GwqFAgqFAuHh4fpdSaIaxrwi0j3mFZHuMa+kwWKoFpk1axYSExOxY8cO7NmzBwkJCUhLS6tSOzNnzsSJEyfg5eWFwMBA3Lx5s9x03t7eWL58ORo0aAC1Wg21Wo2wsDBdrApRrcG8ItI95hWR7jGvpMFiqJbIzc1FZGQklixZgn79+qFz585Yv349iouLK93WtGnTEBQUBJVKhdWrV8Pa2hqRkZHlpjMzM4O1tTUUCgUcHBzg4OAAKysrXawOUa3AvCLSPeYVke4xr6TDYqiWuHTpEgoLC9GjRw/NsEaNGqFdu3aVbsvLy0vzu4mJCbp164aMjAydxElkSJhXRLrHvCLSPeaVdFgMGRCFQgEhhNawoqIiiaIhqhuYV0S6x7wi0j3mlX6wGKol2rRpA1NTUyQnJ2uGZWdn4/z585rPdnZ2UKvVms8XLlxAfn5+ubaOHj2q+b24uBipqalQqVSPXK6ZmRlKSkp0sQpEtQ7zikj3mFdEuse8kg671q4lrKysMGnSJMyaNQuNGzdGkyZNMGfOHBgZ/a9e9fPzw8qVK+Hl5YWSkhLMnj0bpqam5dpatWoV2rZtC5VKhWXLliE7Oxsvv/zyI5fr7OyM3Nxc7Nu3D66urlAqlVAqlXpbT6KaxLwi0j3mFZHuMa+kwytDtchnn30GHx8fBAYGwt/fH71794aHh4dmfEREBFq0aAEfHx+MGTMGYWFhj/yDXbRoERYtWgRXV1ckJSUhNjYWtra2j1ymt7c3pkyZglGjRsHOzg6LFy/W2/oRSYF5RaR7zCsi3WNeSUMhHr75kHRq5syZ2L17d5UfXPP19UXXrl2xfPly3QZGZMDGjh0LtVqN/fv3V2l+5hVReX5+fnB0dER0dHSV5mdeEZWnUqkwaNAgREREVGl+5pX+8coQERERERHJEoshIiIiIiKSJXagUMslJCRIHQJRncO8ItI95hWR7jGv9I9XhoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyZCJ1AFWVmZmJrKwsqcN4KjMzM7Rt2xZpaWlSh1Ihtra2cHJykjoMkoih5JWNjQ0UCgXzigyCoeRV06ZNYW1tzbwig2AoedW2bVuYmZkxr2oxhRBCSB1EZWVmZkKlUiE/P1/qUOocpVKJjIwM2SUCMa/0iXklX8wr/WFeyRfzSn/kmFcGeWUoKysL+fn52LhxI1QqldTh1BkZGRkIDg5GVlaWrJKA7mNe6QfzSt6YV/rBvJI35pV+yDWvDLIYKqNSqeDu7i51GER1CvOKSPeYV0S6x7wiXWAHCkREREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLsiqGzp07BwcHB9y5c0fqUGq1IUOG4Pjx41KHQURERESkVwZRDIWEhEChUEChUMDU1BSBgYEAgIKCgkq189577+HNN99E/fr19RGmwVi1ahWcnZ1hbm6OHj16ICUlRWv8uHHjMHv2bImiI3q0kJAQDBs2rNzwhIQEKBQK5OTk1HhMRIaOeUWke8wrw2IQxRAADBgwAGq1Gr///jtmzJgBAFi7dm2F58/MzERcXBxCQkL0FKFh2LJlC2bMmIF58+YhLS0Nrq6uCAgIwN9//62ZZuDAgUhKSsJvv/0mYaRERERERPplMMVQvXr14ODggBYtWqBv374AgKNHjwIAvvnmG1hZWeHChQua6d944w20b98e+fn5AICtW7fC1dUVzZo100wTFRUFGxsbxMXFoV27dlAqlRgxYgTy8/Oxfv16ODs7o2HDhggNDUVJSYlmvg0bNqBbt26oX78+HBwcMGbMGK1ioqzy37dvH7p16walUglvb2+cO3eu2tshPDwcXbt2xYYNG+Ds7Axra2u89NJLFb71b+nSpXj11VcxceJEdOjQAWvWrIFSqcRXX32lmaZBgwbo1asXNm/eXO14iYiIiIhqK4Mphh508eJFAICpqSkAYPz48Rg0aBDGjh2L4uJi7Nq1C+vWrUN0dDSUSiUA4ODBg+jWrVu5tvLz87FixQps3rwZP/30ExISEjB8+HDs3r0bu3fvxoYNG7B27Vp8//33mnmKioqwYMEC/Prrr4iJicGVK1ceecVpzpw5iIiIwPHjx2FiYoKXX35ZJ+t/6dIlxMTEIC4uDnFxcUhMTMSiRYueOl9hYSFSU1Ph7++vGWZkZAR/f38cOXJEa1pPT08cPHhQJ/ESEREREdVGJlIHUFFxcXGwsrJCcXGx5lmh8ePHa8avXbsWXbp0QWhoKLZt24bw8HB4eHhoxl+9evWRxVBRURFWr16NNm3aAABGjBiBDRs24MaNG7CyskKHDh3Qt29fHDhwAKNGjQIAraKmdevWWLFiBbp3747c3FxYWVlpxi1cuBDPPvssAODdd9/F4MGDce/ePZibm1drW5SWliIqKkrz7NO4ceOwb98+LFy48InzZWVloaSkBPb29lrD7e3tcfbsWa1hTZs2xdWrV6sVJ5GulX0PPOjBq7ZEVHnMKyLdY14ZDoO5MtS3b1+kp6cjOTkZQ4YMAQD069dPM75hw4aIjIzUFDbvvvuu1vx37959ZBGiVCo1hRBwvzBwdnbW+gO2t7fXug0uNTUVgYGBcHJyQv369TUFT2ZmplbbXbp00fzu6OgIAFrtVJWzs7NWJxCOjo46afdBFhYWmlsMiWqLsu+BB3/WrVsndVhEBo15RaR7zCvDYTBXhiwtLeHi4gIAmDdvHuLi4hATEwN3d3fNNL/88guMjY2hVquRl5enVTDY2toiOzu7XLtlt9qVKeux7uFhpaWlAIC8vDwEBAQgICAA0dHRsLOzQ2ZmJgICAlBYWPjYthUKBQBo2qmOJ8X3JLa2tjA2NsaNGze0ht+4cQMODg5aw27dugU7O7tqx0qkSw9+D5T5888/JYqGqG5gXhHpHvPKcBjMlaEHGRndD/uLL77A3bt3AQCHDx/Gp59+ip07d8LKygrTpk3TmsfNzQ1nzpyp9rLPnj2LmzdvYtGiRfDx8UH79u11flVGX8zMzODh4YF9+/ZphpWWlmLfvn3w8vLSmvb06dNwc3Or6RCJiIiIiGqMQRZDZYyNjbFq1SrcuXMH48aNQ2hoKAYOHIjo6Ghs2bJFq9ODgIAAHDlypNr3azo5OcHMzAz/93//h99//x2xsbFYsGBBdVelxsyYMQP//e9/sX79emRkZOD1119HXl4eJk6cqDXdwYMH8dxzz0kUJRERERGR/hl0MfTiiy9i8eLFCA0NhaWlJT7++GMAQOfOnfHxxx9j8uTJuHbtGoD7784xMTHB3r17q7VMOzs7REVF4bvvvkOHDh2waNEiLFmypNrrUlNGjRqFJUuWYO7cuejatSvS09Px008/aXWqcPLkSdy+fRsjRoyQMFIiIiIiIv1SCCGE1EFUVlpaGjw8PJCamqr1zNDTrFq1CrGxsfj555/1GJ3hKtuu/fv3h6+vL95//32pQ6IaVNW8oifjdpU3/v/rB7ervPH/Xz/kul0NpgMFXZg8eTJycnJw584drc4VSJuLiwvefvttqcMgIiIiItIrWRVDJiYmmDNnjtRhoGPHjo99h4+dnR3++eefR45bu3Ytxo4d+9h2MzMz0aFDh8eOP3PmDJycnJ4a3yuvvAILC4unTkdEREREZMhkVQzVFrt370ZRUdEjx5mamj523MMvS31Y06ZNkZ6e/sTxRERERER0H4shCbRs2VIv7ZqYmJTr056IiIiIiB7NoHuTIyIiIiIiqioWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGTJoN8zlJGRIXUIdQq3JwH8O9A1bk8C+Hega9yeBPDvQNfkuj0NshiytbWFUqlEcHCw1KHUOUqlEra2tlKHQRJgXukP80q+mFf6w7ySL+aV/sgxrxRCCCF1EFWRmZmJrKwsqcN4qqVLl+LQoUP44YcfpA6lQmxtbeHk5CR1GCQRQ8mrOXPmICsrC2vXrpU6lAphXsmboeTV5MmTYWtri4ULF0odSoUwr+TNUPIqKCgIvXr1wowZM6QOpULkmFcGeWUIAJycnAziP8ve3h7m5uZwd3eXOhSipzKUvGrUqBEKCgqYV2QQDCWv6tevj0aNGjGvyCAYSl6Zm5vD3t6eeVWLsQMFIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxVIdcuXIFCoUC6enpj50mISEBCoUCOTk5NRYXkSFjXhHpHvOKSPeYV1XDYkhmvL29oVarYW1tDQCIioqCjY2NtEERGTjmFZHuMa+IdI95VZ7BvmeIqsbMzAwODg5Sh0FUpzCviHSPeUWke8yr8nhlqBbJy8vD+PHjYWVlBUdHR0RERMDX1xfTp08HACgUCsTExGjNY2Njg6ioKK1hZ8+ehbe3N8zNzdGpUyckJiZqxj14eTQhIQETJ07E7du3oVAooFAoEB4ert+VJKphzCsi3WNeEeke80oaLIZqkVmzZiExMRE7duzAnj17kJCQgLS0tCq1M3PmTJw4cQJeXl4IDAzEzZs3y03n7e2N5cuXo0GDBlCr1VCr1QgLC9PFqhDVGswrIt1jXhHpHvNKGiyGaonc3FxERkZiyZIl6NevHzp37oz169ejuLi40m1NmzYNQUFBUKlUWL16NaytrREZGVluOjMzM1hbW0OhUMDBwQEODg6wsrLSxeoQ1QrMKyLdY14R6R7zSjoshmqJS5cuobCwED169NAMa9SoEdq1a1fptry8vDS/m5iYoFu3bsjIyNBJnESGhHlFpHvMKyLdY15Jh8WQAVEoFBBCaA0rKiqSKBqiuoF5RaR7zCsi3WNe6QeLoVqiTZs2MDU1RXJysmZYdnY2zp8/r/lsZ2cHtVqt+XzhwgXk5+eXa+vo0aOa34uLi5GamgqVSvXI5ZqZmaGkpEQXq0BU6zCviHSPeUWke8wr6bBr7VrCysoKkyZNwqxZs9C4cWM0adIEc+bMgZHR/+pVPz8/rFy5El5eXigpKcHs2bNhamparq1Vq1ahbdu2UKlUWLZsGbKzs/Hyyy8/crnOzs7Izc3Fvn374OrqCqVSCaVSqbf1JKpJzCsi3WNeEeke80o6vDJUi3z22Wfw8fFBYGAg/P390bt3b3h4eGjGR0REoEWLFvDx8cGYMWMQFhb2yD/YRYsWYdGiRXB1dUVSUhJiY2Nha2v7yGV6e3tjypQpGDVqFOzs7LB48WK9rR+RFJhXRLrHvCLSPeaVNBTi4ZsPSadmzpyJ3bt3V/nBNV9fX3Tt2hXLly/XbWBEBmzs2LFQq9XYv39/leZnXhGV5+fnB0dHR0RHR1dpfuYVUXkqlQqDBg1CREREleZnXukfrwwREREREZEssRgiIiIiIiJZYgcKtVxCQoLUIRDVOcwrIt1jXhHpHvNK/3hliIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLJlIHUFWZmZnIysqSOoynKi0thYODA9LS0qQOpUJsbW3h5OQkdRhET2RtbY2ioiKpwyCqEEPZX1lbW8PMzIz7KyKSFYMshjIzM6FSqZCfny91KBXm4eEhdQgVolQqkZGRwR0M1Wq3b982iINLIkPcX0VFRUkdQoVwf0VEumCQxVBWVhby8/OxceNGqFQqqcOpMzIyMhAcHIysrCzuXIiIdID7K/3g/oqIdMUgi6EyKpUK7u7uUodBRET0RNxfERHVTuxAgYiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMUREBiU9PR0XL15EZmYmvvzyS6Snp0sdEhERERkoFkNEVOuVlJRg48aN8PT0hJubG1JSUnDp0iVMnjwZbm5u8PT0xMaNG1FSUiJ1qERERGRAZFUM3bx5E02aNMGVK1ekDqVWe++99xARESF1GEQAgLt37yIoKAjjxo1DamrqI6dJTU3FuHHjMGLECNy9e7eGI6Tait/5pCvOzs5Yvny5Tto6c+YMmjdvjry8PJ20R0TVYzDFUEhICBQKBRQKBTw9PQEAn3/+Oe7du1fhNhYuXIjnn38ezs7Oeoqy9vvtt98QFBQEZ2dnKBSKR365T5o0CQsXLsTt27drPkCiB5SUlGD06NHYuXMnAKC0tPSR05UNj42NxZgxY3iFqA548Dvf1NQUrVq1wjvvvMPvfAOXm5uLiIgI9O7dGw4ODmjWrBn8/Pywdu1aFBcX15o2q+PevXuYOnUqGjduDCsrKwQFBeHGjRua8R06dEDPnj2xdOnSGo+NiMozmGIIAAYMGAC1Wo3Y2FgAwA8//IB58+ZVaN78/HxERkZi0qRJ+gyx1svPz0fr1q2xaNEiODg4PHIaFxcXtGnTBhs3bqzh6Ii0bdq0CTt27HhsEfSw0tJSxMTEYNOmTXqOjGpC2Xf+77//jmXLlmHt2rU1/p1fUlJS4b8/fSgsLJRs2brwYPypqano0KEDYmJi8OqrryI2NhZxcXGYMGECoqKi0L17d/z999+Vaj8jI0PnbVbX22+/jZ07d+K7775DYmIi/vrrL7zwwgta00ycOBGrV6+WpFgjoocIAzFhwgTx/PPPCyGESE1NFQCEn5+fcHNzE0IIMW/ePAGg3M/XX38thBDiu+++E3Z2dlptHjhwQAAQP/30k+jataswNzcXffv2FTdu3BC7d+8W7du3F/Xr1xejR48WeXl5mvl+/PFH0atXL2FtbS0aNWokBg8eLC5evKgZf/nyZQFA/PDDD8LX11dYWFiILl26iMOHD1d7O3z99dfC2tpa/PTTT6J9+/bC0tJSBAQEiL/++qvSbbVs2VIsW7ZM87lsu6ampoqPPvpI9O7du9rxElVH9+7dhZGR0SNz+3E/RkZGonv37lKHTtX04Hd+mRdeeEEn3/lxcXGic+fOol69eqJHjx7i1KlTmmnKvmN37NghVCqVMDY2FpcvXxb37t0TM2fOFE2bNhVKpVJ4enqKAwcOlJtv+/btwsXFRdSrV08899xzYteuXZrv1YsXL4qhQ4eKJk2aCEtLS9GtWzcRHx+vFWPLli3F/Pnzxbhx40T9+vXFhAkThBBCHDx4UPTu3VuYm5uL5s2bizfffFPk5uZq5lu1apVmuU2aNBFBQUHV/B8Q4tlnnxVvvvmmmDVrlmjYsKGwt7cX8+bNe+I8Zf9v//nPf4Sjo6NwdnYWQghx5coV0aRJE/Hll18+cr7S0lLx4YcfCnd3d1FYWKgZNm/ePNGiRQthZmYmHB0dxZtvvimE+N/+qlGjRpVqUwghAIj//ve/YtiwYcLCwkK4uLiIHTt2PHG9Ht5fPk5OTo4wNTUV3333nWZYRkaGACCOHDmiGVZQUCDq1asn9u7d+9Q2ybC1b99ezJgxQ+ow6AkM6srQw3799VeYmZkBAMLCwqBWqzU/S5YsgVKpRLdu3QAABw8ehIeHxyPbCQ8Px8qVK3H48GH88ccfePHFF7F8+XJ8++232LVrF/bs2YP/+7//00yfl5eHGTNm4Pjx49i3bx+MjIwwfPjwcmcP58yZg7CwMKSnp+OZZ57B6NGjdXIWKD8/H0uWLMGGDRvwyy+/IDMzE2FhYdVu90Genp5ISUlBQUGBTtslqqj09HQcO3as0mflS0tLcezYMfYyV8ecPn0ahw8f1sl3/qxZsxAREYFjx47Bzs4OgYGBKCoq0ozPz8/Hp59+inXr1uG3335DkyZNMG3aNBw5cgSbN2/GyZMnMXLkSAwYMAAXLlzQmm/hwoX45ptvcOjQIeTk5OD999/XjM/NzcWgQYOwb98+nDhxAgMGDEBgYCAyMzO14luyZAlcXV1x4sQJfPjhh7h06RIGDBiAoKAgnDx5Elu2bEFSUhKmTZsGADh+/DhCQ0Mxf/58nDt3Dj/99BP69Omjk+2+fv16WFpaIjk5GYsXL8b8+fMRHx//xHn27duHc+fOIT4+HnFxcQCAd999FxMnTsSrr76KP//8E0OGDEGTJk0QEBCABQsW4PXXX8f8+fNhaWmpuSvhhx9+0FwRvHDhAmJiYtC5c2etZQUGBlaqzTIfffQRXnzxRZw8eRKDBg3C2LFjcevWrWpvr9TUVBQVFcHf318zrH379nBycsKRI0c0w8zMzNC1a1ccPHiw2sskouoxkTqAyoiLi4OVlZVmp5WdnY1Zs2YBAKysrGBlZQUAOHr0KD744AOsX78enTp1AgBcvXoVTZs2fWS7//nPf9CrVy8A95+Xee+993Dp0iW0bt0aADBixAgcOHAAs2fPBgAEBQVpzf/VV1/Bzs4OZ86c0SwPuL+zHjx4MID7X7wdO3bExYsX0b59+2pth6KiIqxZswZt2rQBAEybNg3z58+vVpsPa9q0KQoLC3H9+nW0bNlSp20TVURKSkq15u/fvz/q16+vo2iopv3zzz/Izc2FkdH9c3ZCCAD3i92y7+Yy9+7dw/Xr12Fra4uhQ4cCAG7cuAEjIyOtacs618jKysLkyZMB3L8N7o8//kCzZs1gZWWFO3fuoKioCFeuXEFwcDAAoLi4GH/88QdatGiBCRMmaNpTKBTo3r07GjVqpJnv2rVrGDt2LID7t4hdu3ZNM72rqytcXV01nxcsWIDt27cjNjZWU9gAgJ+fH2bOnKn5/Morr2Ds2LGYPn06AKBt27ZYsWIFnn32WaxevRqZmZmwtLTEkCFDUL9+fbRs2RJubm5V2ezldOnSRXNrYtu2bbFy5Urs27cP/fv3f+w8lpaWWLdunaZwzc3Nxa5du3D58mUAwIQJE2BlZYWffvoJGRkZmDJlima/OmHCBPz888+YOHEiMjMz4eDgAH9/f5iamsLJyUnzzHB+fj4AYPz48ZVqs0xISAhGjx4NAPj444+xYsUKpKSkYMCAAdXaXtevX4eZmRlsbGy0htvb2+P69etaw5o2bYqrV69Wa3lEVH0GVQz17dsXq1evxvHjxzF69GgEBgaWK0wyMzMxbNgwhIWF4cUXX9QMv3v3LszNzR/ZbpcuXTS/29vbQ6lUau1A7e3ttQ7MLly4gLlz5yI5ORlZWVmaM9eZmZlaxdCD7To6OgIA/v7772oXQ0qlUlMIlbWt63uiLSwsAPxvh0NU0+7cuQNjY+MqdYagUCjQpUsX9OjRQw+RUU2Ii4vDnTt3EBAQgKKiIhw7dgxGRkYYNGiQ1nS3b9/G+vXr4eXlhWeffVYzfPPmzWjYsCECAgI0w65evYpvv/0WI0eOhLW1tWb4V199hbZt28LHxwcnT57ETz/9hPHjx0OhUAAALl68iD/++KPcwWxJSQlatWqFYcOG4eTJk/jxxx8xYcIEzXwAEBERoXluJjc3F+Hh4di1axfUajWKi4tx9+7dcleGyq5ulfn1119x8uRJREdHa4YJIVBaWorLly+jf//+aNmyJVq3bo0BAwZgwIABGD58OJRKZaW2+aM8uB8DKra/6dy5s6YQAoDz58/D2dkZjRs3Rl5eHvbv349r166hadOmcHd3R0JCguYkp6OjI7KzswEAI0eOxPLlyzXrNWjQIAQGBsLExERTRNjY2FSqzUetl6WlJRo0aFDjzxZZWFhwH0tUCxhUMWRpaQkXFxf8+++/AO7fNvHgA7J5eXkYOnQovLy8yl0psbW1LfdlWMbU1FTze1nPRQ9SKBRat+oEBgaiZcuW+O9//4umTZuitLQUnTp1Kveg68PtAo/vDasyHhVf2VlTXSm7XcDOzk6n7RJVVP369avcK5wQAqNGjcJrr72m46iopvz111/IycnBmjVrANz/7nR1dUWbNm20vvN79eqF/v37Y9u2bVpFyNWrVyGEwMcff6wZlpCQgG+//RazZ8+Gk5OTZviPP/4If39/zJs3D1FRUUhKSsInn3yiGb9lyxZs27YNp06dgrGxsVacVlZWcHBwQFRUFH7++WcsXLhQczULAFauXKnZN4SFhSE+Ph5LliyBi4sLLCwsMGLEiHL7DktLS63Pubm5mDx5MkJDQ8ttJycnJ5iZmSEtLQ0JCQnYs2cP5s6di/DwcBw7dqzcFYrKetr+8FEejr+4uFhzgq2sQHlwGisrK83+OS0tDS4uLgCAFi1a4Ny5c9i7dy/i4+Pxxhtv4LPPPkNiYqLWd0Nl2qzOelWEg4MDCgsLkZOTo7Xtb9y4Ua7Tolu3bmmd2CQiaRj0M0Mvv/wyPvjgA9y9exdCCAQHB6O0tBQbNmzQ2ikCgJubG86cOVPtZd68eRPnzp3DBx98gH79+kGlUj22yDJkp0+fRvPmzWFrayt1KCRTZbfDSDU/1S5GRkZ4//33dfKdf/ToUc3v2dnZOH/+PFQq1WOX7ebmhpKSEvz9999wcXHR+nnwALe4uBjHjx/XfD537hzu3Lmj+Xzo0CGEhIRg+PDh6Ny5MxwcHCr0DiR3d3ecOXOm3LJdXFw0V2BMTEzg7++PxYsX4+TJk7hy5Qr279//1LZrQuvWrXH+/HkUFRXBxsYGHTt2xMKFC1FUVISzZ89i8+bNKC0txa5du7Bq1SqtWwYtLCwQGBiIFStWICEhAUeOHMGpU6fQrFkzAKhSm/rk4eEBU1NT7Nu3TzPs3LlzyMzMhJeXl9a0p0+f1tntjERUdQZdDPn7+8PY2BirVq1CeHg49u7di7Vr1yI3NxfXr1/H9evXNfeIBwQE4Lfffqt24dKwYUM0btwYX375JS5evIj9+/djxowZulidGlFYWIj09HSkp6dr7mdPT0/HxYsXtaY7ePAgnnvuOYmiJAK6du2K7t27a51lrwgjIyN0794dXbt21U9gJJmRI0fq5Dt//vz52LdvH06fPo2QkBDY2tpi2LBhj13uM888g7Fjx2L8+PHYtm0bLl++jJSUFHzyySfYtWuXZjpTU1O8+eabSE5ORmpqKkJCQrQe+G/bti22bduG9PR0/PrrrxgzZkyFrkbMnj0bhw8fxrRp05Ceno4LFy5gx44dmgP8uLg4rFixAunp6bh69Sq++eYblJaWol27dhXdtFU2fvx4vPfee0+cxtbWFl26dNF0YvD1119j06ZNsLCwgL+/P4YOHYqNGzdi7ty52Lp1q6YwjYqKQmRkJE6fPo3ff/8dGzduhIWFBVq2bImGDRsCuH9VrzJtVlT79u2xffv2ym4OWFtbY9KkSZgxYwYOHDiA1NRUTJw4EV5eXujZs6dmuitXruDatWtaHS0QkTQMuhgyMTHBtGnTsHjxYuzevRu5ubnw9vaGo6Oj5mfLli0A7t/D7O7ujq1bt1ZrmUZGRti8eTNSU1PRqVMnvP322/jss890sTo14q+//oKbmxvc3Nw0PTC5ubnhlVde0UxTUFCgeWcDkZRCQ0Or1Jvco24nIsOnq+/8RYsW4a233oKHhweuX7+OnTt3aj3j8ihff/01xo8fj5kzZ6Jdu3YYNmwYjh07pnW7nVKpxOzZszFmzBj06tULVlZWWrfbLV26FA0bNoS3tzcCAwMREBAAd3f3p653ly5dkJiYiPPnz8PHxwdubm6YO3euplMgGxsbbNu2DX5+flCpVFizZg02bdqEjh07Vmi7VkdmZibUavVTp/vkk08QFhaGtLQ0dO/eHZmZmcjMzMSVK1cQERGBW7duITU1FT4+Ppp5bGxs8N///he9evVCly5dsHfvXuzcuRONGzfWTLN8+fJKtVlR586dq/KLx5ctW4YhQ4YgKCgIffr0gYODA7Zt26Y1zaZNm/Dcc8+xgyKi2kDCbr2r7MH34VRGXFycUKlUoqSkRE+RGbay7fruu++K/v37Sx0OkSguLhbDhg2r8LuGjIyMxPDhw0VxcbHUoVMt8PB3ftl7hrKzs3W+rLL3DD2sqvuruigqKkpYW1uLDz/8UFy4cEGUlpaKoqIicejQITFkyBCxdOnSCrdVtl3Dw8N11mZNKSgoEE5OTiIpKUnqUKgG8D1DtZ9BXxmqrMGDB+O1117T6uqUyjMxMdF6rxKRVIyNjfHtt99qukt+3C1zZcOHDh2K6Ojocg+5kzzxO792mTBhAn755RecOXMGrq6uMDMzQ7169RAcHIzevXtj6tSplW4zMDBQ523qW2ZmJt5//33NKz2ISFoG1ZucLpS9p0FKAwcOfOyL1vLy8sr1xFPm/fff13qB36OUvWvpUX788ccK3S4wfPjwGrnXnKgiLCws8P3332PTpk1YsWIFjh07Vm4aDw8PvPXWW3jppZdYCJGW2vCdL5XMzEx06NDhkePKunR+XPfbZ86c0boFUFe6dOmC77//HsXFxbhx4wbq1atX7Y569NHm40RHR2veUfWwli1b4rfffntqG2WdXxBR7SC7Yqg2WLduneYh34dZWFg8dlyjRo2e2nZ6evpjx5X1vkNkaIyNjREcHIzg4GCkp6dj8uTJuHnzJt555x14enqyswSqEF9fX52/hqBMSEgIQkJC9NJ2VTVt2vSJ+4SnzatPJiYmOt8n6aPNhw0dOvSx7y97uLtuIjIMLIYkoM8va55torqua9eucHFxgaWlJd8jRPQEJiYm3CfoWP369VG/fn2pwyAiHZLVM0NERERERERlWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJYN+z1BGRobUIdQp3J5kKOrXr4979+5JHQZRhfH7Vbe4PYlIVwyyGLK1tYVSqURwcLDUodQ5SqUStra2UodB9ER37txBdna21GEQPRX3V/rD/RUR6YJBFkNOTk7IyMhAVlaW1KE81dKlS3Ho0CH88MMPUodSIba2tnBycpI6DCKiOsGQ9leTJ0+Gra0tFi5cKHUoFcL9FRHpgkEWQ8D9HYwhfAna29vD3Nwc7u7uUodCREQSMJT9Vf369dGoUSPur4hIVtiBAhERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGKpDrly5AoVCgfT09MdOk5CQAIVCgZycnBqLi8iQMa+IdI95RUS1BYshmfH29oZarYa1tTUAICoqCjY2NtIGRWTgmFdEuse8IqKaYLDvGaKqMTMzg4ODg9RhENUpzCsi3WNeEVFN4JWhWiQvLw/jx4+HlZUVHB0dERERAV9fX0yfPh0AoFAoEBMTozWPjY0NoqKitIadPXsW3t7eMDc3R6dOnZCYmKgZ9+BtBwkJCZg4cSJu374NhUIBhUKB8PBw/a4kUQ1jXhHpHvOKiOoKFkO1yKxZs5CYmIgdO3Zgz549SEhIQFpaWpXamTlzJk6cOAEvLy8EBgbi5s2b5abz9vbG8uXL0aBBA6jVaqjVaoSFheliVYhqDeYVke4xr4iormAxVEvk5uYiMjISS5YsQb9+/dC5c2esX78excXFlW5r2rRpCAoKgkqlwurVq2FtbY3IyMhy05mZmcHa2hoKhQIODg5wcHCAlZWVLlaHqFZgXhHpHvOKiOoSFkO1xKVLl1BYWIgePXpohjVq1Ajt2rWrdFteXl6a301MTNCtWzdkZGToJE4iQ8K8ItI95hUR1SUshgyIQqGAEEJrWFFRkUTRENUNzCsi3WNeEZGhYDFUS7Rp0wampqZITk7WDMvOzsb58+c1n+3s7KBWqzWfL1y4gPz8/HJtHT16VPN7cXExUlNToVKpHrlcMzMzlJSU6GIViGod5hWR7jGviKguYdfatYSVlRUmTZqEWbNmoXHjxmjSpAnmzJkDI6P/1at+fn5YuXIlvLy8UFJSgtmzZ8PU1LRcW6tWrULbtm2hUqmwbNkyZGdn4+WXX37kcp2dnZGbm4t9+/bB1dUVSqUSSqVSb+tJVJOYV0S6x7wiorqEV4Zqkc8++ww+Pj4IDAyEv78/evfuDQ8PD834iIgItGjRAj4+PhgzZgzCwsIeuSNYtGgRFi1aBFdXVyQlJSE2Nha2traPXKa3tzemTJmCUaNGwc7ODosXL9bb+hFJgXlFpHvMKyKqKxTi4Zt6SadmzpyJ3bt3V/mBUF9fX3Tt2hXLly/XbWBEBmzs2LFQq9XYv39/leZnXhGV5+fnB0dHR0RHR1dpfuYVUXkqlQqDBg1CRESE1KHQY/DKEBERERERyRKLISIiIiIikiV2oFDLJSQkSB0CUZ3DvCLSPeYVERkiXhkiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyZKJ1AFUVWZmJrKysqQO46ny8/NhZWWFtLQ0qUOpEFtbWzg5OUkdBknEUPKqpKQEJiYmzCsiHWrQoAGUSqXUYRBViKHsr6ysrJCfn8/9VS2mEEIIqYOorMzMTKhUKuTn50sdSp2jVCqRkZEhu0Qg5pU+Ma/IEPj5+cHR0RHR0dFSh0L0RNxf6Y8c91cGeWUoKysL+fn52LhxI1QqldTh1BkZGRkIDg5GVlaWrJKA7mNe6QfziohIt7i/0g+57q8Mshgqo1Kp4O7uLnUYRHUK84qIiAwB91ekC+xAgYiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERCRz6enp+Ouvv5CRkYEvv/wS6enpUodERFQjWAwRERHJUElJCTZu3AhPT0+4ubnh3LlzSE9Px+TJk+Hm5gZPT09s3LgRJSUlUodKRKQ3siqGzp07BwcHB9y5c0fqUGq1IUOG4Pjx41KHQUREenL37l0EBQVh3LhxSE1N1QwXQmh+T01Nxbhx4zBixAjcvXtXijCpFrp58yaaNGmCK1euSB0KGThnZ2csX75cJ21lZWWhSZMm+PPPPys9r0EUQyEhIVAoFFAoFDA1NUVgYCAAoKCgoFLtvPfee3jzzTdRv359fYRpED755BN0794d9evXR5MmTTBs2DCcO3dOa5px48Zh9uzZEkVI9GghISEYNmxYueEJCQlQKBTIycmp8ZiIDFFJSQlGjx6NnTt3AgBKS0sfOV3Z8NjYWIwZM4ZXiOqAh4+nWrVqhXfeeQf37t2rcBsLFy7E888/D2dnZ/0FSpWSm5uLiIgI9O7dGw4ODmjWrBn8/Pywdu1aFBcX15o2q+PLL7+Er68vGjRo8Mh9vq2tLcaPH4958+ZVum2DKIYAYMCAAVCr1fj9998xY8YMAMDatWsrPH9mZibi4uIQEhKipwgNQ2JiIqZOnYqjR48iPj4eRUVFeO6555CXl6eZZuDAgUhKSsJvv/0mYaRERKQPmzZtwo4dOx5bBD2stLQUMTEx2LRpk54jo5rw4PHUsmXLsHbt2gofQObn5yMyMhKTJk2qVgwlJSUV/vvTh8LCQsmWrQsPxp+amooOHTogJiYGr776KmJjYxEXF4cJEyYgKioK3bt3x99//12p9jMyMnTeZnXl5+djwIABeP/99x87zcSJExEdHY1bt25Vqm2DKYbq1asHBwcHtGjRAn379gUAHD16FADwzTffwMrKChcuXNBM/8Ybb6B9+/bIz88HAGzduhWurq5o1qyZZpqoqCjY2NggLi4O7dq1g1KpxIgRI5Cfn4/169fD2dkZDRs2RGhoqNYZsQ0bNqBbt26oX78+HBwcMGbMGK0/irIz1fv27UO3bt2gVCrh7e1d7gpMVYSHh6Nr167YsGEDnJ2dYW1tjZdeeqnCt/799NNPCAkJQceOHeHq6oqoqChkZmZq3SbRoEED9OrVC5s3b652vEREVLusWLECRkaV2/0bGRlhxYoVeoqIatKDx1PDhg2Dv78/4uPjAdw/xii7cvTgT1RUFABg9+7dqFevHnr27Klpr+yYZ9euXejSpQvMzc3Rs2dPnD59WjNN2fFWbGwsOnTogHr16iEzMxMFBQUICwtDs2bNYGlpiR49eiAhIaHcfDExMWjbti3Mzc0REBCA69eva6a5dOkSnn/+edjb28PKygrdu3fH3r17tdbZ2dkZCxYswPjx49GgQQO89tprAICkpCT4+PjAwsICLVq0QGhoqNbJ4S+++EKzXHt7e4wYMaLa29/X1xehoaF455130KhRIzg4OCA8PPyJ85TdGbFw4UI0bdoU7dq1AwBcvXoVgwYNwocffoiDBw9iwoQJmmcAJ0yYgMOHDyMwMBADBw5EUVERgPu3woaHh8PJyQn16tVD06ZNERoaqrW80NDQSrUJAAqFAuvWrcPw4cOhVCrRtm1bxMbGVnt7lZk+fTreffddrb+9h3Xs2BFNmzbF9u3bK9W2wRRDD7p48SIAwNTUFAAwfvx4DBo0CGPHjkVxcTF27dqFdevWITo6GkqlEgBw8OBBdOvWrVxb+fn5WLFiBTZv3oyffvoJCQkJGD58OHbv3o3du3djw4YNWLt2Lb7//nvNPEVFRViwYAF+/fVXxMTE4MqVK4+84jRnzhxERETg+PHjMDExwcsvv6yT9b906RJiYmIQFxeHuLg4JCYmYtGiRVVq6/bt2wCARo0aaQ339PTEwYMHqx0rERHVHunp6Th27Filz8qXlpbi2LFj7GWujjl9+jQOHz4MMzMzAEBYWBjUarXmZ8mSJVAqlZrjp4MHD8LDw+ORbc2aNQsRERE4duwY7OzsEBgYqHWwnJ+fj08//RTr1q3Db7/9hiZNmmDatGk4cuQINm/ejJMnT2LkyJEYMGCA1snt/Px8LFy4EN988w0OHTqEnJwcrasDubm5GDRoEPbt24cTJ05gwIABCAwMRGZmplZ8S5YsgaurK06cOIEPP/wQly5dwoABAxAUFISTJ09iy5YtSEpKwrRp0wAAx48fR2hoKObPn49z587hp59+Qp8+fXSy3devXw9LS0skJydj8eLFmD9/vqYgfZx9+/bh3LlziI+PR1xcHADg3XffxcSJE/Hqq6/izz//xJAhQ9CkSRMEBARgwYIFeP311zF//nxYWlpi48aNAIAffvhBc0XwwoULiImJQefOnbWWFRgYWKk2y3z00Ud48cUXcfLkSc1xeWWv0lRXVY5fTfQUi87FxcXBysoKxcXFmmeFxo8frxm/du1adOnSBaGhodi2bRvCw8O1Evbq1auPLIaKioqwevVqtGnTBgAwYsQIbNiwATdu3ICVlRU6dOiAvn374sCBAxg1ahQAaBU1rVu3xooVK9C9e3fk5ubCyspKM27hwoV49tlnAdz/gx08eDDu3bsHc3Pzam2L0tJSREVFaZ59GjduHPbt24eFCxdWup3p06ejV69e6NSpE9LS0jTjmjZtiqtXr1YrTiJdK/seeBCfYyCquJSUlGrNP2zYMNjZ2ekoGqpply9fxs2bN2FsbAwhhKbDDEtLS3Tv3l1r2tzcXJw7dw6tWrXCxIkTAdw/GW1iYqI17b///gvg/nPcZUVKcXExMjMz0b59ezRq1AhZWVkoKipCTk4O3nrrLc30p06dQpcuXTSPPwCAmZkZ+vTpg+bNm2vmy83N1Vy9uHv3rtZt/K6urnB1ddV8XrBgAbZv347Y2FhNYQMAfn5+mDlzpubzK6+8grFjx2L69OkAgLZt22LFihV49tlnsXr1amRmZsLS0hJDhgxB/fr10bJlS7i5uVVxy2vr0qWL5tbEtm3bYuXKldi3bx/69+//2HksLS2xbt06TeGam5uLXbt24fLlywCACRMmwMrKCj/99BMyMjIwZcoUBAUFacb9/PPPmDhxIjIzM+Hg4AB/f3+YmprCyckJnp6eAKC5m6rs+LqibZYJCQnB6NGjAQAff/wxVqxYgZSUFAwYMEAn260imjZtihMnTlRqHoMphvr27YvVq1cjLy8PH3zwAeLi4tCvXz/N+IYNGyIyMhIBAQHw9vbGu+++qzX/3bt3H1mEKJVKTSEEAPb29nB2dtY64LK3t9e6DS41NRXh4eH49ddfkZ2drTnDlpmZiQ4dOmim69Kli+Z3R0dHAMDff/8NJyenqm4GAPcv9z7YCYSjo2OV7t2cOnUqTp8+jaSkpHLjLCwsNElBVFuUfQ88KDk5GcHBwRJFRGRY7ty5A2Nj4yqdRFAoFHB0dNQ68CTD8u+//8Lc3Bw+Pj4oLi7GyZMnYWRkpDlxW+bOnTvYvn073NzctAqf69evo0GDBlonm//66y+cP38ePXv21Do2+eOPP9CwYUN4eHjg3LlzyMzMRO/evaFQKADcP0l96tQpnDlzRmvZpaWlsLOz05qvT58+mvkA4Pz585qrTrm5uQgPD8euXbugVqtRXFyMu3fvlrsy9PAJ8V9//RUnT55EdHS0ZpgQAqWlpbh8+TL69++Pli1bonXr1hgwYAAGDBiguQWsuh48PgQqdhzXuXNnTSEE3N8Gzs7OaNy4MfLy8rB//35cu3YNTZs2hbu7OxISEjTbyNHREdnZ2QCAkSNHYvny5Zr1GjRoEAIDA2FiYqI5CW5jY1OpNh+1XpaWlmjQoEGNP1tUleNXgymGLC0t4eLiAgCYN28e4uLiEBMTA3d3d800v/zyC4yNjaFWq5GXl6eVlLa2tuX+04D/3WpXpqyHlYeHlRU8eXl5CAgIQEBAAKKjo2FnZ4fMzEwEBASUeyDvwXbKklgXDww+Kb6KmjZtGuLi4vDLL7+gefPm5cbfunWLZ/+o1nnwe6BMVbrRJJKr+vXrV/lqqhACEydO1DxvQYYnJCQEOTk5ms4wSktL4erqiu7du2s6RcjLy0OvXr0QEBCAbdu2aRUhY8eOhRACa9as0QxLSEjAzp078cknn2id7E1OTkZgYCDmzZuHqKgonDhxQqvjqy1btiA+Ph6nT5+GsbGxVpxWVlZwcHBAVFQUkpKSsHr1aq3n3L799lvNQXlYWBji4+OxZMkSuLi4wMLCAiNGjCh3TGZpaan1OTc3F5MnTy73vAwAODk5wczMDGlpaUhISMCePXswd+5chIeH49ixY7CxsanQ9n6cqhzHPRx/cXExLCwsAECzLR6cxsrKSnPcm5aWptl3tmjRAufOncPevXsRHx+PN954A5999hkSExO1vhsq02Z11kvXqnL8apDPDJUlxBdffKF598Hhw4fx6aefYufOnbCystK6NAoAbm5u5c4+VMXZs2dx8+ZNLFq0CD4+Pmjfvn2NV73VIYTAtGnTsH37duzfvx+tWrV65HSnT5/W2eVgIiKqHcpuh5FqfqpdjIyM8P777+ODDz7A3bt3IYRAcHAwSktLsWHDBq1CCHjysVRZp1YAkJ2djfPnz0OlUj122W5ubigpKcHff/8NFxcXrR8HBwfNdMXFxVrvPjx37pxWp1GHDh1CSEgIhg8fjs6dO8PBwaFC70Byd3fHmTNnyi3bxcVFcwXGxMQE/v7+WLx4MU6ePIkrV65g//79T227JrRu3VpzhczGxgYdO3bEwoULUVRUhLNnz2Lz5s0oLS3Frl27sGrVKq3jYgsLCwQGBmLFihVISEjAkSNHcOrUKU0nY1Vps7aoyvGrQRZDZYyNjbFq1SrcuXMH48aNQ2hoKAYOHIjo6Ghs2bJFq9ODgIAAHDlypNrPF5SdLfi///s//P7774iNjcWCBQuquyo1ZurUqdi4cSO+/fZb1K9fH9evX8f169fLvVDv4MGDeO655ySKkoiI9KFr167o3r17lXqT6969O7p27aqfwEgyI0eO1BxPhYeHY+/evVi7di1yc3PLHSMEBATgt99+e+SdNvPnz8e+fftw+vRphISEwNbW9pHvhivzzDPPYOzYsRg/fjy2bduGy5cvIyUlBZ988gl27dqlmc7U1BRvvvkmkpOTkZqaipCQEK0H/tu2bYtt27YhPT0dv/76K8aMGVOhqxGzZ8/G4cOHMW3aNKSnp+PChQvYsWOH5gA/Li4OK1asQHp6Oq5evYpvvvkGpaWlmp7c9Gn8+PF47733njiNra0tunTpounE4Ouvv8amTZtgYWEBf39/DB06FBs3bsTcuXOxdetWTWEaFRWFyMhInD59Gr///js2btwICwsLtGzZEg0bNgQA/Pjjj5Vqs6Lat29f6Z7eyly/fh3p6emaTtROnTqF9PR0rQ4a8vPzkZqaWunjV4Muhl588UUsXrwYoaGhsLS0xMcffwzg/n2VH3/8MSZPnoxr164BuP/uHBMTk3LdLVaWnZ0doqKi8N1336FDhw5YtGgRlixZUu11qSmrV6/G7du34evrC0dHR83Pli1bNNOcPHkSt2/f1kkXkkREVLuEhoZWqTe5R91ORIbPxMQE06ZNw+LFi7F7927k5ubC29v7kccInTt3hru7O7Zu3VqunUWLFuGtt96Ch4cHrl+/jp07d2o94/IoX3/9NcaPH4+ZM2eiXbt2GDZsGI4dO6Z1u51SqcTs2bMxZswY9OrVC1ZWVvjkk08045cuXYqGDRvC29sbgYGBCAgI0HqE4nG6dOmCxMREnD9/Hj4+PnBzc8PcuXPRtGlTAPefm9m2bRv8/PygUqmwZs0abNq0CR07dqzQdq2OzMxMqNXqp073ySefICwsDGlpaejevTsyMzORmZmJK1euICIiArdu3UJqaip8fHw089jY2OC///0vevXqhS5dumDv3r3YuXMnGjdurJlm+fLllWqzos6dO6fpxbiy1qxZAzc3N7z66qsAgD59+sDNzU2r++4dO3bAycmp8rEJA5SamioAiNTU1ErNt3LlSvHcc8/pKSrDV7Zd+/fvLxYuXCh1OFTDqppX9GTcrlTbFBcXi2HDhgkjIyMB4Kk/RkZGYvjw4aK4uFjq0KkWiIuLEyqVSpSUlAghhDhw4IAAILKzs3W+rK+//lpYW1uXG87v1f+JiooS1tbW4sMPPxQXLlwQpaWloqioSBw6dEgMGTJELF26tMJtlW3X8PBwnbVZk3r06CGio6MrPZ9BXxmqrMmTJ6NPnz4VfkGpXLm4uODtt9+WOgwiItIDY2NjfPvttxg6dCgAPPaWubLhQ4cORXR0dLmH3EmeBg8ejNdee01z5w1Ja8KECfjll19w5swZuLq6wszMDPXq1UNwcDB69+6NqVOnVrrNwMBAnbepb1lZWXjhhRc0XXtXhsH0JqcLJiYmmDNnjtRhoGPHjo99h4+dnR3++eefR45bu3Ytxo4d+9h2H+7a+2FnzpypULfer7zyiqaHEiIiqnssLCzw/fffY9OmTVixYgWOHTsG4H7vT+L/v3vGw8MDb731Fl566SUWQqSl7N08cvSkY62yLp0f1/12RY/DKqtLly74/vvvUVxcjBs3bqBevXqwtbWtdW0+TnR0NCZPnvzIcS1bttR6r9Tj2Nra4p133qnS8mVVDNUWu3fv1nor84NMTU0fO87e3v6J7TZt2vSJbwcvuw+WiIjI2NgYwcHBCA4ORnp6Ol566SUolUpMmTIFnp6e7CyBKsTX11dTQOtaSEgIQkJC9NJ2VT3tWOtp8+qTiYmJpke42tzmw4YOHYoePXo8ctzD3XXrA4shCbRs2VIv7ZqYmJTr852IiOhpunbtiqZNm8LR0ZHvESJ6Ah5r6V79+vW13g1a02T1zBAREREREVEZFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkyaDfM5SRkSF1CHUKtycB/DvQNW5PMhRKpRL16tWTOgyiCuP3q27JdXsaZDFka2sLpVKJ4OBgqUOpc5RKJWxtbaUOgyTAvNIf5hUZgvz8fBQUFEgdBtFTcX+lP3LcXxlkMeTk5ISMjAxkZWVJHcpTLV26FIcOHcIPP/wgdSgVYmtrCycnJ6nDIAkYUl7NmTMHWVlZWLt2rdShVAjziohIdwxpfxUUFIRevXphxowZUodSIXLcXxlkMQTcTwRD+M+yt7eHubk53N3dpQ6F6KkMJa8aNWqEgoIC5hURkUwZyv7K3Nwc9vb23F/VYuxAgYiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDNUhV65cgUKhQHp6+mOnSUhIgEKhQE5OTo3FRWTImFdEuse8ItI95lXVsBiSGW9vb6jValhbWwMAoqKiYGNjI21QRAaOeUWke8wrIt1jXpVnsO8ZoqoxMzODg4OD1GEQ1SnMKyLdY14R6R7zqjxeGapF8vLyMH78eFhZWcHR0RERERHw9fXF9OnTAQAKhQIxMTFa89jY2CAqKkpr2NmzZ+Ht7Q1zc3N06tQJiYmJmnEPXh5NSEjAxIkTcfv2bSgUCigUCoSHh+t3JYlqGPOKSPeYV0S6x7ySBouhWmTWrFlITEzEjh07sGfPHiQkJCAtLa1K7cycORMnTpyAl5cXAgMDcfPmzXLTeXt7Y/ny5WjQoAHUajXUajXCwsJ0sSpEtQbzikj3mFdEuse8kgaLoVoiNzcXkZGRWLJkCfr164fOnTtj/fr1KC4urnRb06ZNQ1BQEFQqFVavXg1ra2tERkaWm87MzAzW1tZQKBRwcHCAg4MDrKysdLE6RLUC84pI95hXRLrHvJIOi6Fa4tKlSygsLESPHj00wxo1aoR27dpVui0vLy/N7yYmJujWrRsyMjJ0EieRIWFeEeke84pI95hX0mExZEAUCgWEEFrDioqKJIqGqG5gXhHpHvOKSPeYV/rBYqiWaNOmDUxNTZGcnKwZlp2djfPnz2s+29nZQa1Waz5fuHAB+fn55do6evSo5vfi4mKkpqZCpVI9crlmZmYoKSnRxSoQ1TrMKyLdY14R6R7zSjrsWruWsLKywqRJkzBr1iw0btwYTZo0wZw5c2Bk9L961c/PDytXroSXlxdKSkowe/ZsmJqalmtr1apVaNu2LVQqFZYtW4bs7Gy8/PLLj1yus7MzcnNzsW/fPri6ukKpVEKpVOptPYlqEvOKSPeYV0S6x7ySDq8M1SKfffYZfHx8EBgYCH9/f/Tu3RseHh6a8REREWjRogV8fHwwZswYhIWFPfIPdtGiRVi0aBFcXV2RlJSE2NhY2NraPnKZ3t7emDJlCkaNGgU7OzssXrxYb+tHJAXmFZHuMa+IdI95JQ2FePjmQ9KpmTNnYvfu3VV+cM3X1xddu3bF8uXLdRsYkQEbO3Ys1Go19u/fX6X5mVdE5fn5+cHR0RHR0dFVmp95RVSeSqXCoEGDEBERUaX5mVf6xytDREREREQkSyyGiIiIiIhIltiBQi2XkJAgdQhEdQ7zikj3mFdEuse80j9eGSIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJkonUAVRVZmYmsrKypA7jqW7dugUASEtLkziSirG1tYWTk5PUYZBEDCWv7ty5g4KCAuYVGQRDyauCggLcuXOHeUUGwVDyCrh/LMi8qr0UQgghdRCVlZmZCZVKhfz8fKlDqXOUSiUyMjJklwjEvNIn5pV8Ma/0h3klX8wr/ZFjXhnklaGsrCzk5+dj48aNUKlUUodTZ2RkZCA4OBhZWVmySgK6j3mlH8wreWNe6QfzSt6YV/oh17wyyGKojEqlgru7u9RhENUpzCsi3WNeEeke84p0gR0oEBERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMaQHISEhGDZsWLnhCQkJUCgUyMnJqfGYiAwd84pI95hXRLrHvDIsLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZMpE6gLoqLi4OVlZWWsNKSkokioaobmBeEeke84pI95hXhoPFkJ707dsXq1ev1hqWnJyM4OBgiSIiMnzMKyLdY14R6R7zynCwGNITS0tLuLi4aA37888/JYqGqG5gXhHpHvOKSPeYV4aDzwwREREREZEssRgiIiIiIiJZYjFERERERESyxGeG9CAqKuqRw319fSGEqNlgiOoI5hWR7jGviHSPeWVYeGWIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUTqQOojoyMDKlDqFO4PQng34GucXsSwL8DXeP2JIB/B7om1+1pkMWQra0tlEolgoODpQ6lzlEqlbC1tZU6DJIA80p/mFfyxbzSH+aVfDGv9EeOeaUQQgipg6iKzMxMZGVlSR3GUy1duhSHDh3CDz/8IHUoFWJrawsnJyepwyCJGEpezZkzB1lZWVi7dq3UoVQI80reDCWvJk+eDFtbWyxcuFDqUCqEeSVvhpJXQUFB6NWrF2bMmCF1KBUix7wyyCtDAODk5GQQ/1n29vYwNzeHu7u71KEQPZWh5FWjRo1QUFDAvCKDYCh5Vb9+fTRq1Ih5RQbBUPLK3Nwc9vb2zKtajB0oEBERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBERERERyRKLoTrkypUrUCgUSE9Pf+w0CQkJUCgUyMnJqbG4iAwZ84pI95hXRLrHvKoaFkMy4+3tDbVaDWtrawBAVFQUbGxspA2KyMAxr4h0j3lFpHvMq/IM9j1DVDVmZmZwcHCQOgyiOoV5RaR7zCsi3WNelccrQ7VIXl4exo8fDysrKzg6OiIiIgK+vr6YPn06AEChUCAmJkZrHhsbG0RFRWkNO3v2LLy9vWFubo5OnTohMTFRM+7By6MJCQmYOHEibt++DYVCAYVCgfDwcP2uJFENY14R6R7zikj3mFfSYDFUi8yaNQuJiYnYsWMH9uzZg4SEBKSlpVWpnZkzZ+LEiRPw8vJCYGAgbt68WW46b29vLF++HA0aNIBarYZarUZYWJguVoWo1mBeEeke84pI95hX0mAxVEvk5uYiMjISS5YsQb9+/dC5c2esX78excXFlW5r2rRpCAoKgkqlwurVq2FtbY3IyMhy05mZmcHa2hoKhQIODg5wcHCAlZWVLlaHqFZgXhHpHvOKSPeYV9JhMVRLXLp0CYWFhejRo4dmWKNGjdCuXbtKt+Xl5aX53cTEBN26dUNGRoZO4iQyJMwrIt1jXhHpHvNKOiyGDIhCoYAQQmtYUVGRRNEQ1Q3MKyLdY14R6R7zSj9YDNUSbdq0gampKZKTkzXDsrOzcf78ec1nOzs7qNVqzecLFy4gPz+/XFtHjx7V/F5cXIzU1FSoVKpHLtfMzAwlJSW6WAWiWod5RaR7zCsi3WNeSYdda9cSVlZWmDRpEmbNmoXGjRujSZMmmDNnDoyM/lev+vn5YeXKlfDy8kJJSQlmz54NU1PTcm2tWrUKbdu2hUqlwrJly5CdnY2XX375kct1dnZGbm4u9u3bB1dXVyiVSiiVSr2tJ1FNYl4R6R7zikj3mFfS4ZWhWuSzzz6Dj48PAgMD4e/vj969e8PDw0MzPiIiAi1atICPjw/GjBmDsLCwR/7BLlq0CIsWLYKrqyuSkpIQGxsLW1vbRy7T29sbU6ZMwahRo2BnZ4fFixfrbf2IpMC8ItI95hWR7jGvpKEQD998SDo1c+ZM7N69u8oPrvn6+qJr165Yvny5bgMjMmBjx46FWq3G/v37qzQ/84qoPD8/Pzg6OiI6OrpK8zOviMpTqVQYNGgQIiIiqjQ/80r/eGWIiIiIiIhkicUQERERERHJEjtQqOUSEhKkDoGozmFeEeke84pI95hX+scrQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZMpE6gKrKzMxEVlaW1GE81Y0bN3Dv3j2kpaVJHUqF2NrawsnJSeowiIiIiB7LUI4D7927hxs3bvA4sBYzyGIoMzMTKpUK+fn5UodSYR4eHlKHUCFKpRIZGRmySwQyLEZGRjA2NpY6DCIikoChHQdeuXIF0dHRUodRIXI8DjTIYigrKwv5+fnYuHEjVCqV1OHUGRkZGQgODkZWVpaskoAMT2lpKUpKSqQOg4iIJMDjQP2Q63GgQRZDZVQqFdzd3aUOg4iIiIhqGI8DSRfYgQIREREREckSiyEiIiIiIpIlFkNERERERCRLLIaIiIiIiEiWWAwREREREZEssRgiIiIiIiJZYjFERERERESyxGKIiAxKeno6Ll68iMzMTHz55ZdIT0+XOiQiIiIyUCyGiKjWKykpwcaNG+Hp6Qk3NzekpKTg0qVLmDx5Mtzc3ODp6YmNGzeipKRE6lCJiIjIgEhaDN28eRNNmjTBlStXqjT/8ePHoVAokJOT88jxV65cgUKh0Jw5TkhI0Jo+KioKNjY2WvN8+eWXaNGiBYyMjLB8+fIKxeHr64vp06dXaR1qq4e3lT69++67ePPNN/W+HDJMd+/eRVBQEMaNG4fU1NRHTpOamopx48ZhxIgRuHv3bg1HSEREUjl37hwcHBxw584dqUORnTVr1iAwMFDqMKqtysVQSEgIFAoFFAoFTE1N0apVK7zzzju4d+9ehdtYuHAhnn/+eTg7O1c1jCdq0aIF1Go1OnXqVKHp//33X0ybNg2zZ8/GtWvX8Nprr1V52WXFhL6cOXMGr7/+OlQqFRo3boy2bdtiwoQJOHLkiN6WWVUnT56Ej48PzM3N0aJFCyxevFhrfFhYGNavX4/ff/9dogiptiopKcHo0aOxc+dOAEBpaekjpysbHhsbizFjxvAKERFRLaeL40gAeO+99/Dmm2+ifv36eoq05uj72LGyMjMzMXjwYCiVSjRp0gSzZs1CcXGxZvzLL7+MtLQ0HDx4UMIoq69aV4YGDBgAtVqN33//HcuWLcPatWsxb968Cs2bn5+PyMhITJo0qTohPJGxsTEcHBxgYmJSoekzMzNRVFSEwYMHw9HREUqlUm+xVceiRYvQo0cPlJaWYsmSJUhMTMTXX3+N1q1bY+jQoXjvvfekDlHj33//xXPPPYeWLVsiNTUVn332GcLDw/Hll19qprG1tUVAQABWr14tYaRUG23atAk7dux4bBH0sNLSUsTExGDTpk16joyIiKqrOseRAKBWqxEXF4eQkBD9BSlTJSUlGDx4MAoLC3H48GGsX78eUVFRmDt3rmYaMzMzjBkzBitWrJAw0uqrVjFUr149ODg4oEWLFhg2bBj8/f0RHx8PAAgPD9dU/A/+REVFAQB2796NevXqoWfPnk9cxtWrVxEYGIiGDRvC0tISHTt2RFJS0iOnzc/Px8CBA9GrVy/k5OSUu03uSaKiotC5c2cAQOvWraFQKHDlyhWEhIRg2LBhWtNOnz4dvr6+j2zH19cXCoUCffv2BYBy6/0kCoUC69atw/Dhw6FUKtG2bVvExsZqTbNq1SqsW7cOqampWLt2LQYPHoxOnTqhd+/emDdvHs6cOYOff/4ZERERWutmY2ODn3/+GSqVClZWVpovIH2Ljo5GYWEhvvrqK3Ts2BEvvfQSQkNDsXTpUq3pAgMDsXnzZr3HQ4ZlxYoVMDKq3NeUkZGRwX8xExHJwZOOI7/55htYWVnhwoULmunfeOMNtG/fXnM7dHx8PFxdXdGsWbMnLufXX39F3759Ub9+fTRo0AAeHh44fvw4gPuPbIwePRrNmjWDUqlE586dy51Qc3Z2LvfoRNeuXREeHq75nJOTg8mTJ8Pe3h7m5ubo1KkT4uLiNOOTkpLg4+MDCwsLtGjRAqGhocjLywOg/2PHqtizZw/OnDmDjRs3omvXrhg4cCAWLFiAVatWobCwUDNdYGAgYmNjDfoWdZ09M3T69GkcPnwYZmZmAO7f+qRWqzU/S5YsgVKpRLdu3QAABw8ehIeHx1PbnTp1KgoKCvDLL7/g1KlT+PTTTx95xSYnJwf9+/dHaWkp4uPjyz0L9DSjRo3C3r17AQApKSlQq9Vo0aJFpdoAgG3btkGtVuOHH34AAM36jxo1qkLzf/TRR3jxxRdx8uRJDBo0CGPHjsWtW7cAAFlZWZg7dy62b9+OZ555Btu3b0enTp3QtGlTfPDBB+jfvz/Onj2LTZs2YeHChVr3z+bn52PJkiXYsGEDfvnlF2RmZiIsLKzS61dZR44cQZ8+fTR/FwAQEBCAc+fOITs7WzPM09MTf/75Z5WfH6O6Jz09HceOHavwVaEypaWlOHbsGHuZIyIyIA8fR44fP15zHFRcXIxdu3Zh3bp1iI6OhoWFBYD7+4my48onGTt2LJo3b45jx44hNTUV7777LkxNTQEA9+7dg4eHB3bt2oXTp0/jtddew7hx45CSklLh2EtLSzFw4EAcOnQIGzduxJkzZ7Bo0SIYGxsDAC5duoQBAwYgKCgIJ0+exJYtW5CUlIRp06YB0O+xY1UdOXIEnTt3hr29vWZYQEAA/v33X/z222+aYd26dUNxcTGSk5OrtTwpVez+sceIi4uDlZUViouLUVBQACMjI6xcuRIAYGVlBSsrKwDA0aNH8cEHH2D9+vWa53euXr2Kpk2bPnUZmZmZCAoK0rpqk5aWpjXN9evXMWrUKLRt2xbffvut1oF3RVlYWKBx48YAADs7Ozg4OFS6DQBo1KiR1r+VbSckJASjR48GAHz88cdYsWIFUlJSMGDAAGzfvh19+/ZF586dcenSJYwePRoRERHo1asXVq5ciQMHDmDOnDlo164dOnbsiEOHDmHAgAEAgKKiIqxZswZt2rQBAEybNg3z58/XWnbZGYr3338fRUVFAIBXXnmlStuzzIEDB2BpaYkxY8Zoht2+fRsAEBwcDGtra018Zctr0qRJlZdHdcfFixerNX9KSgq6du2qm2CIiEjnnnQcCQBr165Fly5dEBoaim3btiE8PBweHh6a40C1Wo1+/fo9dTmZmZmYNWsW2rdvDwBo27atZlyzZs20Tg6/+eab+Pnnn7F161Z4enpWaD327t2LlJQUZGRk4JlnngFw/3i1zCeffIKxY8dqOttq27YtVqxYgWeffRarV6/W67FjVV2/fl2rEAKg+Xz9+nXNMKVSCWtra1y9erXKy5JatYqhvn37YvXq1cjLy8OyZctgYmKCoKAgrWkyMzMxbNgwhIWF4cUXX9QMv3v3LszNzbWm7dixo2Zj+vj44Mcff0RoaChef/117NmzB/7+/uXaB4D+/fvD09MTW7Zs0VThhqpLly6a3y0tLdGgQQP8/fffAIBTp07B29sbAPDzzz+jT58+mDp1KgDgiy++0Lqs6+joqHXlRalUagqhsvFl7ZYpOwN/69YtzQNyf//9t+bsSVWUfbnduHFDM6ys6Lp586bmQcmyZWdlZUEIUeXlUd3x4N9vZRkbG7NnISKiWu5px5ENGzZEZGQkAgIC4O3tjXfffVdr/oKCgnLHkmUn4oH7J13XrFmDGTNm4JVXXsGGDRvg7++PkSNHao6JSkpK8PHHH2Pr1q24du0aCgsLUVBQUKnnxtPT09G8eXNNIfSwX3/9FSdPnkR0dLRmmBACpaWluHz5MlQqVYWX9ShPOnasCRYWFsjPz6+x5elatYohS0tLuLi4AAC++uoruLq6anWKkJeXh6FDh8LLy6vcVQhbW9tyBzu7d+/WXCEouwT6yiuvICAgALt27cKePXvwySeflOvGevDgwfjhhx9w5swZzRUkXTEyMip3cF4Woz48XHgoFApNoVBcXKzZLoWFhbC0tNRMZ2ZmprmCU1paivT0dMyaNeuJ7T68XmU9saxZswb//vsv+vbti9jY2Erfcvig8ePH499//0VMTIxm2IEDB+Dn54cff/wRDRs2BADcuHEDDg4OWLduXYUueVPd9+WXX2Ly5MlVmrekpKRO9CxERFSXPe04EgB++eUXGBsbQ61WIy8vT+u73cbGptyx5IO3SDdo0ADA/efYx4wZg127duHHH3/EvHnzsHnzZgwfPhyfffYZPv/8cyxfvhydO3eGpaUlpk+frvVczNOOBcuOzR4nNzcXkydPRmhoaLlxTk5OT5y3Ip507FhVDg4O5W4VLDux/fCVq1u3bsHOzq5ay5OSzp4ZMjIywvvvv48PPvgAd+/ehRACwcHBKC0txYYNG8p1Fejm5oYzZ85oDWvZsiVcXFzg4uKi9TBcixYtMGXKFGzbtg0zZ87E9u3bteZbtGgRJkyYgH79+pVrs7rs7OzKdTQg1bMILi4uOHXqFACgd+/e2LNnD44ePYqSkhKsXLkSOTk5+PfffzFz5kw0a9YM3bt3lyTOB3l5eeGXX37R+tKIj49Hu3btNIUQcP9eYVNTU3Ts2FGKMKkWqujtCfqan4iIas7Dx5EAcPjwYXz66afYuXMnrKysNM/YlGnXrl25476y40gXFxet2+6feeYZvP3229izZw9eeOEFfP311wCAQ4cO4fnnn0dwcDBcXV3RunVrnD9/XqvNh48F//33X1y+fFnzuUuXLvjzzz/LzVfG3d0dZ86c0Yqt7Kc6jyLok5eXF06dOqV1hSk+Ph4NGjRAhw4dNMMuXbqEe/fuwc3NTYowdUKnL10dOXIkjI2NsWrVKoSHh2Pv3r1Yu3YtcnNzcf36dVy/fl3zBx4QEIDffvvtqbfCTJ8+HT///DMuX76MtLQ0HDhwAK1atSo33ZIlSzB27Fj4+fnh7NmzOlsnPz8/HD9+HN988w0uXLiAefPm4fTp0zprvzKGDh2K7777Drdu3UK3bt3w7rvvwsfHB/Xq1cOePXvg4eGBl156CdnZ2eUKxqfZvn07XnjhBZ3HPGbMGJiZmWHSpEn47bffsGXLFnz++eeYMWOG1nQHDx7U9LJCBNzvqad79+5V6k2ue/fufF6IiMjAPHgceefOHYwbNw6hoaEYOHAgoqOjsWXLFnz//fea6b28vHDkyJEnvlvu7t27mDZtGhISEnD16lUcOnQIx44d09ya1rZtW8THx+Pw4cPIyMjA5MmTtW7tB+4fC27YsAEHDx7EqVOnMGHCBK3HMp599ln06dMHQUFBiI+Px+XLl/Hjjz/ip59+AgDMnj0bhw8fxrRp05Ceno4LFy5gx44d5Yq72uS5555Dhw4dMG7cOPz666/4+eef8cEHH2Dq1KmoV6+eZrqDBw+idevWWo9iGBqdFkMmJiaYNm0aFi9ejN27dyM3Nxfe3t5wdHTU/GzZsgUA0LlzZ7i7u2Pr1q1PbLOkpARTp06FSqXCgAED8Mwzz5S7Z7TMsmXL8OKLL8LPz++x1XllBQQE4MMPP8Q777yD7t27486dOxg/frxO2q4sFxcXjBw5EqNHj0Z+fj4+/PBD/Pvvv/jrr78QGxuL3bt3IycnR9OVdmXcvn1bLw+/WVtbY8+ePbh8+TI8PDwwc+ZMzJ07t9wLbTdv3oxXX31V58snwxYaGlql3uQedSsCERHVbg8eR4aGhsLS0hIff/wxgPvHjR9//DEmT56suVrh7e0NExMTTW/Aj2JsbIybN29i/PjxeOaZZ/Diiy9i4MCB+OijjwAAH3zwAdzd3REQEABfX184ODiUe6XKe++9h2effRZDhgzB4MGDMWzYsHIH/z/88AO6d++O0aNHo0OHDnjnnXc0RVqXLl2QmJiI8+fPw8fHB25ubpg7d26FOhKTirGxMeLi4mBsbAwvLy8EBwdj/Pjx5R572bRpk+EfvwkJxcXFCZVKJUpKSio1X2pqqgAgUlNT9RRZ9R04cEDoY/MWFBSIoUOHCpVKJTZt2iRycnKEEEJkZ2eLyMhI0bFjR/HHH39UqW2ptuvu3buFSqUSRUVFNbpcqv2Ki4vFsGHDhJGRkQDw1B8jIyMxfPhwUVxcLHXoRAanb9++YsyYMVKHQfRUDx6vrFy5Ujz33HNSh6QT+jp2rKjKHgeePn1aNGnSRHMsaqh0emWosgYPHozXXnsN165dkzIMvfD29tbLS03NzMwQExODd955B59++ilsbGxQr1492NnZYePGjVixYgWaN2+u8+XqU15eHr7++muYmFSrPw+qg4yNjfHtt99i6NChAPDYW+bKhg8dOhTR0dEG36skERFVzOTJk9GnT5860YOovo4d9UWtVuObb77RvCbFUElaDAH3nwmqystNazszMzOt3jaio6M17156+KeynQYoFAqEhITgxIkTuHPnDi5cuIB///0X+/fvh5+fn65XBQMHDnxs7AqF4rHjyi5tP82IESPQo0cPncdNdYOFhQW+//57bNiw4bEvavbw8MDGjRvx/fff87kzIiIZMTExwZw5c+pED6K6PnacMmXKY+d/1LjevXsDQIWP3/z9/REQEFC1la1FeCq+hgwdOvSxB/zVeY/Pgy+31Zd169ZpOr54mIWFxWPHlb08jKi6jI2NERwcjODgYKSnp2Py5Mm4efMm3nnnHXh6erKzBCIiqnOqe+w4f/58rRfKPqhBgwblxv32228YNmwYpkyZUvlgDRiLoRpSv359gz1r8WA350RS69q1K1xcXGBpaVmuIw4iIqK6orrHjk2aNNHqXvxR4x/077//ApDfyWzJb5MjIiIiIiKSAoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREsmTQ7xnKyMiQOoQ6hduTiIiIDAWPW3RLrtvTIIshW1tbKJVKBAcHSx1KnaNUKmFrayt1GERPZGpqinr16kkdBhERSYDHgfojx+NAgyyGnJyckJGRgaysLKlDeaqlS5fi0KFD+OGHH6QOpUJsbW3h5OQkdRhET1RUVISCggKpwyAiIgkY0nFgUFAQevXqhRkzZkgdSoXI8TjQIIsh4H4iGMJ/lr29PczNzeHu7i51KERERER1gqEcB5qbm8Pe3p7HgbUYO1AgIiIiIiJZYjFERERERESyxGKIiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDdciVK1egUCiQnp7+2GkSEhKgUCiQk5NTY3ERGTLmFRERGQLur6qGxZDMeHt7Q61Ww9raGgAQFRUFGxsbaYMiMnDMKyIiMgTcX5VnsO8ZoqoxMzODg4OD1GEQ1SnMKyIiMgTcX5XHK0O1SF5eHsaPHw8rKys4OjoiIiICvr6+mD59OgBAoVAgJiZGax4bGxtERUVpDTt79iy8vb1hbm6OTp06ITExUTPuwcujCQkJmDhxIm7fvg2FQgGFQoHw8HD9riRRDWNeERGRIeD+ShoshmqRWbNmITExETt27MCePXuQkJCAtLS0KrUzc+ZMnDhxAl5eXggMDMTNmzfLTeft7Y3ly5ejQYMGUKvVUKvVCAsL08WqENUazCsiIjIE3F9Jg8VQLZGbm4vIyEgsWbIE/fr1Q+fOnbF+/XoUFxdXuq1p06YhKCgIKpUKq1evhrW1NSIjI8tNZ2ZmBmtraygUCjg4OMDBwQFWVla6WB2iWoF5RUREhoD7K+mwGKolLl26hMLCQvTo0UMzrFGjRmjXrl2l2/Ly8tL8bmJigm7duiEjI0MncRIZEuYVEREZAu6vpMNiyIAoFAoIIbSGFRUVSRQNUd3AvCIiIkPA/ZV+sBiqJdq0aQNTU1MkJydrhmVnZ+P8+fOaz3Z2dlCr1ZrPFy5cQH5+frm2jh49qvm9uLgYqampUKlUj1yumZkZSkpKdLEKRLUO84qIiAwB91fSYdfatYSVlRUmTZqEWbNmoXHjxmjSpAnmzJkDI6P/1at+fn5YuXIlvLy8UFJSgtmzZ8PU1LRcW6tWrULbtm2hUqmwbNkyZGdn4+WXX37kcp2dnZGbm4t9+/bB1dUVSqUSSqVSb+tJVJOYV0REZAi4v5IOrwzVIp999hl8fHwQGBgIf39/9O7dGx4eHprxERERaNGiBXx8fDBmzBiEhYU98g920aJFWLRoEVxdXZGUlITY2FjY2to+cpne3t6YMmUKRo0aBTs7OyxevFhv60ckBeYVEREZAu6vpKEQD998SDo1c+ZM7N69u8oPrvn6+qJr165Yvny5bgMjMmBjx46FWq3G/v37qzQ/84qoPD8/Pzg6OiI6OlrqUIjqDJVKhUGDBiEiIqJK83N/pX+8MkRERERERLLEYoiIiIiIiGSJHSjUcgkJCVKHQFTnMK+IiMgQcH+lf7wyREREREREssRiiIiIiIiIZInFEBERERERyRKLISIiIiIikiUWQ0REREREJEsshoiIiIiISJZYDBERERERkSyxGCIiIiIiIlliMURERERERLLEYoiIiIiIiGSJxRAREREREckSiyEiIiIiIpIlE6kDqKrMzExkZWVJHcZT3bhxA/fu3UNaWprUoVSIra0tnJycpA6DJGIoeXXr1i3cuXOHeUVEJFOGsr+6d+8ebty4wf1VLaYQQgipg6iszMxMqFQq5OfnSx1KnaNUKpGRkSG7RCDmlT4xr8gQ+Pn5wdHREdHR0VKHQvRE3F/pjxz3VwZ5ZSgrKwv5+fnYuHEjVCqV1OHUGRkZGQgODkZWVpaskoDuY17pB/OKiEi3uL/SD7nurwyyGCqjUqng7u4udRhEdQrzioiIDAH3V6QL7ECBiIiIiIhkicUQERERERHJEoshIiIiIiKSJRZDREREREQkSyyGiIiIiIhIllgMERERERGRLLEYIiIiIiIiWWIxREREREREssRiiIiIiIiIZInFEBEREREZpJs3b6JJkya4cuWK1KHIzpkzZ9C8eXPk5eVJHUq1sBjSg5CQEAwbNqzc8ISEBCgUCuTk5NR4TESGjnlFRFS3hISEQKFQQKFQwNTUFK1atcI777yDe/fuVbiNhQsX4vnnn4ezs7P+Aq0hV65cgUKhqDWF3cKFC+Ht7Q2lUgkbG5ty4zt06ICePXti6dKlNR+cDrEYIiIiIiJJDBgwAGq1Gr///juWLVuGtWvXYt68eRWa9+7du4iMjMSkSZP0HKU8FRYWYuTIkXj99dcfO83EiROxevVqFBcX12BkusViiIiIiIgkUa9ePTg4OKBFixYYNmwY/P39ER8fDwAIDw/XXDl68Cc2NhYAcOjQIdSrVw89e/Z84jKuXr2KwMBANGzYEJaWlujYsSN2794NACgpKcGkSZPQqlUrWFhYoF27dvj888+15vf19cX06dO1hg0bNgwhISGazwUFBZg9ezZatGiBevXqwcXFBZGRkZrxp0+fxsCBA2FlZQV7e3uMGzcOWVlZAP53haxVq1YAgFatWkGhUCA8PPyp28/Z2Rkff/wxXn75ZdSvXx9OTk748ssvnzpfRXz00Ud4++230blz58dO079/f9y6dQuJiYk6WaYUWAwRERERkeROnz6Nw4cPw8zMDAAQFhYGtVqt+VmyZAmUSiU6dOgAADhx4gQ8PDye2u7UqVNRUFCAX375BadOncKnn34KKysrAEBpaSmaN2+O7777DmfOnMHcuXPx/vvvY+vWrZWKffz48di0aRNWrFiBjIwMrF27VrOMnJwc+Pn5wc3NDcePH8dPP/2EGzdu4MUXXwQAfP7551Cr1UhJSQEApKSkQK1WIywsrELLjoiIQLdu3XDixAm88cYbeP3113Hu3LlKxV9VZmZm6Nq1Kw4ePFgjy9MHE6kDqKvi4uI0SVCmpKREomiI6gbmFRFR3VL2vV5cXIyCggIYGRlh5cqVAAArKyvNd/7Ro0fxwQcfYP369XBxcQEAqNVqtGnT5qnLyMzMRFBQkOYKR+vWrTXjTE1N8dFHH2k+t2rVCkeOHMHWrVs1xcrTnD9/Hlu3bkV8fDz8/f3LLWPlypVwc3PDxx9/rBn21VdfoUWLFjh//jyeeeYZWFtba56VsrOzg4ODQ4WWDQCDBg3CG2+8AQCYPXs2li1bhgMHDqBdu3YVbqM6mjZtiqtXr9bIsvSBxZCe9O3bF6tXr9YalpycjODgYIkiIjJ8zCsiorql7Hs9Ly8Py5Ytg4mJCYKCgrSmyczMxLBhwxAWFoYXX3wRaWlpAO7fmmZubq41bceOHTUH5j4+Pvjxxx//X3t3HlZF3f9//HVAERDFDAUsTVNTKndvEMQFcl9RK0txL3OrvF3LJc2yNPfK7TINK+5bK819IxOTxYNL5FfFNLIsv0fLpRKXUJjfH/3kG2mKLA6HeT6ui+vizPL5vOfoMOd1ZuYzeuGFFzR48GBt27ZNLVq0ULdu3VS7du2sdebPn69ly5bpxIkTunz5stLT01W3bt0cb0NycrJcXV3VrFmzm87/+uuvtWPHjhu+zJOk1NRUPfTQQznu62b+ui02m01+fn76+eef89TmnfDw8NClS5fuWn/5jTBUQEqWLJn1zcV1P/30k0nVAEUD+xUAFC1//bu+bNky1alTJ9ugCBcvXlSnTp0UHBysKVOmZFu3TJkyOn/+fLZpmzZt0tWrVyX9+SFdkp555hm1bt1aGzdu1LZt2/Tmm29q1qxZev7557VixQqNGjVKs2bNUnBwsEqVKqUZM2bIbrdnteni4iLDMLL1c72Pv/bzT9LS0tSxY0dNnz79hnn+/v63XDcnihcvnu21zWZTZmZmntvNqXPnzuXoDF1hxT1DAAAAMJ2Li4vGjRunCRMm6PLlyzIMQ5GRkcrMzNSHH34om82WbfkaNWro8OHD2aY98MADqlatmqpVq6b77rsva3rFihU1aNAgrV69WiNHjtSSJUsk/TkIQ0hIiIYMGaJ69eqpWrVqSk1NzdZmuXLl5HA4sl5nZGTo4MGDWa9r1aqlzMzMfxxEoH79+jp06JAqV66cVdv1n5IlS+buzSpEDh48qHr16pldRq4RhgAAAFAoPPHEE3J1ddX8+fM1efJkff7551q8eLHS0tJ06tQpnTp1KuvemuDgYB06dOiGs0N/N3z4cG3dulXHjx/X/v37tWPHDgUEBEiSqlevrr1792rr1q06evSoJk6cqD179mRbPzw8XBs3btTGjRt15MgRDR48ONuz7SpXrqw+ffqof//+WrNmjY4fP67Y2NisQRiGDh2qc+fO6emnn9aePXuUmpqqrVu3ql+/foX6vtcTJ04oOTlZJ06cUEZGhpKTk5WcnKy0tLSsZb7//nudPHky614pZ0QYAgAAQKFQrFgxDRs2TG+99ZY2bdqktLQ0hYSEyN/fP+tn27Ztkv4MMvXr17/tyG8ZGRkaOnSoAgIC1KZNGz300ENasGCBJOm5555T165d1b17dwUFBens2bNZgxFc179/f/Xp00e9e/dWs2bN9OCDDyosLCzbMgsXLtTjjz+uIUOGqGbNmnr22Wd18eJFSX8OMBAfH6+MjAy1atVKtWrV0vDhw1WmTBm5uBTej+KvvPKK6tWrp0mTJiktLU316tXLGhHvuv/+979q1aqVHnjgARMrzRub8feLIJ3A/v371aBBA+3bt0/169c3u5wig/fV2vj3Lxi8r3AW4eHh8vf3V3R0tNmlALf017+rDodDo0eP1sGDBwt1sMiJ77//XlWqVNHx48dVuXLlu97/nR6v0tPTVb16df3nP/9R48aN70KFBYMBFAAAAOCU2rdvr2PHjunkyZOqWLGi2eXkScWKFeVwOFSuXDmzS8mREydOaNy4cU4dhCTCEAAAAJzY8OHDzS4hX7i6umZ7vtCuXbvUtm3bf1z+r/fu3Mwbb7yR7dlGf3Xx4sUbBm+4PgLd888/r/j4+NvWe30QCGdHGAIAAAAKmYYNGyo5OTnX6w8aNOgfHxzr4eGhy5cvZ5t26NAhRUREaOLEibnu0xkRhgAAAIBCxsPDI09nXsqWLauyZcvmePnff/9dklS+fPlc9+mMnPtOMwAAAADIJcIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJKd+zlBKSorZJRQpvJ+Q+H+Q33g/AaBg8Pc1f1n1/XTKMOTj4yNPT09FRkaaXUqR4+npKR8fH7PLgAnYrwoO+xUA5B+OVwXHiscrpwxDlSpVUkpKis6cOWN2Kbc1e/ZsxcfHa9WqVWaXkiM+Pj6qVKmS2WXABM60X40fP15nzpzR4sWLzS4lR9ivACD/ONPxqlu3bmrcuLFGjBhhdik5YsXjlVOGIenPHcEZ/rF8fX3l7u6u+vXrm10KcFvOsl+VLVtWf/zxB/sVAFiUsxyv3N3d5evry/GqEGMABQAAAACWRBgCAAAAYEmEIQAAAACWRBgCAAAAYEmEIQAAAACWRBgqQr7//nvZbDYlJyf/4zKxsbGy2Wz69ddf71pdgDNjvwIAOAOOV7lDGLKYkJAQORwOeXt7S5KioqJUpkwZc4sCnBz7FQDAGXC8upHTPmcIuePm5iY/Pz+zywCKFPYrAIAz4Hh1I84MFSIXL15U79695eXlJX9/f82aNUvNmzfX8OHDJUk2m01r1qzJtk6ZMmUUFRWVbdqRI0cUEhIid3d3Pfroo9q5c2fWvL+eHo2NjVW/fv3022+/yWazyWazafLkyQW7kcBdxn4FAHAGHK/MQRgqREaPHq2dO3dq7dq12rZtm2JjY7V///5ctTNy5Eh99dVXCg4OVseOHXX27NkblgsJCdHcuXNVunRpORwOORwOjRo1Kj82BSg02K8AAM6A45U5CEOFRFpampYuXaqZM2fqscceU61atbR8+XJdu3btjtsaNmyYunXrpoCAAC1cuFDe3t5aunTpDcu5ubnJ29tbNptNfn5+8vPzk5eXV35sDlAosF8BAJwBxyvzEIYKidTUVKWnpysoKChrWtmyZVWjRo07bis4ODjr92LFiqlhw4ZKSUnJlzoBZ8J+BQBwBhyvzEMYciI2m02GYWSbdvXqVZOqAYoG9isAgDPgeFUwCEOFRNWqVVW8eHHZ7fasaefPn9fRo0ezXpcrV04OhyPr9bFjx3Tp0qUb2tq9e3fW79euXdO+ffsUEBBw037d3NyUkZGRH5sAFDrsVwAAZ8DxyjwMrV1IeHl5acCAARo9erTuvfdelS9fXuPHj5eLy//l1fDwcL377rsKDg5WRkaGxo4dq+LFi9/Q1vz581W9enUFBARozpw5On/+vPr373/TfitXrqy0tDRt375dderUkaenpzw9PQtsO4G7if0KAOAMOF6ZhzNDhciMGTPUpEkTdezYUS1atFBoaKgaNGiQNX/WrFmqWLGimjRpoh49emjUqFE3/Q87bdo0TZs2TXXq1FFcXJzWrVsnHx+fm/YZEhKiQYMGqXv37ipXrpzeeuutAts+wAzsVwAAZ8Dxyhw24+8XHyJfjRw5Ups2bcr1jWvNmzdX3bp1NXfu3PwtDHBiPXv2lMPh0BdffJGr9dmvgBuFh4fL399f0dHRZpcCFBkBAQFq166dZs2alav1OV4VPM4MAQAAALAkwhAAAAAAS2IAhUIuNjbW7BKAIof9CgDgDDheFTzODAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsiDAEAAACwJMIQAAAAAEsqZnYBubV06VKlpqaaXcZtxcXF6ZdfftG4cePMLuW2bDabJk+erOLFi5tdCkwyd+5c/fzzz2aXcVvJyclKS0tziv3K09NT48ePl81mM7sUACgyTpw4oTNnzphdxm1duXJFp0+f1v79+80uJUd8fHxUqVIls8u4q2yGYRhmF3Gnrl27pvLly+v8+fMqV66cvLy8zC7plgzDKNQfhDIyMnTixAlJf4a3xo0bm1wRzPDbb7/Jx8dH165dk5+fnzw8PMwu6ZYK+36Vnp6ukydPSpK+/fZbVa1a1eSKgFsLDw+Xv7+/oqOjzS4FuKUTJ04oICBAly5dMruUIsfT01MpKSmWCkROeWaoWLFi2rBhgyIiIlSqVClt2LBBAQEBZpfllNLS0hQZGamffvpJs2fPVkhIiNklwSTe3t5au3atnnrqKfn6+mr9+vWqWLGi2WU5pTNnzqhbt2765ZdftHTpUoIQAOSjM2fO6NKlS/roo4/4/JePUlJSFBkZqTNnzhCGnEFISIiSkpLUoUMHBQcH65NPPlHLli3NLsup/Pjjj+rYsaO+++47rV+/Xu3atTO7JJisXbt2io+PV8eOHRUYGKh169bpX//6l9llOZWUlBR16NBBFy5c0BdffMGZVgAoIAEBAapfv77ZZcDJOfUACpUrV1ZCQoJCQkLUtm1bLVy40OySnMaePXsUGBioX3/9VfHx8QQhZKlVq5aSkpJUpUoVNW3aVB9//LHZJTmNmJgYBQcHy8PDQ0lJSQQhAAAKOacOQ5JUunRprVu3TsOGDdOQIUP0wgsv6Nq1a2aXVah9/PHHatq0qSpXriy73a5atWqZXRIKmfLly+uLL75Q165d1b17d73++utywtsL76qFCxeqbdu2CgkJUUJCgipXrmx2SQAA4DacPgxJf95DNHfuXC1YsEALFixQp06d9Pvvv5tdVqFjGIZef/11de/eXV26dNGOHTvk6+trdlkopNzd3fXRRx9pypQpmjhxonr16qUrV66YXVahc+3aNb344osaMmSIhg4dqnXr1ql06dJmlwUAAHKgSISh6wYPHqzNmzdnXTp3/Phxs0sqNK5cuaJevXpp4sSJmjJliqKjo+Xu7m52WSjkbDabJk6cqJUrV2rVqlV67LHHnGLo7bvl999/V6dOnTR//nwtWLBA8+bNU7FiTnsrJgAAllOkwpAktWzZUrt379aVK1cUFBSkhIQEs0sy3c8//6zHHntMq1at0sqVKzVx4sRCPSQxCp8nn3xSO3fuVGpqqoKCgnTw4EGzSzLd8ePHsy6J27x5swYPHmx2SQAA4A4VuTAkSTVr1pTdbldAQIDCwsIs/cyEgwcPKigoSKmpqdq5c6eefPJJs0uCkwoMDFRSUpK8vb0VEhKizZs3m12SaRISEhQUFKTLly8rMTGRkSwBAHBSRTIMSdK9996rmJgY9ezZU5GRkZo4caIyMzPNLuuu2rx5s0JCQuTt7a2kpCQFBgaaXRKcXKVKlRQXF6ewsDB16NBBb7/9tuUGVoiOjlZYWFi2L10AAIBzKrJhSJLc3Ny0dOlSTZ8+XVOnTlX37t0t8bRiwzD09ttvq0OHDgoLC1NcXJylHp6FguXl5aXVq1drxIgRWQMHXL161eyyClxmZqYmTpyoyMhI9ejRQzExMfLx8TG7LAAAkAdFOgxJf94APmbMGK1evVqbNm1Ss2bN5HA4zC6rwFy9elVDhgzRiy++qBEjRmj16tXy8vIyuywUMa6urpoxY4bee+89vffee2rXrp1+/fVXs8sqMJcuXVL37t01depUTZ8+XcuWLVOJEiXMLgsAAORRkQ9D10VERCguLk4Oh0OBgYH66quvzC4p3/36669q165d1gfUGTNmyNXV1eyyUIQNGDBAMTEx2r9/v4KDg/Xtt9+aXVK+czgcatasmTZt2qRVq1ZpzJgxDEACAIXEN998Iz8/P124cMHsUixn0aJF6tixo9ll5JllwpAk1atXT0lJSfLz81NoaKjWrFlT4H3Onz9flStXlru7u4KCgpSUlFQg/Xz77bcKDg7W/v37FRMTowEDBhRIP8DfNW/eXLt371ZmZqaCgoK0c+fOAu/zbu1XX331lQIDA+VwOBQXF6cuXboUSD8AYDV9+/aVzWaTzWZT8eLFVaVKFY0ZM+aOn2f38ssv6/nnn1epUqUKqNK7JzY2ttB82fb111/r6aefVsWKFeXh4aGAgADNmzcv2zL9+/fX/v37tWvXLpOqzB+WCkOSVKFCBe3cuVPt2rVT165d9dZbbxXYDeArV67UiBEjNGnSJO3fv1916tRR69at8/05LTt37lRQUJAyMzO1e/duNW/ePF/bB26nevXq2r17t+rWrauWLVvq/fffL7C+7tZ+tWbNGoWGhsrX11dJSUmqV69evrYPAFbXpk0bORwOfffdd5ozZ44WL16sSZMm5Xh9h8OhDRs2qG/fvgVXpEXt27dP5cuX10cffaRDhw5p/Pjxevnll/Xuu+9mLePm5qYePXro7bffNrHSfGBYVEZGhjFhwgRDktGvXz/jjz/+yPc+AgMDjaFDh2brs0KFCsabb76Zb30sW7bMKF68uBEeHm6cO3cu39oFciM9Pd0YOHCgIckYM2aMkZGRke99FPR+lZmZaUyfPt2w2WzG448/bly8eDFf2gUKu7CwMKNHjx5mlwGL6NOnj9G5c+ds07p27WrUq1fPMAzDWL58uVGyZEnj6NGjWfMHDx5s1KhRw4iLizMkGS+++KLRsGHD2/aVnJxsNG/e3PDy8jJKlSpl1K9f39izZ49hGIZx5swZ46mnnjIqVKhgeHh4GI8++qjxn//8J9v6DzzwgDFnzpxs0+rUqWNMmjQp6/X58+eNgQMHGuXLlzdKlChhPPLII8b69euz5u/atcsIDQ013N3djfvvv994/vnnjbS0NMMwDKNZs2aGpBt+3n///dtumyRjyZIlRkREhOHh4WFUq1bNWLt27W3Xu5l9+/YZkox9+/bddP6QIUOMsLCwbNN27txpuLm5GZcuXcpVn4WB5c4MXefi4qLXXntNH374oaKjo9WyZUudOXMm39pPT0/Xvn371KJFi2x9tmjRQomJiXluPzMzU2PHjlX//v3Vr18/bdmyRffcc0+e2wXyonjx4lq0aJHmzJmjmTNnqmvXrkpLS8u39gt6v0pPT9eAAQM0duxYjRs3TitXrpSnp2ee2wUA3NrBgweVkJAgNzc3SVLv3r3Vrl079ezZU9euXdPGjRv13nvvKTo6Wh4eHpKk5ORkNWzY8LZt9+zZU/fff7/27Nmjffv26aWXXlLx4sUlSVeuXFGDBg20ceNGHTx4UAMHDlSvXr3u6PLrzMxMtW3bVvHx8froo490+PBhTZs2Leu+7dTUVLVp00bdunXTgQMHtHLlSsXFxWnYsGGSpNWrV8vhcGjVqlWS/jzj5XA41L179xz1/+qrr+rJJ5/UgQMHst6zc+fO5bj+nPrtt99UtmzZbNMaNmyoa9euyW6353t/d43ZaawwiIuLM8qVK2dUrVrVSElJyZc2T548aUgyEhISsk0fPXq0ERgYmKe209LSjIiICMNmsxlz5swxMjMz89QeUBA2bNhgeHl5GXXr1jV+/PHHfGmzIPerM2fOGE2bNjXc3NyMDz74IE9tAc6IM0O4m/r06WO4uroaJUuWNEqUKGFIMlxcXIxPP/00a5lz584Z999/vzF48GDD19fXmDp1qmEY/3cG46GHHjKmTJly275KlSplREVF5bi29u3bGyNHjsx6fbszQ1u3bjVcXFyMb7755qbtDRgwwBg4cGC2abt27TJcXFyMy5cvZ03bsWOHcacfzSUZEyZMyHqdlpZmSDI2b958R+0Yxq3PDMXHxxvFihUztm7desO8e+65547e38LGsmeG/qpx48ay2+0qUaKEGjVqpM8//9zskv7RTz/9pCZNmujzzz/XunXrNHz48EJzsx3wV+3bt1d8fLzOnTunwMBA7d271+yS/tGRI0cUFBSkw4cP64svvlCvXr3MLgkAirywsDAlJyfLbrerT58+6tevn7p165Y1/5577tHSpUu1cOFCVa1aVS+99FK29f/44w+5u7tnm+bl5ZX1M2jQIEnSiBEj9Mwzz6hFixaaNm2aUlNTs5bPyMjQa6+9plq1aqls2bLy8vLS1q1bdeLEiRxvR3Jysu6//3499NBDN53/9ddfKyoqKlttrVu3VmZmpo4fP57jfv5J7dq1s34vWbKkSpcuna/30R48eFCdO3fWpEmT1KpVqxvme3h4OPVzPAlD/1+VKlWUkJCg4OBgtWnTRosWLcpTez4+PnJ1ddXp06ezTT99+rT8/Pxy1ebevXsVGBios2fPKj4+Xh06dMhTjUBBq127tpKSkvTAAw+oadOm+vTTT/PUXkHsV59//rkaNWqkEiVKKCkpSY0bN85TjQCAnClZsqSqVaumOnXqaNmyZbLb7Vq6dGm2Zb788ku5urrK4XDo4sWL2eaVKVNG58+fzzYtOTk562fKlCmSpMmTJ+vQoUNq3769vvjiCz388MP67LPPJEkzZszQvHnzNHbsWO3YsUPJyclq3bq10tPTs9p0cXG5YbCtvz5s/Pple/8kLS1Nzz33XLbavv76ax07dkxVq1bN4bv1z65f8nedzWZTZmZmntuVpMOHD+uxxx7TwIEDNWHChJsuc+7cOZUrVy5f+jMDYegvvL29tX79eg0dOlSDBw/W8OHDlZGRkau23Nzc1KBBA23fvj1rWmZmprZv367g4OA7bu/TTz9V06ZN9cADDygpKSnbtwBAYebr66sdO3YoIiJCTzzxhKZOnZrrERzze79atGiR2rRpo+DgYCUkJKhKlSq5qgsAkDcuLi4aN26cJkyYoMuXL0uSEhISNH36dK1fv15eXl5Z99hcV6NGDR0+fDjbtGrVqmX9lC9fPmv6Qw89pH//+9/atm2bunbtmjXqaXx8vDp37qzIyEjVqVNHDz74oI4ePZqtzXLlysnhcGS9/v3337Od0aldu7Z++umnG9a7rn79+jp8+HC22q7/XL9HqjA6dOiQwsLC1KdPH02dOvWmy6SmpurKlStOPeIqYehvihUrpnnz5mn+/Pl699131alTJ/3++++5amvEiBFasmSJli9frpSUFA0ePFgXL15Uv379ctyGYRiaOnWqnnjiCUVERGjHjh3y9fXNVT2AWdzd3RUdHa1XX31VEyZMUO/evfXHH3/kqq382K8yMjI0fPhwDR48WEOHDtX69evl7e2dq3oAAPnjiSeekKurq+bPn68LFy6oV69eeuGFF9S2bVtFR0dr5cqV2a4wCA4OVmJi4i2/uL58+bKGDRum2NhY/fDDD4qPj9eePXsUEBAg6c9HQ8TExCghIUEpKSl67rnnbrj6IDw8XB9++KF27dql//mf/1GfPn2yPdS+WbNmatq0qbp166aYmBgdP35cmzdv1pYtWyRJY8eOVUJCgoYNG6bk5GQdO3ZMa9euvSHcFSYHDx5UWFiYWrVqpREjRujUqVM6deqUfvnll2zL7dq1Sw8++GC+nOEyjcn3LBVq27ZtM7y9vY1HH33UOH78eK7aeOedd4xKlSoZbm5uRmBgoLF79+4cr3vlyhUjMjLSkGS8+uqrDJSAImHFihWGu7u7ERISYpw+fTpXbeRlv/rtt9+Mdu3aGa6ursb8+fNz1T9QFDGAAu6mmw2tbRiG8eabbxrlypUz+vbta9SqVcu4cuVK1rxZs2YZZcuWNTZv3mxIMux2u1GhQgVjy5Yt/9jPH3/8YTz11FNGxYoVDTc3N6NChQrGsGHDsgYuOHv2rNG5c2fDy8vLKF++vDFhwgSjd+/e2Wr77bffjO7duxulS5c2KlasaERFRd0wtPbZs2eNfv36Gffee6/h7u5uPProo8aGDRuy5iclJRktW7Y0vLy8jJIlSxq1a9fOGhDiutwOoPDZZ59lm+bt7Z2jYbn/7q8DKEyaNOmmw30/8MAD2dZp1apVvj4yxgw2wyigJ44WESkpKerQoYMuXLigNWvWKCQk5K70+/PPP6tLly7av3+/oqKicjy8IuAM7Ha7OnfuLA8PD23YsEGPPPLIXen3+++/V8eOHfXjjz/qk08+UcuWLe9Kv4AzCA8Pl7+/v6Kjo80uBbil/fv3q0GDBtq3b58SExO1bt06bd261eyy8iw2NlZhYWG5vpQ8r/76vtavX/+2yx86dEjh4eE6evSoU19dwWVytxEQECC73a6aNWsqLCzsrhwkDh06pKCgIKWmpio2NpYghCInKChISUlJKl26tIKDg7MuJShICQkJCgwM1KVLl5SYmEgQAoAi4LnnnlPTpk114cIFs0vJs5CQkGz3JhV2DodDH3zwgVMHIYkwlCM+Pj6KiYlRjx49FBkZqVdeeSXfRun4uy1btig4OFilS5dWUlKSgoKCCqQfwGyVKlVSXFycmjdvrvbt2+udd94psG/DoqOjFRYWppo1a8put2ddKw4AcG7FihXT+PHjVapUKbNLyTM3N7dsI6NGR0dnG477rz85uaJi0KBB/7j+zeaFhoZKkt54440c1duiRQu1bt06dxtbiBQzuwBnUaJECS1btkwBAQF66aWXdOTIEUVFReXb0+kNw9C7776r4cOHq3379oqOji4SOzZwK6VKldJnn32msWPH6oUXXlBKSormzZt3wzChuZWZmanJkyfrtddeU9++fbVo0SKVKFEiX9oGAKAgderU6R+/FM/JcXLKlCkaNWrUTeeVLl36hnmHDh1SRERE1vOZrIIwdAdsNpvGjBmj6tWrKzIyUs2bN9fatWvl7++fp3avXr2qF198UQsXLtTIkSM1ffr0bKOUAEWZq6urZs6cqZo1a2rw4MH69ttv9fHHH6tMmTJ5avfSpUvq27evPvnkE02bNk1jxozhAcUAAKdRqlSpPH0xXr58+WzDi99s/l9dHz25bNmyue7TGXGZXC506dJFcXFx+t///V8FBgYqOTk51239+uuvat++vZYsWaIlS5Zo5syZBCFY0jPPPKNt27Zp7969Cg4OzvaE8DvlcDjUvHlzbdy4UatXr9bYsWMJQgAA4AaEoVyqV6+ekpKS5Ovrq9DQUK1bt+6O20hNTVVwcLD27t2rbdu26ZlnnimASgHnERYWJrvdrszMTAUFBenLL7+84zaSk5MVGBiokydPateuXerSpUsBVAoAAIoCwlAeVKhQQV9++aXatm2riIgIzZw5M8c3gO/atUtBQUHKzMyU3W5XWFhYAVcLOIfq1atr9+7dqlOnjlq0aKGoqKgcr7tu3TqFhobK19dXSUlJORoaFAAAWBdhKI88PT21cuVKjRs3TqNHj9azzz6r9PT0W64TFRWlxx57TLVr11ZiYqKqV69+l6oFnMM999yjLVu2qG/fvurXr59efvnlW47gaBiGZs6cqYiICLVp00Zffvml7rvvvrtYMQAAcEaEoXzg4uKi119/XR988IE+/PBDtWrVSmfPnr1huczMTL388svq16+f+vbtq61bt1ruJjUgp4oXL67Fixdr9uzZmj59uh5//HFdvHjxhuXS09P17LPPavTo0Xr55Zf18ccf59sojwAAoGgjDOWjXr16afv27Tp06JAaNWqkb775JmvexYsX9fjjj2v69OmaPXu2Fi9enG/DBwNFlc1m07///W+tXbtWMTExatKkiU6ePJk1/+zZs2rVqpU+/PBDffDBB5o6dapcXPizBgAAcoZPDfksNDRUdrtdbm5uatSokbZv366TJ0+qSZMmiomJ0dq1a/Xvf/+bka2AO9CxY0fFx8fr7Nmz+te//qW9e/fqm2++UaNGjXTo0CFt375dvXr1MrtMAADgZHjOUAF48MEHlZCQoO7du6t169YqW7asPDw8FB8fr9q1a5tdHuCUateuLbvdroiICDVt2lQlSpRQhQoVZLfb9eCDD5pdHgDgLktJSTG7hCLFqu8nYaiAeHt7a8OGDRoxYoSSkpK0Zs0a+fn5mV0W4NT8/Py0Y8cO9e/fX+fPn9fKlSvl7e1tdlkAgLvIx8dHnp6eioyMNLuUIsfT01M+Pj5ml3FX2YycjgWNXLt27ZqKFSN3AvnFMAxlZGSwXwH5KDw8XP7+/oqOjja7FOC2Tpw4oTNnzphdxm1169ZNjRs31ogRI8wuJUd8fHxUqVIls8u4q/gkcRfwgQ3IXzabjf0KACysUqVKTvGh3d3dXb6+vjz3rhBjAAUAAAAAlkQYAgAAAGBJhCEAAAAAlkQYAgAAAGBJhCEAAAAAlkQYAgAAAJzc999/L5vNpuTk5H9cJjY2VjabTb/++utdq6uwIwwBAAAAFhASEiKHw5H1wPKoqCiVKVPG3KJMxoM6AAAAAAtwc3OTn5+f2WUUKpwZAgAAAEx28eJF9e7dW15eXvL399esWbPUvHlzDR8+XNKfDxxfs2ZNtnXKlCmjqKiobNOOHDmikJAQubu769FHH9XOnTuz5v31MrnY2Fj169dPv/32m2w2m2w2myZPnlywG1kIEYYAAAAAk40ePVo7d+7U2rVrtW3bNsXGxmr//v25amfkyJH66quvFBwcrI4dO+rs2bM3LBcSEqK5c+eqdOnScjgccjgcGjVqVH5silMhDAEAAAAmSktL09KlSzVz5kw99thjqlWrlpYvX65r167dcVvDhg1Tt27dFBAQoIULF8rb21tLly69YTk3Nzd5e3vLZrPJz89Pfn5+8vLyyo/NcSqEIQAAAMBEqampSk9PV1BQUNa0smXLqkaNGnfcVnBwcNbvxYoVU8OGDZWSkpIvdRZFhCEAAACgkLPZbDIMI9u0q1evmlRN0UEYAgAAAExUtWpVFS9eXHa7PWva+fPndfTo0azX5cqVk8PhyHp97NgxXbp06Ya2du/enfX7tWvXtG/fPgUEBNy0Xzc3N2VkZOTHJjgthtYGAAAATOTl5aUBAwZo9OjRuvfee1W+fHmNHz9eLi7/d94iPDxc7777roKDg5WRkaGxY8eqePHiN7Q1f/58Va9eXQEBAZozZ47Onz+v/v3737TfypUrKy0tTdu3b1edOnXk6ekpT0/PAtvOwogzQwAAAIDJZsyYoSZNmqhjx45q0aKFQkND1aBBg6z5s2bNUsWKFdWkSRP16NFDo0aNumlwmTZtmqZNm6Y6deooLi5O69atk4+Pz037DAkJ0aBBg9S9e3eVK1dOb731VoFtX2FlM/5+8SEAALCc8PBw+fv7Kzo62uxSgCIjICBA7dq106xZs3K1fvPmzVW3bl3NnTs3fwtDFs4MAQAAALAkwhAAAAAAS2IABQAAAKAQio2NNbuEIo8zQwAAAAAsiTAEAAAAwJIIQwAAAAAsiTAEAAAAwJIIQwAAAAAsiTAEAAAAwJIIQwAAAAAsiTAEAAAAwJIIQwAAAAAsiTAEAAAAwJIIQwAAAAAsiTAEAAAAwJKKmV0AAAAAkFPXrl3TvHnzZBiG2aXc1tmzZ7V3717NnDnT7FJuq3Llynr88cfNLuOusxnO8D8JAAAUqPDwcPn7+ys6OtrsUoBbiouLU5MmTSRJpUuXls1mM7ki53b16lVdunRJZcqU0S+//KJixax1rsRaWwsAAACnFhISookTJ+q1115T165dtXjxYrm5uZldllP64Ycf1LFjR/3www/673//a7kgJBGGAAAA4ERcXFw0ZcoU1ahRQ/3799d3332nVatWycfHx+zSnMru3bvVuXNnlSxZUomJiXr44YfNLskUDKAAAAAAp9OzZ0/t2LFDKSkpCgoKUkpKitklOY0VK1aoefPmql69uux2u2WDkEQYAgAAgJMKCQlRUlKSPDw8FBwcrJiYGLNLKtQMw9DkyZP19NNPq3v37tq+fbvKlStndlmmIgwBAADAaVWuXFkJCQkKCQlR27ZttXDhQrNLKpQuX76sHj166NVXX9Ubb7yhqKgolShRwuyyTMc9QwAAAHBqpUuX1rp16zRq1CgNGTJEKSkpmj17tiUHBLiZU6dOKSIiQgcOHNCnn36qbt26mV1SocH/EAAAADi9YsWKae7cuapZs6aGDRumY8eOacWKFfL29ja7NFMdOHBAHTp0UEZGhnbt2qUGDRqYXVKhwmVyAAAAKDIGDRqkLVu2KDExUY0bN9bx48fNLsk069evV0hIiHx8fJSUlEQQugnCEAAAAIqUFi1aaPfu3bpy5YoCAwMVHx9vdkl3lWEYmj17tjp37qyWLVtq165duu+++8wuq1AiDAEAAKDIqVmzZtaw0eHh4froo4/MLumuSE9P18CBAzVy5EiNHTtWq1atUsmSJc0uq9AiDAEAAKBIuvfeexUTE6OePXuqV69emjBhgjIzM80uq8CcO3dObdq00fLly/X+++/rzTfflIsLH/dvhQEUAAAAUGS5ublp6dKlqlmzpl566SV98803Wr58uTw9Pc0uLV8dPXpUHTp00Llz5/T555+radOmZpfkFIiKAAAAKNJsNpvGjBmj1atXa9OmTWrWrJkcDkeB9jl//nxVrlxZ7u7uCgoKUlJSUoH1tWPHDjVq1Eiurq6y2+0EoTtAGAIAAIAlREREKC4uTg6HQ4GBgfrqq68KpJ+VK1dqxIgRmjRpkvbv3686deqodevW+vnnn/O9r/fee0+tWrVSw4YNlZiYqKpVq+Z7H0UZYQgAAACWUa9ePSUlJcnPz0+hoaFas2ZNvvcxe/ZsPfvss+rXr58efvhhLVq0SJ6enlq2bFm+9ZGRkaFRo0bp2Wef1cCBA7Vp0yaVKVMm39q3CsIQAAAALKVChQrauXOn2rVrp65du+qtt96SYRj50nZ6err27dunFi1aZE1zcXFRixYtlJiYmC99XLhwQREREZozZ47eeecdzZ8/X8WKMRRAbhCGAAAAYDmenp5auXKlxo8fr7Fjx2rAgAFKT0/Pc7tnzpxRRkaGfH19s0339fXVqVOn8tz+iRMnFBoaqp07d2rjxo0aNmxYntu0MiIkAAAALMnFxUWvvfaaatSooQEDBig1NVWrVq2Sj4+P2aXdlN1uV+fOneXh4aHExEQ98sgjZpfk9DgzBAAAAEuLjIzUF198ocOHD6tRo0Y6cuRIrtvy8fGRq6urTp8+nW366dOn5efnl+t2V6xYoWbNmqlq1aqy2+0EoXxCGAIAAIDlNW7cWElJSSpRooQaNWqkzz//PFftuLm5qUGDBtq+fXvWtMzMTG3fvl3BwcF33J5hGHr11Vf19NNP64knntD27dtVvnz5XNWGGxGGAAAAAElVqlRRQkKCgoOD1aZNGy1atChX7YwYMUJLlizR8uXLlZKSosGDB+vixYvq16/fHbVz+fJl9ezZU5MnT9brr7+uDz74QO7u7rmqCTfHPUMAAADA/+ft7a3169dr5MiRGjx4sFJSUjRr1qw7Gq2te/fu+uWXX/TKK6/o1KlTqlu3rrZs2XLDoAq3curUKUVEROjAgQP65JNP9Pjjj+dmc3AbNiO/xhEEAABOKzw8XP7+/oqOjja7FKDQWLBggV544QW1atVKK1asUOnSpe9KvwcOHFDHjh119epVrVu3Tg0bNrwr/VoRl8kBAAAANzFkyBBt3rxZCQkJCgkJ0fHjxwu8zw0bNqhx48YqW7askpKSCEIFjDAEAAAA/IOWLVsqMTFRly9fVlBQkBISEgqkH8MwNGfOHHXq1EktWrTQrl27dP/99xdIX/g/hCEAAADgFgICAmS321WzZk2FhYXl++WkV69e1aBBgzRixAiNGTNGq1atkpeXV772gZsjDAEAAAC34ePjo5iYGD399NOKjIzUK6+8oszMzDy3e/78ebVp00bvv/++li1bpmnTpsnFhY/odwujyQEAAAA5UKJECb3//vsKCAjQSy+9pCNHjigqKkqenp65au/YsWPq0KGDzpw5o5iYGDVr1iyfK8btEDsBAACAHLLZbBo7dqxWr16tjRs3qnnz5nI4HHfcTmxsrIKCgmSz2WS32wlCJiEMAQAAAHeoS5cu2rVrl06ePKnAwEAlJyfneN333ntPLVu2VIMGDZSYmKhq1aoVXKG4JcIQAAAAkAv169dXUlKSfH19FRoaqrVr195y+YyMDI0aNUrPPvusnnnmGW3atEn33HPPXaoWN0MYAgAAAHLpvvvu05dffqk2bdqoS5cumjFjhgzDuGG5tLQ0denSRXPmzNG8efO0YMECFS9e3ISK8VcMoAAAAADkgaenpz7++GNNnDhRY8aM0ZEjR7Rw4UK5ublJkk6cOKFOnTrpu+++04YNG9S2bVuTK8Z1hCEAAAAgj1xcXDR16lTVrFlTzzzzjFJTU7Vq1Sp9++236ty5s9zd3ZWQkKBHH33U7FLxF4QhAAAAIJ/06tVLVapUUZcuXdSgQQOdPn1a9evX12effaby5cubXR7+hnuGAAAAgHwUGhoqu92ukiVLqlu3btq+fTtBqJDizBAAAACQzx588EHt2bNHHh4estlsZpeDf0AYAgAAAAqAp6en2SXgNrhMDgAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAlEYYAAAAAWBJhCAAAAIAl2QzDMMwuAgAAmCs2NlYeHh4KCgoyuxQAuGsIQwAAAAAsicvkAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFgSYQgAAACAJRGGAAAAAFjS/wMjmnamoMZj7gAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from lambeq import AtomicType, IQPAnsatz\n", "\n", "ansatz = IQPAnsatz({AtomicType.NOUN: 1, AtomicType.SENTENCE: 1},\n", " n_layers=1, n_single_qubit_params=3)\n", "\n", "train_circuits = [ansatz(diagram) for diagram in train_diagrams]\n", "dev_circuits = [ansatz(diagram) for diagram in dev_diagrams]\n", "test_circuits = [ansatz(diagram) for diagram in test_diagrams]\n", "\n", "train_circuits[0].draw(figsize=(8, 8))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Training\n", "### Instantiate model" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "We instantiate a :py:class:`.PennyLaneModel`, by passing all diagrams to the class method :py:meth:`.PennyLaneModel.from_diagrams`. \n", "\n", "We also set `probabilities=True` so that the model outputs probabilities, rather than quantum states, which follows the behaviour of real quantum computers. \n", "\n", "Furthermore, we set `normalize=True` so that the output probabilities sum to one. This helps to prevent passing very small values to any following layers in a hybrid model." ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [], "source": [ "from lambeq import PennyLaneModel\n", "\n", "all_circuits = train_circuits + dev_circuits + test_circuits\n", "\n", "# if no backend_config is provided, the default is used, which is the same as below\n", "backend_config = {'backend': 'default.qubit'} # this is the default PennyLane simulator\n", "model = PennyLaneModel.from_diagrams(all_circuits,\n", " probabilities=True,\n", " normalize=True,\n", " backend_config=backend_config)\n", "model.initialise_weights()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Running on a real quantum computer" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "We can choose to run the model on a real quantum computer, using :term:`Qiskit` with IBMQ, or the Honeywell QAPI.\n", "\n", "To use IBM devices we have to save our IBMQ API token to the :term:`PennyLane` configuration file, as in the cell below." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [], "source": [ "import pennylane as qml\n", "\n", "qml.default_config['qiskit.ibmq.ibmqx_token'] = 'my_API_token'\n", "qml.default_config.save(qml.default_config.path)\n", "backend_config = {'backend': 'qiskit.ibmq',\n", " 'device': 'ibmq_manila',\n", " 'shots': 1000}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "if TESTING:\n", " backend_config = None" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [], "source": [ "q_model = PennyLaneModel.from_diagrams(all_circuits,\n", " probabilities=True,\n", " normalize=True,\n", " backend_config=backend_config)\n", "q_model.initialise_weights()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "To use Honeywell/Quantinuum devices we have to pass the email address of an account with access to the Honeywell/Quantinuum QAPI to the :term:`PennyLane` configuration file.\n", "\n", "The first time you run a circuit on a Honeywell device, you will be prompted to enter your password. \n", "\n", "You can then run circuits without entering your password again for 30 days." ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [], "source": [ "qml.default_config['honeywell.global.user_email'] = ('my_Honeywell/Quantinuum_'\n", " 'account_email')\n", "qml.default_config.save(qml.default_config.path)\n", "\n", "backend_config = {'backend': 'honeywell.hqs',\n", " 'device': 'H1-1E',\n", " 'shots': 1000}" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "if TESTING:\n", " backend_config = None" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [], "source": [ "h_model = PennyLaneModel.from_diagrams(all_circuits,\n", " probabilities=True,\n", " normalize=True,\n", " backend_config=backend_config)\n", "h_model.initialise_weights()" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Running these models on a real quantum computer takes a significant amount of time as the circuits must be sent to the backend and queued, so in the remainder of this tutorial we will use `model`, which uses the default :term:`PennyLane` simulator, 'default.qubit'." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Create datasets" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "To facilitate data shuffling and batching, ``lambeq`` provides a native :py:class:`.Dataset` class. Shuffling is enabled by default, and if not specified, the batch size is set to the length of the dataset." ] }, { "cell_type": "code", "execution_count": 15, "metadata": {}, "outputs": [], "source": [ "from lambeq import Dataset\n", "\n", "train_dataset = Dataset(train_circuits,\n", " train_labels,\n", " batch_size=BATCH_SIZE)\n", "\n", "val_dataset = Dataset(dev_circuits, dev_labels)" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "Training can either by done using the :py:class:`.PytorchTrainer`, or by using native PyTorch. We give examples of both in the following section." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Define loss and evaluation metric" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "When using :py:class:`.PytorchTrainer` we first define our evaluation metrics and loss function, which in this case will be the accuracy and the mean-squared error, respectively." ] }, { "cell_type": "code", "execution_count": 16, "metadata": {}, "outputs": [], "source": [ "def acc(y_hat, y):\n", " return (torch.argmax(y_hat, dim=1) ==\n", " torch.argmax(y, dim=1)).sum().item()/len(y)\n", "\n", "def loss(y_hat, y):\n", " return torch.nn.functional.mse_loss(y_hat, y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initialise trainer" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "As :term:`PennyLane` is compatible with PyTorch autograd, :py:class:`.PytorchTrainer` can automatically use many of the PyTorch optimizers, such as Adam to train our model." ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [], "source": [ "from lambeq import PytorchTrainer\n", "\n", "trainer = PytorchTrainer(\n", " model=model,\n", " loss_function=loss,\n", " optimizer=torch.optim.Adam,\n", " learning_rate=LEARNING_RATE,\n", " epochs=EPOCHS,\n", " evaluate_functions={'acc': acc},\n", " evaluate_on_train=True,\n", " use_tensorboard=False,\n", " verbose='text',\n", " seed=SEED)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "We can now pass the datasets to the :py:meth:`~lambeq.Trainer.fit` method of the trainer to start the training." ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "Epoch 1: train/loss: 0.1207 valid/loss: 0.0919 train/time: 0.71s valid/time: 0.16s train/acc: 0.7857 valid/acc: 0.8667\n", "Epoch 2: train/loss: 0.0486 valid/loss: 0.1035 train/time: 0.50s valid/time: 0.17s train/acc: 0.9286 valid/acc: 0.9000\n", "Epoch 3: train/loss: 0.0364 valid/loss: 0.0621 train/time: 0.49s valid/time: 0.17s train/acc: 0.9429 valid/acc: 0.9333\n", "Epoch 4: train/loss: 0.0466 valid/loss: 0.0392 train/time: 0.62s valid/time: 0.18s train/acc: 0.9857 valid/acc: 1.0000\n", "Epoch 5: train/loss: 0.0120 valid/loss: 0.0126 train/time: 0.49s valid/time: 0.18s train/acc: 0.9857 valid/acc: 1.0000\n", "Epoch 6: train/loss: 0.0014 valid/loss: 0.0178 train/time: 0.49s valid/time: 0.17s train/acc: 1.0000 valid/acc: 1.0000\n", "Epoch 7: train/loss: 0.0022 valid/loss: 0.0079 train/time: 0.60s valid/time: 0.17s train/acc: 1.0000 valid/acc: 1.0000\n", "Epoch 8: train/loss: 0.0041 valid/loss: 0.0061 train/time: 0.48s valid/time: 0.17s train/acc: 1.0000 valid/acc: 1.0000\n", "Epoch 9: train/loss: 0.0003 valid/loss: 0.0108 train/time: 0.47s valid/time: 0.17s train/acc: 1.0000 valid/acc: 1.0000\n", "Epoch 10: train/loss: 0.0001 valid/loss: 0.0205 train/time: 0.60s valid/time: 0.17s train/acc: 1.0000 valid/acc: 0.9667\n", "Epoch 11: train/loss: 0.0001 valid/loss: 0.0281 train/time: 0.50s valid/time: 0.16s train/acc: 1.0000 valid/acc: 0.9667\n", "Epoch 12: train/loss: 0.0005 valid/loss: 0.0309 train/time: 0.54s valid/time: 0.22s train/acc: 1.0000 valid/acc: 0.9667\n", "Epoch 13: train/loss: 0.0004 valid/loss: 0.0314 train/time: 0.57s valid/time: 0.18s train/acc: 1.0000 valid/acc: 0.9667\n", "Epoch 14: train/loss: 0.0004 valid/loss: 0.0308 train/time: 0.52s valid/time: 0.17s train/acc: 1.0000 valid/acc: 0.9667\n", "Epoch 15: train/loss: 0.0011 valid/loss: 0.0286 train/time: 0.47s valid/time: 0.17s train/acc: 1.0000 valid/acc: 0.9667\n", "\n", "Training completed!\n", "train/time: 8.06s train/time_per_epoch: 0.54s train/time_per_step: 0.08s valid/time: 2.60s valid/time_per_eval: 0.17s\n" ] } ], "source": [ "trainer.fit(train_dataset, val_dataset)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Results\n", "\n", "Finally, we visualise the results and evaluate the model on the test data." ] }, { "cell_type": "code", "execution_count": 19, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Final test accuracy: 0.9666666666666667\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA18AAAIjCAYAAAD80aFnAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAACWZ0lEQVR4nOzdd3gU1dvG8e9ueg+QRiASmtIJUkIRRUUDKIqgNBVEXqyAGCtIE0tUikhRLD8FC4oooqKiVJUiSFcpgnRCAgGSQEjfef8IrEQSCCGbySb357r2YvfsmZlnRpzDs3PmGYthGAYiIiIiIiLiUFazAxAREREREakIlHyJiIiIiIiUAiVfIiIiIiIipUDJl4iIiIiISClQ8iUiIiIiIlIKlHyJiIiIiIiUAiVfIiIiIiIipUDJl4iIiIiISClQ8iUiIiIiIlIKlHyJlHH33XcfkZGRxVp27NixWCyWkg1IREQqjL1792KxWJg5c6bZoYiUC0q+RIrJYrEU6bV8+XKzQ3V6s2fPZvLkyWaHISJSImbOnJlvnPD09CQ8PJyYmBimTJnCyZMnzQ5RgJdffpn58+eX+nbj4+MZO3YsmzZtKvVti+NZDMMwzA5CxBl9/PHH+T5/+OGHLFq0iI8++ihf+0033URoaGixt5OdnY3NZsPDw+OSl83JySEnJwdPT89ib78suPXWW/nzzz/Zu3ev2aGIiFy2mTNnMmDAAMaNG0fNmjXJzs4mISGB5cuXs2jRIq644gq++eYbmjRpYnao7N27l5o1a/LBBx9w3333mR1OqfL19eXOO+8s9at+69ato2XLlhXymFcErmYHIOKs7rnnnnyff/vtNxYtWnRe+3+dPn0ab2/vIm/Hzc2tWPEBuLq64uqq/81FRMqizp0706JFC/vn4cOHs3TpUm699VZuu+02tm3bhpeXl4kRikhJ07RDEQfq0KEDjRo1Yv369Vx77bV4e3szYsQIAL7++mtuueUWwsPD8fDwoHbt2rzwwgvk5ubmW8d/7/k6O/9+woQJvPPOO9SuXRsPDw9atmzJ77//nm/Zgu75slgsDB48mPnz59OoUSM8PDxo2LAhCxcuPC/+5cuX06JFCzw9PalduzZvv/12ke8j27lzJz169CAsLAxPT0+qV69O7969SUlJydfv448/pnnz5nh5eVG5cmV69+7NgQMH8h3D7777jn379tmn6BT3HjgRkbLuhhtuYNSoUezbt++8GRbbt2/nzjvvpHLlynh6etKiRQu++eYb+/fr1q3DYrEwa9as89b7448/YrFYWLBggb3t0KFD3H///YSGhtrHgvfff79IcS5dupT27dvj4+NDYGAgt99+O9u2bcvX5+x4sX37dnr27Im/vz9VqlThscceIyMjI1/fs2PT3LlzadCgAV5eXrRp04Y//vgDgLfffps6derg6elJhw4dCpwJsWbNGjp16kRAQADe3t5cd911rFy5ssCYdu3axX333UdgYCABAQEMGDCA06dP54snLS2NWbNm2ceei12Fmjp1Kg0bNsTb25tKlSrRokULZs+ena/PxY758uXLadmyJQADBgywb1v33JUf+klcxMGOHTtG586d6d27N/fcc499CuLMmTPx9fUlNjYWX19fli5dyujRo0lNTWX8+PEXXe/s2bM5efIkDz74IBaLhddee43u3buze/fui14tW7FiBfPmzeORRx7Bz8+PKVOm0KNHD/bv30+VKlUA2LhxI506daJq1ao8//zz5ObmMm7cOIKDgy8aW1ZWFjExMWRmZjJkyBDCwsI4dOgQCxYsIDk5mYCAAABeeuklRo0aRc+ePfm///s/jh49ytSpU7n22mvZuHEjgYGBPPfcc6SkpHDw4EFef/11IG8qiIhIeXXvvfcyYsQIfvrpJwYNGgTAX3/9Rbt27ahWrRrPPvssPj4+fP7553Tr1o0vv/ySO+64gxYtWlCrVi0+//xz+vfvn2+dc+bMoVKlSsTExACQmJhI69at7UlPcHAwP/zwAwMHDiQ1NZVhw4YVGt/ixYvp3LkztWrVYuzYsaSnpzN16lTatWvHhg0bzvuBrGfPnkRGRhIXF8dvv/3GlClTOHHiBB9++GG+fr/++ivffPMNjz76KABxcXHceuutPP3007z55ps88sgjnDhxgtdee43777+fpUuX2pddunQpnTt3pnnz5owZMwar1coHH3zADTfcwK+//kqrVq3Oi6lmzZrExcWxYcMG3nvvPUJCQnj11VcB+Oijj/i///s/WrVqxQMPPABA7dq1Cz0m7777LkOHDuXOO++0J5dbtmxhzZo19O3bt8jHvH79+owbN47Ro0fzwAMP0L59ewDatm1b6LbFyRgiUiIeffRR47//S1133XUGYMyYMeO8/qdPnz6v7cEHHzS8vb2NjIwMe1v//v2NGjVq2D/v2bPHAIwqVaoYx48ft7d//fXXBmB8++239rYxY8acFxNguLu7G7t27bK3bd682QCMqVOn2tu6du1qeHt7G4cOHbK37dy503B1dT1vnf+1ceNGAzDmzp1baJ+9e/caLi4uxksvvZSv/Y8//jBcXV3ztd9yyy35joGIiDP74IMPDMD4/fffC+0TEBBgNGvWzP75xhtvNBo3bpxvfLDZbEbbtm2NunXr2tuGDx9uuLm55RsfMjMzjcDAQOP++++3tw0cONCoWrWqkZSUlG+7vXv3NgICAuxj1Nkx54MPPrD3iYqKMkJCQoxjx47Z2zZv3mxYrVajX79+9razY9Btt92WbxuPPPKIARibN2+2twGGh4eHsWfPHnvb22+/bQBGWFiYkZqamm8fAXtfm81m1K1b14iJiTFsNpu93+nTp42aNWsaN91003kxnXssDMMw7rjjDqNKlSr52nx8fIz+/fsbRXH77bcbDRs2vGCfoh7z33///bxjLuWHph2KOJiHhwcDBgw4r/3cefwnT54kKSmJ9u3bc/r0abZv337R9fbq1YtKlSrZP5/9dWz37t0XXbZjx475fsFr0qQJ/v7+9mVzc3NZvHgx3bp1Izw83N6vTp06dO7c+aLrP3tl68cff8w3jeNc8+bNw2az0bNnT5KSkuyvsLAw6taty7Jlyy66HRGR8srX19de9fD48eMsXbqUnj172seLpKQkjh07RkxMDDt37uTQoUNA3tiQnZ3NvHnz7Ov66aefSE5OplevXgAYhsGXX35J165dMQwj3zk4JiaGlJQUNmzYUGBchw8fZtOmTdx3331UrlzZ3t6kSRNuuukmvv/++/OWOXsl66whQ4YAnNf3xhtvzHfVLDo6GoAePXrg5+d3XvvZMWvTpk3s3LmTvn37cuzYMfu+pKWlceONN/LLL79gs9nybeuhhx7K97l9+/YcO3aM1NTUAvf7YgIDAzl48OB50//PupxjLuWLph2KOFi1atVwd3c/r/2vv/5i5MiRLF269LyT/X/viyrIFVdcke/z2UTsxIkTl7zs2eXPLnvkyBHS09OpU6fOef0KavuvmjVrEhsby6RJk/jkk09o3749t912G/fcc489Mdu5cyeGYVC3bt0C13E5hUZERJzdqVOnCAkJAWDXrl0YhsGoUaMYNWpUgf2PHDlCtWrVaNq0KfXq1WPOnDkMHDgQyJtyGBQUxA033ADA0aNHSU5O5p133uGdd94pdH0F2bdvHwBXXXXVed/Vr1+fH3/8kbS0NHx8fOzt/z3P165dG6vVet59W/8dm86OFxEREQW2nx2zdu7cCXDeVMtzpaSk5PvB8kJjqL+/f6HrKcwzzzzD4sWLadWqFXXq1OHmm2+mb9++tGvXDri8Yy7li5IvEQcrqFJVcnIy1113Hf7+/owbN47atWvj6enJhg0beOaZZ877ha4gLi4uBbYbRXh6xOUsW1QTJ07kvvvu4+uvv+ann35i6NCh9vn+1atXx2azYbFY+OGHHwqMR/d1iUhFdfDgQVJSUuw/dp0dE5588kn7PVv/de4PY7169eKll14iKSkJPz8/vvnmG/r06WOvfnt2fffcc0+hCYsjy9wXVrSpsLHpYmPW2f0ZP348UVFRBfb975hS0uNg/fr12bFjBwsWLGDhwoV8+eWXvPnmm4wePZrnn3/e9GMuZYeSLxETLF++nGPHjjFv3jyuvfZae/uePXtMjOpfISEheHp6smvXrvO+K6itMI0bN6Zx48aMHDmSVatW0a5dO2bMmMGLL75I7dq1MQyDmjVrcuWVV15wPUWprigiUl6cfV7k2USrVq1aQN6MgI4dO150+V69evH888/z5ZdfEhoaSmpqKr1797Z/HxwcjJ+fH7m5uUVa37lq1KgBwI4dO877bvv27QQFBeW76gV5V6Zq1qxp/7xr1y5sNluJVa49O43e39//kvfnQi517PHx8aFXr1706tWLrKwsunfvzksvvcTw4cMv6ZhrzCvfdM+XiAnO/uJ27i9sWVlZvPnmm2aFlI+LiwsdO3Zk/vz5xMfH29t37drFDz/8cNHlU1NTycnJydfWuHFjrFYrmZmZAHTv3h0XFxeef/75835pNAyDY8eO2T/7+PgUaSqmiIizW7p0KS+88AI1a9bk7rvvBvJ+EOvQoQNvv/02hw8fPm+Zo0eP5vtcv359GjduzJw5c5gzZw5Vq1bN90Ofi4sLPXr04Msvv+TPP/+86PrOVbVqVaKiopg1axbJycn29j///JOffvqJLl26nLfM9OnT832eOnUqQJHuIS6K5s2bU7t2bSZMmMCpU6fO+/5C+3MhPj4++fbxQs4dswDc3d1p0KABhmGQnZ19Scf8bPJa1G2Lc9GVLxETtG3blkqVKtG/f3+GDh2KxWLho48+KtFpf5dr7Nix/PTTT7Rr146HH36Y3Nxcpk2bRqNGjdi0adMFl126dCmDBw/mrrvu4sorryQnJ4ePPvrIPvhA3i+VL774IsOHD2fv3r1069YNPz8/9uzZw1dffcUDDzzAk08+CeQNrHPmzCE2NpaWLVvi6+tL165dHX0IREQc6ocffmD79u3k5OSQmJjI0qVLWbRoETVq1OCbb77B09PT3nf69Olcc801NG7cmEGDBlGrVi0SExNZvXo1Bw8eZPPmzfnW3atXL0aPHo2npycDBw7Eas3/e/srr7zCsmXLiI6OZtCgQTRo0IDjx4+zYcMGFi9ezPHjxwuNe/z48XTu3Jk2bdowcOBAe6n5gIAAxo4de17/PXv2cNttt9GpUydWr17Nxx9/TN++fWnatOnlHcAzrFYr7733Hp07d6Zhw4YMGDCAatWqcejQIZYtW4a/vz/ffvvtJa+3efPmLF68mEmTJhEeHk7NmjXtxT7+6+abbyYsLIx27doRGhrKtm3bmDZtGrfccou9WEhRj3nt2rUJDAxkxowZ+Pn54ePjQ3R0dL6rh+LESr/Aokj5VFip+cJKz65cudJo3bq14eXlZYSHhxtPP/208eOPPxqAsWzZMnu/wkrNjx8//rx1AsaYMWPsnwsrNf/oo4+et2yNGjXOK6m7ZMkSo1mzZoa7u7tRu3Zt47333jOeeOIJw9PTs5CjkGf37t3G/fffb9SuXdvw9PQ0KleubFx//fXG4sWLz+v75ZdfGtdcc43h4+Nj+Pj4GPXq1TMeffRRY8eOHfY+p06dMvr27WsEBgYagMrOi4hTO1tq/uzL3d3dCAsLM2666SbjjTfeyFdW/Vz//POP0a9fPyMsLMxwc3MzqlWrZtx6663GF198cV7fnTt32te/YsWKAteXmJhoPProo0ZERITh5uZmhIWFGTfeeKPxzjvv2PsUVGreMAxj8eLFRrt27QwvLy/D39/f6Nq1q7F169Z8fc6OQVu3bjXuvPNOw8/Pz6hUqZIxePBgIz09PV/fgsamwsa7ZcuWFfg4k40bNxrdu3c3qlSpYnh4eBg1atQwevbsaSxZsuS8mI4ePZpv2bP/Tc4tdb99+3bj2muvNby8vAzggmXn3377bePaa6+1b7t27drGU089ZaSkpOTrV5Rjbhh5j49p0KCB/fEuKjtfflgMowz91C4iZV63bt3466+/7NWlRERECjJ27Fief/55jh49SlBQkNnhiJQJuudLRAqVnp6e7/POnTv5/vvv6dChgzkBiYiIiDgx3fMlIoWqVasW9913H7Vq1WLfvn289dZbuLu78/TTT5sdmoiIiIjTUfIlIoXq1KkTn376KQkJCXh4eNCmTRtefvnlQh+MLCIiIiKF0z1fIiIiIiIipUD3fImIiIiIiJQCJV8iIiIiIiKlQPd8FZPNZiM+Ph4/Pz8sFovZ4YiIVBiGYXDy5EnCw8PPe3BsRaexSUTEHEUdm5R8FVN8fDwRERFmhyEiUmEdOHCA6tWrmx1GmaKxSUTEXBcbm5R8FZOfnx+Qd4D9/f1NjkZEpOJITU0lIiLCfh6Wf2lsEhExR1HHJiVfxXR2Ooe/v78GOBERE2ha3fk0NomImOtiY5Mmy4uIiIiIiJQCJV8iIiIiIiKlQMmXiIiIiIhIKVDyJSIiIiIiUgqUfJnIMAyzQxARERERkVKi5MsEv+89Tt93f+OJzzebHYqIiIiIiJQSlZo3gQVY9c8xKnm7YbMZWK0qlywiIiIiUt7pypcJmkYE4u3uwonT2WxLSDU7HBERERERKQVKvkzg5mKlVc3KAKz+55jJ0YiIiIiISGkwPfmaPn06kZGReHp6Eh0dzdq1awvt+9dff9GjRw8iIyOxWCxMnjz5vD5jx47FYrHke9WrVy9fn4yMDB599FGqVKmCr68vPXr0IDExsaR37YLa1Q4CYOWupFLdroiIiIiImMPU5GvOnDnExsYyZswYNmzYQNOmTYmJieHIkSMF9j99+jS1atXilVdeISwsrND1NmzYkMOHD9tfK1asyPf9448/zrfffsvcuXP5+eefiY+Pp3v37iW6bxfTtk4VANbuOU52rq1Uty0iIiIiIqXP1ORr0qRJDBo0iAEDBtCgQQNmzJiBt7c377//foH9W7Zsyfjx4+nduzceHh6FrtfV1ZWwsDD7KygoyP5dSkoK//vf/5g0aRI33HADzZs354MPPmDVqlX89ttvJb6Phakf5k8lbzfSsnLZcjC51LYrIiIiIiLmMC35ysrKYv369XTs2PHfYKxWOnbsyOrVqy9r3Tt37iQ8PJxatWpx9913s3//fvt369evJzs7O99269WrxxVXXHHB7WZmZpKamprvdTmsVgttaudd/Vq5S/d9iYjIpSvpsUlERBzLtOQrKSmJ3NxcQkND87WHhoaSkJBQ7PVGR0czc+ZMFi5cyFtvvcWePXto3749J0+eBCAhIQF3d3cCAwMvabtxcXEEBATYXxEREcWO8ay2Z+77WvWP7vsSEZFL54ixSUREHMf0ghslrXPnztx11100adKEmJgYvv/+e5KTk/n8888va73Dhw8nJSXF/jpw4MBlx9r2zJWvDfuSSc/Kvez1iYhIxeKIsUlERBzHtIcsBwUF4eLicl6VwcTExAsW07hUgYGBXHnllezatQuAsLAwsrKySE5Oznf162Lb9fDwuOB9ZsVRM8iHqgGeHE7JYN2+47SvG1yi6xcRkfLNEWOTiIg4jmlXvtzd3WnevDlLliyxt9lsNpYsWUKbNm1KbDunTp3in3/+oWrVqgA0b94cNze3fNvdsWMH+/fvL9HtFoXF8u99X6v0vC8RERERkXLNtCtfALGxsfTv358WLVrQqlUrJk+eTFpaGgMGDACgX79+VKtWjbi4OCCvSMfWrVvt7w8dOsSmTZvw9fWlTp06ADz55JN07dqVGjVqEB8fz5gxY3BxcaFPnz4ABAQEMHDgQGJjY6lcuTL+/v4MGTKENm3a0Lp161I/Bu1qBzFvwyFW6XlfIiIiIiLlmqnJV69evTh69CijR48mISGBqKgoFi5caC/CsX//fqzWfy/OxcfH06xZM/vnCRMmMGHCBK677jqWL18OwMGDB+nTpw/Hjh0jODiYa665ht9++43g4H+n9L3++utYrVZ69OhBZmYmMTExvPnmm6Wz0/9x9nlffxxKISU9mwAvN1PiEBERERERx7IYhmGYHYQzSk1NJSAggJSUFPz9/S9rXTdMWM7upDTeubc5NzcsufvdRETKo5I8/5Y3OjYiIuYo6vm33FU7dEZnr37pvi8RERERkfJLyVcZ0E7P+xIRERERKfeUfJUBrWtVwWKBvxNPceRkhtnhiIiIiIiIAyj5KgMq+bjToGre3NDVmnooIiIiIlIuKfkqI9qefd7XLiVfIiIiIiLlkZKvMqJtnbz7vlbqvi8RERERkXJJyVcZ0SqyMq5WCwdPpLP/2GmzwxERERERkRKm5KuM8PFwJSoiEFDVQxERERGR8kjJVxny79RD3fclIiIiIlLeKPkqQ9qdKbqx+p8kDMMwORoRERERESlJSr7KkKgrAvF0s5J0Kou/E0+ZHY6IiIiIiJQgJV9liIerCy0jKwOwcpfu+xIRERERKU+UfJUxbWvn3fe1Svd9iYiIiIiUK0q+yph2dfLu+1qz+xg5uTaToxERERERkZKi5KuMaRgegL+nKyczc/jjUIrZ4YiIiIiISAlR8lXGuFgttK6Vd/VLUw9FRERERMoPJV9lULs6Z+/7UtENEREREZHyQslXGXT2vq91e0+QkZ1rcjQiIiIiIlISlHyVQbWDfQnx8yAzx8aG/SfMDkdEREREREqA6cnX9OnTiYyMxNPTk+joaNauXVto37/++osePXoQGRmJxWJh8uTJ5/WJi4ujZcuW+Pn5ERISQrdu3dixY0e+Ph06dMBiseR7PfTQQyW9a8VmsVhoW/vMfV+7dN+XiIiIiEh5YGryNWfOHGJjYxkzZgwbNmygadOmxMTEcOTIkQL7nz59mlq1avHKK68QFhZWYJ+ff/6ZRx99lN9++41FixaRnZ3NzTffTFpaWr5+gwYN4vDhw/bXa6+9VuL7dzn+fd6X7vsSERERESkPXM3c+KRJkxg0aBADBgwAYMaMGXz33Xe8//77PPvss+f1b9myJS1btgQo8HuAhQsX5vs8c+ZMQkJCWL9+Pddee6293dvbu9AErixoe+a+r80HUziZkY2fp5vJEYmIiIiIyOUw7cpXVlYW69evp2PHjv8GY7XSsWNHVq9eXWLbSUnJe1ZW5cqV87V/8sknBAUF0ahRI4YPH87p06cvuJ7MzExSU1PzvRypeiVvalTxJtdmsHbPcYduS0REnFNpj00iInJ5TEu+kpKSyM3NJTQ0NF97aGgoCQkJJbINm83GsGHDaNeuHY0aNbK39+3bl48//phly5YxfPhwPvroI+65554LrisuLo6AgAD7KyIiokRivBD7fV963peIiBTAjLFJRESKz9Rph4726KOP8ueff7JixYp87Q888ID9fePGjalatSo33ngj//zzD7Vr1y5wXcOHDyc2Ntb+OTU11eGDXNvaQXy69gArd+m+LxEROZ8ZY5OIiBSfaclXUFAQLi4uJCYm5mtPTEwskXuxBg8ezIIFC/jll1+oXr36BftGR0cDsGvXrkKTLw8PDzw8PC47rkvR5syVr+0JJzl2KpMqvqW7fRERKdvMGJtERKT4TJt26O7uTvPmzVmyZIm9zWazsWTJEtq0aVPs9RqGweDBg/nqq69YunQpNWvWvOgymzZtAqBq1arF3q4jBPl6UC/MD4DVuzX1UERERETEmZk67TA2Npb+/fvTokULWrVqxeTJk0lLS7NXP+zXrx/VqlUjLi4OyCvSsXXrVvv7Q4cOsWnTJnx9falTpw6QN9Vw9uzZfP311/j5+dnvHwsICMDLy4t//vmH2bNn06VLF6pUqcKWLVt4/PHHufbaa2nSpIkJR+HC2tYOYnvCSVbuOsatTcLNDkdERERERIrJ1OSrV69eHD16lNGjR5OQkEBUVBQLFy60F+HYv38/Vuu/F+fi4+Np1qyZ/fOECROYMGEC1113HcuXLwfgrbfeAvIepHyuDz74gPvuuw93d3cWL15sT/QiIiLo0aMHI0eOdOzOFlPb2lV4f+UeVut5XyIiIiIiTs1iGIZhdhDOKDU1lYCAAFJSUvD393fYdk5mZBM1bhG5NoOVz95AtUAvh21LRMQZlNb51xnp2IiImKOo51/T7vmSovHzdKNJ9QAAVT0UEREREXFiSr6cwNnnfa3W875ERERERJyWki8n0K52EJB35UuzREVEREREnJOSLydwdY1KuLtaOXIyk3+OppkdjoiIiIiIFIOSLyfg6eZCixqVAFilqociIiIiIk5JyZeTaFfn36mHIiIiIiLifJR8OYk2Z4pu/Lb7OLk23fclIiIiIuJslHw5iSbVAvDzcCUlPZut8almhyMiIiIiIpdIyZeTcHWxEl2rMgArdd+XiIiIiIjTUfLlRNqcKTm/Ss/7EhERERFxOkq+nEi7Onn3ff2+5zhZOTaToxERERERkUuh5MuJXBXqRxUfd9Kzc9m4/4TZ4YiIiIiIyCVQ8uVELBaLveqhph6KiIiIiDgXJV9O5uzzvvSwZRERERER56Lky8m0PXPla+P+ZE5n5ZgcjYiIiIiIFJWSLydzRWVvqgV6kWMzWLvnuNnhiIiIiIhIESn5cjIWi8Ve9VD3fYmIiIiIOA8lX06obW3d9yUiIiIi4myUfDmhs/d9/RWfSvLpLJOjERERubDs3GwOnjxodhgiIqZT8uWEQvw9qRvii2HAak09FBGRMuz3hN/p/k13Os/rzILdC8wOR0TEVKYnX9OnTycyMhJPT0+io6NZu3ZtoX3/+usvevToQWRkJBaLhcmTJxdrnRkZGTz66KNUqVIFX19fevToQWJiYknulsO11fO+RESkDDuRcYKRK0Zy/4/3szd1LwBvb34bm2EzNzAREROZmnzNmTOH2NhYxowZw4YNG2jatCkxMTEcOXKkwP6nT5+mVq1avPLKK4SFhRV7nY8//jjffvstc+fO5eeffyY+Pp7u3bs7ZB8dpe2Z532t1H1fIiJShhiGwde7vua2+bfx9T9fY8HCnVfeia+bL3tT9/LrwV/NDlFExDSmJl+TJk1i0KBBDBgwgAYNGjBjxgy8vb15//33C+zfsmVLxo8fT+/evfHw8CjWOlNSUvjf//7HpEmTuOGGG2jevDkffPABq1at4rfffnPYvpa01jWrYLXA7qNpJKRkmB2OiIgIe1P28n8//R8jV44kOTOZOoF1+LDzh4xpM4YedXsA8OHWD02OUkTEPKYlX1lZWaxfv56OHTv+G4zVSseOHVm9erXD1rl+/Xqys7Pz9alXrx5XXHHFBbebmZlJampqvpeZArzdaFQtAFDVQxGRiqqsjE1ZuVm8tfktun/TnbUJa/F08WTY1cP4vOvnRIVEAXB3/btxsbiwNmEt245tMyVOERGzmZZ8JSUlkZubS2hoaL720NBQEhISHLbOhIQE3N3dCQwMvKTtxsXFERAQYH9FREQUK8aSdLbk/Mpduu9LRKQiKgtj07qEddz57Z28uelNsm3ZtAtvx7zb5zGw8UDcrG72flV9q3Jz5M0AzNo6q9TjFBEpC0wvuOEshg8fTkpKiv114MABs0OyF91Y/U8ShmGYHI2IiJQ2M8emlMwUxqwaw4AfB7AnZQ9VPKvw2rWv8VbHt4jwKzgJ7N+gPwA/7vmRhLTi/dAqIuLMXM3acFBQEC4uLudVGUxMTCy0mEZJrDMsLIysrCySk5PzXf262HY9PDwKvc/MLC0jK+PuYiU+JYO9x05TM8jH7JBERKQUmTE2GYbBgt0LmLBuAsczjgNw15V38djVjxHgEXDBZRsGNaR5aHPWJ65n9vbZxDaPLY2QRUTKDNOufLm7u9O8eXOWLFlib7PZbCxZsoQ2bdo4bJ3NmzfHzc0tX58dO3awf//+Ym/XLF7uLjS7IhCAlbt035eIiDjWvtR9DFo0iBErRnA847i9oMboNqMvmniddfbq1xc7viAtO82R4YqIlDmmXfkCiI2NpX///rRo0YJWrVoxefJk0tLSGDBgAAD9+vWjWrVqxMXFAXkFNbZu3Wp/f+jQITZt2oSvry916tQp0joDAgIYOHAgsbGxVK5cGX9/f4YMGUKbNm1o3bq1CUfh8rStHcSaPcdZ/c8x7mldw+xwRESkHMrOzeaDvz7g7c1vk2XLwsPFg4eaPkT/Bv1xc3G7+ArOcV3EdUT6R7I3dS9f7fyKexrc46CoRUTKHlOTr169enH06FFGjx5NQkICUVFRLFy40F4wY//+/Vit/16ci4+Pp1mzZvbPEyZMYMKECVx33XUsX768SOsEeP3117FarfTo0YPMzExiYmJ48803S2enS1i7OlV4fXFexUObzcBqtZgdkoiIlCMbEjfw/Orn2Z2yG4A2VdswqvUoIvyLV9zDarFyb4N7eeG3F/h428f0qdcHF6tLSYYsIlJmWQxVaiiW1NRUAgICSElJwd/f37Q4snJsRI37idNZuXw/tD0Nws2LRUSkNJSV829ZVJLHJiUzhdfXv86XO78EoLJnZZ5u+TRdanbBYrm8H/rSc9K5+YubSc5MZuJ1E+1VEEVEnFVRz7+qdujk3F2ttKpZGdDzvkRE5PIZhsF3u7/jtvm32ROvHnV78E23b7il1i2XnXgBeLl60fOqnoDKzotIxaLkqxxoZ3/el5IvEREpvgOpB3ho8UM8++uzHM84Tq2AWszsNJOxbccWuaBGUfWp1wc3qxtbjm5h05FNJbpuEZGySslXOdDmzPO+1u45TnauzeRoRETEGc36axZ3fHMHq+JX4W51Z3DUYL7o+gXNQ5s7ZHtBXkHcWutWAD7c+qFDtiEiUtYo+SoHGlT1p5K3G2lZuWw5mGx2OCIi4oSycrPIzM0kumo0826fx4NNH7zkSoaX6t4G9wKwZP8SDpwsvQdEi4iYRclXOWC1WuxXv1buOmZyNCIi4ozua3gfkzpM4t2b3qWGf+k8uqRupbq0C2+HzbDx8daPS2WbIiJmUvJVTrQ5c9+Xim6IiEhxuLm4cVONm0qkoMal6NewHwBf7fqKlMyUUt22iEhpU/JVTrQ7c+Vrw75k0rNyTY5GRESkaNpUbUPdSnVJz0nni7+/MDscERGHUvJVTtQM8iHM35OsXBvr950wOxwREZEisVgs9GuQd/Vr9rbZZOdmmxyRiIjjKPkqJywWC23rnLnvS1MPRUTEiXSp2YUgryCOpB9h4d6FZocjIuIwxUq+Dhw4wMGDB+2f165dy7Bhw3jnnXdKLDC5dGef97VKz/sSEREn4u7iTt96fYG8svOGYZgckYiIYxQr+erbty/Lli0DICEhgZtuuom1a9fy3HPPMW7cuBINUIru7JWvPw6lkJKuaRsiIuI87rryLjxdPNl+fDtrE9aaHY6IiEMUK/n6888/adWqFQCff/45jRo1YtWqVXzyySfMnDmzJOOTS1A1wItaQT7YDFizWyXnRUTEeQR6BnJ7nduBvAc+i4iUR8VKvrKzs/Hw8ABg8eLF3HbbbQDUq1ePw4cPl1x0csnOXv1atuOoyZGIiIhcmn4N+mHBwq+HfmV38m6zwxERKXHFSr4aNmzIjBkz+PXXX1m0aBGdOnUCID4+nipVqpRogHJpOtYPBeCz3/ezbPsRk6MREREpuiv8r+D6iOuBvHu/RETKm2IlX6+++ipvv/02HTp0oE+fPjRt2hSAb775xj4dUczR4aoQ+kZfgWHA0M828s/RU2aHJCIiUmRnH7r87T/fcixdU+hFpHwpVvLVoUMHkpKSSEpK4v3337e3P/DAA8yYMaPEgpPiGdu1IS0jK3EyI4dBH64jNUPFN0RExDlcHXI1jao0IsuWxec7Pjc7HBGRElWs5Cs9PZ3MzEwqVaoEwL59+5g8eTI7duwgJCSkRAOUS+fuauXNu5tTNcCT3UfTePyzTdhsKtsrIiJln8VioX/D/gB8tuMzMnIyTI5IRKTkFCv5uv322/nww7y52MnJyURHRzNx4kS6devGW2+9VaIBSvEE+3nw9r3N8XC1smT7ESYt+tvskERERIqkY42OVPWpyvGM4yzYvcDscERESkyxkq8NGzbQvn17AL744gtCQ0PZt28fH374IVOmTCnRAKX4mlQP5NUeTQCYtmwX321RJUoRESn7XK2u3F3/biCv8IbNsJkckYhIyShW8nX69Gn8/PwA+Omnn+jevTtWq5XWrVuzb9++Eg1QLk+3ZtUY1L4mAE/O3czW+FSTIxIREbm4HnV74Ovmy56UPaw4tMLscERESkSxkq86deowf/58Dhw4wI8//sjNN98MwJEjR/D397/k9U2fPp3IyEg8PT2Jjo5m7doLP9l+7ty51KtXD09PTxo3bsz333+f73uLxVLga/z48fY+kZGR533/yiuvXHLszuCZTvVoXzeI9OxcHvhoHcfTsswOSURE5IJ83X3pUbcHAB/+pbLzIlI+FCv5Gj16NE8++SSRkZG0atWKNm3aAHlXwZo1a3ZJ65ozZw6xsbGMGTOGDRs20LRpU2JiYjhypOBnVK1atYo+ffowcOBANm7cSLdu3ejWrRt//vmnvc/hw4fzvd5//30sFgs9evTIt65x48bl6zdkyJBLPBLOwdXFyrQ+V1OjijcHT6Tz6CcbyM7VFA4RESnb7q5/Ny4WF9YkrGH78e1mhyMictkshmEUqwxeQkIChw8fpmnTpliteTnc2rVr8ff3p169ekVeT3R0NC1btmTatGkA2Gw2IiIiGDJkCM8+++x5/Xv16kVaWhoLFvx7A27r1q2JiooqtMx9t27dOHnyJEuWLLG3RUZGMmzYMIYNG1bkWM+VmppKQEAAKSkpxbraZ4a/E09yx/SVpGXlcl/bSMbe1tDskERELpkznn9LS3k8Nk///DQ/7P2BrrW68nL7l80OR0SkQEU9/xbryhdAWFgYzZo1Iz4+noMHDwLQqlWrS0q8srKyWL9+PR07dvw3IKuVjh07snr16gKXWb16db7+ADExMYX2T0xM5LvvvmPgwIHnfffKK69QpUoVmjVrxvjx48nJySk01szMTFJTU/O9nM2VoX5M6hUFwMxVe/l83QFzAxIRkctSHsamizlbdv6HPT+QmJZocjQiIpenWMmXzWZj3LhxBAQEUKNGDWrUqEFgYCAvvPACNlvRp7MlJSWRm5tLaGhovvbQ0FASEhIKXCYhIeGS+s+aNQs/Pz+6d++er33o0KF89tlnLFu2jAcffJCXX36Zp59+utBY4+LiCAgIsL8iIiKKsotlTkzDMIZ1rAvAyK/+ZMP+EyZHJCIixVVexqYLaRjUkOahzckxcpi9fbbZ4YiIXJZiJV/PPfcc06ZN45VXXmHjxo1s3LiRl19+malTpzJq1KiSjvGyvP/++9x99914enrma4+NjaVDhw40adKEhx56iIkTJzJ16lQyMzMLXM/w4cNJSUmxvw4ccN6rRkNvqMvNDULJyrXx0EfrSUzVAyxFRJxReRqbLqR/g7yrX3P/nsvp7NMmRyMiUnzFSr5mzZrFe++9x8MPP0yTJk1o0qQJjzzyCO+++y4zZ84s8nqCgoJwcXEhMTH/NILExETCwsIKXCYsLKzI/X/99Vd27NjB//3f/100lujoaHJycti7d2+B33t4eODv75/v5aysVguTekVxZagvR05m8uBH68nIzjU7LBERuUTlaWy6kOsirqOGfw1OZp3kq11fmR2OiEixFSv5On78eIH3dtWrV4/jx48XeT3u7u40b948XyEMm83GkiVL7BUU/6tNmzb5+gMsWrSowP7/+9//aN68OU2bNr1oLJs2bcJqtRISElLk+J2Zr4cr7/ZrQYCXG5sOJDNq/p8Us/aKiIiIQ1ktVu6tfy8AH2/9mFybfjAUEedUrOSradOm9uqE55o2bRpNmjS5pHXFxsby7rvvMmvWLLZt28bDDz9MWloaAwYMAKBfv34MHz7c3v+xxx5j4cKFTJw4ke3btzN27FjWrVvH4MGD8603NTWVuXPnFnjVa/Xq1UyePJnNmzeze/duPvnkEx5//HHuueceKlWqdEnxO7MaVXyY1rcZVgvMXX+QWav2mh2SiIhIgW6rcxsBHgEcPHWQpQeWmh2OiEixuBZnoddee41bbrmFxYsX2684rV69mgMHDpz3wOOL6dWrF0ePHmX06NEkJCQQFRXFwoUL7UU19u/fby9lD9C2bVtmz57NyJEjGTFiBHXr1mX+/Pk0atQo33o/++wzDMOgT58+523Tw8ODzz77jLFjx5KZmUnNmjV5/PHHiY2NvdRD4fTa1w1meOf6vPT9Nl74bhtXhvnRtnaQ2WGJiIjk4+XqRc8re/LuH+/y4V8fclONm8wOSUTkkhX7OV/x8fFMnz6d7dvzHnpYv359HnjgAV588UXeeeedEg2yLCpPz1IxDIPYzzfz1cZDVPJ245vB1xBR2dvssEREClSezr8lrbwfm6T0JG7+4maybdl83OVjmgZf/LYCEZHSUNTzb7GTr4Js3ryZq6++mtzc8j8Xu7wNcBnZufR8ezVbDqZQL8yPeY+0xdu9WBdGRUQcqrydf0tSRTg2o1aOYv6u+dxU4yYmdZhkdjgiIkApPGRZyhdPNxdm3NOcIF93tiec5Km5W1SAQ0REypx+DfoBsGT/Eg6ePGhyNCIil0bJl9iFB3rx1j3NcXOx8N0fh3lz+T9mhyQiIpJP3Up1aRveFpth45Ntn5gdjojIJVHyJfm0jKzM87flFS+Z8NMOlm5PvMgSIiIipevsQ5fn7ZxHalaqydGIiBTdJd3U07179wt+n5ycfDmxSBnRN/oK/opP4ZM1+3ns00189Wg76oT4mh2WiIgIAG3C21C3Ul12ntjJF39/wf2N7jc7JBGRIrmkK18BAQEXfNWoUYN+/fo5KlYpRWO6NqRVZGVOZubwwIfrSEnPNjskERERACwWi/3er0+2fUK2TWOUiDiHEq12WJFUhIpSSacyuW3qCuJTMrj+qmDe698SF6vF7LBEpIKrCOff4qpIxyYrN4uYL2NISk8irn0ct9a61eyQRKQCU7VDuWxBvh68fW8LPFytLNtxlIk/7TA7JBEREQDcXdzpU68PAB/+9aEq9IqIU1DyJRfUuHoAr93ZBIA3l//Dt5vjTY5IREQkT88re+Lp4sm249tYuHeh2eGIiFyUki+5qNujqvHgtbUAeOqLzWyNV2UpERExX6BnIPc2uBeAMavGsOO4ZmiISNmm5EuK5OlO9bj2ymAysm0M/WwjGdm5ZockIiLCI1GP0Da8Lek56QxdOpQTGSfMDklEpFBKvqRIXKwW3ugVRZCvB7uOnGLCj/p1UUREzOdqdeW1a1/jCr8riE+L54mfn1D1QxEps5R8SZFV8nHn1R6NAfjfyj38tvuYyRGJiIhAgEcAU26YgrerN78n/M7438ebHZKISIGUfMklubF+KL1aRGAY8OTczZzKzDE7JBEREWoH1iaufRwAn27/lHk755kckYjI+ZR8ySUbeWt9qlfy4uCJdF5csNXscERERAC44YobeDTqUQBe+O0FNh3ZZG5AIiL/oeRLLpmfpxsT7mqKxQKf/X6ApdsTzQ5JREQEgAeaPMBNNW4ix5bDsGXDSEhLMDskERE7JV9SLK1rVWFgu5oAPPPlH5xIyzI5IhEREbBarLzY7kXqVqrLsYxjDFs2jIycDLPDEhEBlHzJZXgy5irqhPhy9GQmI+f/iWEYZockIiKCt5s3U66fQoBHAH8d+4vnVz+vMUpEygQlX1Jsnm4uvN4zClerhe/+OMw3m+PNDklERASA6n7VmXjdRFwsLizYvYAPt35odkgiIkq+5PI0rh7A4BvqADD6679ISNHUDhERKRuiq0bzVMunAJi0fhKr4leZHJGIVHRlIvmaPn06kZGReHp6Eh0dzdq1ay/Yf+7cudSrVw9PT08aN27M999/n+/7++67D4vFku/VqVOnfH2OHz/O3Xffjb+/P4GBgQwcOJBTp06V+L5VBI9eX4fG1QJISc/mmS+3aGqHiIiUGX3r9aVbnW7YDBtP/fwU+1P3mx2SiFRgpidfc+bMITY2ljFjxrBhwwaaNm1KTEwMR44cKbD/qlWr6NOnDwMHDmTjxo1069aNbt268eeff+br16lTJw4fPmx/ffrpp/m+v/vuu/nrr79YtGgRCxYs4JdffuGBBx5w2H6WZ24uVl7v1RR3Vys//32U2Ws1sImISNlgsVgY1XoUTYKbkJqVytClQ0nLTjM7LBGpoCyGyZcpoqOjadmyJdOmTQPAZrMRERHBkCFDePbZZ8/r36tXL9LS0liwYIG9rXXr1kRFRTFjxgwg78pXcnIy8+fPL3Cb27Zto0GDBvz++++0aNECgIULF9KlSxcOHjxIeHj4ReNOTU0lICCAlJQU/P39L3W3y6X3ft3Ni99tw9vdhR8ea0+NKj5mhyQi5ZDOv4XTsSnckdNH6L2gN0fTj3J9xPVMvn4yVovpv0GLSDlR1POvqWedrKws1q9fT8eOHe1tVquVjh07snr16gKXWb16db7+ADExMef1X758OSEhIVx11VU8/PDDHDt2LN86AgMD7YkXQMeOHbFaraxZs6bA7WZmZpKamprvJfnd364m0TUrczorlyc+30yuTdMPRUQcSWNT0YV4hzD5+sm4Wd1YdmAZb21+y+yQRKQCMjX5SkpKIjc3l9DQ0HztoaGhJCQU/FDEhISEi/bv1KkTH374IUuWLOHVV1/l559/pnPnzuTm5trXERISkm8drq6uVK5cudDtxsXFERAQYH9FRERc8v6Wd1arhQl3NcXXw5V1+07w3q+7zQ5JRKRc09h0aZoEN2FMmzEAzNg8g8X7FpsckYhUNOXyenvv3r257bbbaNy4Md26dWPBggX8/vvvLF++vNjrHD58OCkpKfbXgQMHSi7gciSisjejb20AwMSf/mZ7gn6FFRFxFI1Nl+72OrdzT/17ABixYgR/n/jb5IhEpCIxNfkKCgrCxcWFxMTEfO2JiYmEhYUVuExYWNgl9QeoVasWQUFB7Nq1y76O/xb0yMnJ4fjx44Wux8PDA39//3wvKdhdLapzY70QsnJtxM7ZTFaOzeyQRETKJY1NxfNEiyeIrhpNek46Q5cOJTkj2eyQRKSCMDX5cnd3p3nz5ixZssTeZrPZWLJkCW3atClwmTZt2uTrD7Bo0aJC+wMcPHiQY8eOUbVqVfs6kpOTWb9+vb3P0qVLsdlsREdHX84uCXmVpeJ6NKaStxtbD6cyZclOs0MSERGxc7W6MuHaCVTzrcahU4d48ucnybHlmB2WiFQApk87jI2N5d1332XWrFls27aNhx9+mLS0NAYMGABAv379GD58uL3/Y489xsKFC5k4cSLbt29n7NixrFu3jsGDBwNw6tQpnnrqKX777Tf27t3LkiVLuP3226lTpw4xMTEA1K9fn06dOjFo0CDWrl3LypUrGTx4ML179y5SpUO5uBA/T166ozEAby7fxcb9J0yOSERE5F+BnoFMuWEKXq5erElYw8R1E80OSUQqANOTr169ejFhwgRGjx5NVFQUmzZtYuHChfaiGvv37+fw4cP2/m3btmX27Nm88847NG3alC+++IL58+fTqFEjAFxcXNiyZQu33XYbV155JQMHDqR58+b8+uuveHh42NfzySefUK9ePW688Ua6dOnCNddcwzvvvFO6O1/OdWlcldujwrEZ8MTnm0nPyjU7JBEREbsrK13Jy9e8DMDH2z5m/q755gYkIuWe6c/5clZ6lkrRpJzO5ubJP5OYmsl9bSMZe1tDs0MSESen82/hdGyK581Nb/LW5rdws7oxs9NMmgQ3MTskEXEyTvGcLyn/ArzdeO3OpgDMXLWXlbuSTI5IREQkv4eaPsQNETeQbctm2LJhHDl95OILiYgUg5IvcbjrrgzmntZXAPDU3M2kZmSbHJGIiMi/rBYrL7d/mTqBdTiafpTHlz1OZm6m2WGJSDmk5EtKxYgu9alRxZv4lAye/2ar2eGIiIjk4+Pmw5Trp+Dv7s+WpC28sPoFdGeGiJQ0JV9SKrzdXZl4V1OsFvhyw0F+/CvB7JBERETyifCPYPx147FarHz9z9fM3j7b7JBEpJxR8iWlpkVkZR64tjYAI+b9QdIpTekQEZGypW14W2KbxwIw/vfx/Hb4N5MjEpHyRMmXlKrHb6pLvTA/jqVlMWLeH5rSISIiZU6/Bv3oWqsruUYuT/78JAlpmq0hIiVDyZeUKg9XFyb1jMLNxcJPWxOZt+GQ2SGJiIjkY7FYGN1mNA2qNCAlM4WRK0diM2xmhyUi5YCSLyl1DcL9GdbxSgDGfvMX8cnpJkckIiKSn6erJ6+2fxVPF0/WHF7DJ9s+MTskESkHlHyJKR68thbNrgjkZGYOT32xGZtN0w9FRKRsiQyI5MkWTwIwef1kdp3YZXJEIuLslHyJKVxdrEzqGYWnm5WVu47x0W/7zA5JRETkPD2v6sk11a4hy5bF8BXDyc7VsypFpPiUfIlpagb5MKJLfQDiftjG7qOnTI5IREQkP4vFwri24wj0CGT78e1M3zTd7JBExIkp+RJT3RNdg2vqBJGRbSP2883k5OqGZhERKVuCvYMZ02YMAO//+T4bEjeYHJGIOCslX2Iqq9XCa3c2wc/TlU0Hkhn04TqO6flfIiJSxnSs0ZHbat+GgcGIFSM4laXZGiJy6ZR8ienCA72YcFdT3F2tLNtxlM5v/MqqXUlmhyUiIpLP8FbDCfcJ59CpQ7z2+2tmhyMiTshi6Cm3xZKamkpAQAApKSn4+/ubHU65sO1wKkM+3ciuI6ewWOCRDrV5vOOVuLroNwIR+ZfOv4XTsXG8dQnruP/H+zEwmHz9ZG684kazQxJxKJthIzM3k6zcLLJys+zvM3Mz870/tw3Aw8UDD1cPPF08cXdxx9PFEw9Xj7x2F498n60W5/+3XlHPv0q+ikkDnGOczsph3Ldb+ez3AwA0r1GJN3pHUb2St8mRiUhZofNv4XRsSsek9ZP44M8PqORRiXm3zyPIK8jskEQuKNuWzcGTB9mXuo99qfvYm7qXpNNJ+RMoW8GJVI4tx+HxuVnd/k3SXD3tCVpBCVygZyDhPuFU9alKVd+qVPWpSqBHIBaLxeFxXoiSLwfTAOdY326OZ8S8PziZmYO/pyuv9mhC58ZVzQ5LRMoAnX8Lp2NTOrJys+j7XV92nNjBtdWvZdoN00z/h5+IYRgknk7Ml2CdfX/w5EFyjdzL3obVYsXDxQN3F3c8rGf+dPn3z7PvAXvylpmbSUZOBlm5WWTkZuS15WSSY5RcUufl6kWYT1heQnb25fvv+1CfUNysbiW2vYIo+XIwDXCOd+D4aYZ8upFNB5IB6Bt9BaNvbYCnm4u5gYmIqXT+LZyOTenZeWInvRb0ItuWzajWo+h5VU+zQ5IKIiUz5bwEa2/KXvaf3E96Tnqhy3m5elHDv4b9FeYTljf175yk6b/v/9vmanUtsf3IseXYE7Ks3CwycjIunLCdaTuWfozDaYdJSEsgPi2epPSL1wmwYCHYO5iqPlUJ9wknzDfMfvUszCeMcN9w/Nz9Lmt/lHw5mAa40pGda2PiT38z4+d/ALgq1I+pfZtxZejl/Q8iIs5L59/C6diUrll/zWLCugl4uXoxt+tcavjXMDskKUdOZJxgfeJ69qbuZW/Kv1exTmSeKHQZF4sL1f2qE+kfaU+yzr4P8Q4pl1dos3KzSEhL4HDa4bzXqcP/vj/zOcuWddH1+Lr50jioMe/c/E6x4ijq+bfk0lcRB3BzsfJs53q0q1OFx+dsZkfiSW6btoIxXRvSu2VEuTyJiIiIc7i3wb38cvAX1iasZcSvI5jVeVaJXhmQiscwDDYd3cScHXP4ae9PZNuyC+wX4h1SYIJVza+aw6fXlTXuLu5c4X8FV/hfUeD3hmFwLOOYPUGLPxWfd9XsVLz9CtqJzBOcyj51wSuHJaVMXPmaPn0648ePJyEhgaZNmzJ16lRatWpVaP+5c+cyatQo9u7dS926dXn11Vfp0qULANnZ2YwcOZLvv/+e3bt3ExAQQMeOHXnllVcIDw+3ryMyMpJ9+/blW29cXBzPPvtskWLWr4ul7+jJTGI/38SvO/MuL9/SuCovd29MgFfFOsmIVHQ6/xZOx6b0HT51mB7f9OBk9kkeiXqEh5s+bHZI4oROZZ1iwe4FfP735+w8sdPeXiewDvUq18uXYNXwr4G3mwqRlaTT2adJOJ1Adm42V1W+qljrcJpph3PmzKFfv37MmDGD6OhoJk+ezNy5c9mxYwchISHn9V+1ahXXXnstcXFx3HrrrcyePZtXX32VDRs20KhRI1JSUrjzzjsZNGgQTZs25cSJEzz22GPk5uaybt06+3oiIyMZOHAggwYNsrf5+fnh4+NTpLg1wJnDZjN499fdjP9xBzk2g+qVvJjSpxlXX1HJ7NBEpJTo/Fs4HRtzLNi9gOG/DsfF4sLHXT6mUVAjs0MSJ7Ht2DY+//tzvtv9nf2qi6eLJ51rdqbXVb1oGNTQ5AilqJwm+YqOjqZly5ZMmzYNAJvNRkREBEOGDCnwKlSvXr1IS0tjwYIF9rbWrVsTFRXFjBkzCtzG77//TqtWrdi3bx9XXJF3STIyMpJhw4YxbNiwYsWtAc5cmw4kM/TTjew/fhoXq4Unbr6Sh66tjdWqaYgi5Z3Ov4XTsTGHYRg89ctT/Lj3RyL9I/m86+d4uXqZHZaUURk5GSzcu5C5O+ayJWmLvb1WQC16XtWTrrW74u+u/3+dTVHPv6Y+0SwrK4v169fTsWNHe5vVaqVjx46sXr26wGVWr16drz9ATExMof0BUlJSsFgsBAYG5mt/5ZVXqFKlCs2aNWP8+PHk5BRe8jIzM5PU1NR8LzFPVEQgC4ZeQ9em4eTaDF5buIN+76/lyMkMs0MTESk1GpvKBovFwqjWowjxCmFv6l4mrZtkdkhSBu1J2cOra1/lhrk3MGrlKLYkbcHV6krnyM68H/M+82+fz93171biVc6ZeldoUlISubm5hIaG5msPDQ1l+/btBS6TkJBQYP+EhIQC+2dkZPDMM8/Qp0+ffFno0KFDufrqq6lcuTKrVq1i+PDhHD58mEmTCj5hxsXF8fzzz1/K7omD+Xu6MaV3FO3rBDHmm79YsSuJzpN/ZWLPpnS46vwpqyIi5Y3GprIjwCOAF655gQcXPchnOz7juojruKbaNWaHJSbLzs1m6YGlfL7jc9YmrLW3V/Otxp1X3km3Ot30kO4KplyX5MnOzqZnz54YhsFbb72V77vY2Fj7+yZNmuDu7s6DDz5IXFwcHh4e561r+PDh+ZZJTU0lIiLCccFLkVgsFnq2jODqGoEMnr2R7Qknue+D33ng2lo8efNVuLuaenFXRMShNDaVLW3D29K3Xl9mb5/NqJWj+Oq2rwj0DDQ7LDFB/Kl4vvj7C+btnMexjGNA3gOKr61+LT2v7Em7au2wWvRvlIrI1OQrKCgIFxcXEhMT87UnJiYSFhZW4DJhYWFF6n828dq3bx9Lly696Nz36OhocnJy2Lt3L1dddX6VEw8PjwKTMikb6oT4Mf/Rdrz8/TY+XL2Pd37ZzZrdx5jSpxk1qhStiIqIiLPR2FT2PN78cVYfXs2elD2M+20cE6+bqMeiVBC5tlxWxq9kzo45/HrwVwzyyioEewXTvW53etTtQVXfqiZHKWYzNeV2d3enefPmLFmyxN5ms9lYsmQJbdq0KXCZNm3a5OsPsGjRonz9zyZeO3fuZPHixVSpUuWisWzatAmr1VpghUVxDp5uLoy7vRFv39ucAC83Nh9M4ZYpK/h60yGzQxMRkQrC09WTuPZxuFpcWbRvEd/u/tbskMTBktKTeHfLu3SZ14VHlzzKLwd/wcAgumo0kzpM4sc7f2Rws8FKvAQoA9MOY2Nj6d+/Py1atKBVq1ZMnjyZtLQ0BgwYAEC/fv2oVq0acXFxADz22GNcd911TJw4kVtuuYXPPvuMdevW8c47eU+jzs7O5s4772TDhg0sWLCA3Nxc+/1glStXxt3dndWrV7NmzRquv/56/Pz8WL16NY8//jj33HMPlSqpZLmzi2kYRuNqAQz7bBNr9x7nsc82sWJnEs/f3hBvd9P/youISDnXsEpDHo56mKkbp/LympdpHtqcar7VzA5LStjhU4eZsWUG3+z6hhwjr2hbgEcA3Wp3484r7yQyINLcAKVMMr3UPMC0adPsD1mOiopiypQpREdHA9ChQwciIyOZOXOmvf/cuXMZOXKk/SHLr732mv0hy3v37qVmzZoFbmfZsmV06NCBDRs28Mgjj7B9+3YyMzOpWbMm9957L7GxsUWevqFyvmVfTq6NKUt3MW3pTmwG1Anx5c27r+bKUD+zQxORy6Dzb+F0bMqOHFsOAxYOYNPRTTQPbc7/bv4fLlYXs8OSEnAs/Rjv/fEec3bMIduWDUDT4Kb0uqoXN9W4CU9XT5MjFDM4zXO+nJUGOOfx2+5jDP10I0dOZuLpZmXc7Y24q3l1zcEXcVI6/xZOx6ZsOZB6gB7f9iA9J53Y5rEMaDTA7JDkMpzMOsnMv2by0daP7A9EbhnWkqHNhhIVEmVucGI6p3jOl0hpaF2rCt8/1p72dYPIyLbx9BdbeOLzzaRlFv5cNxERkcsV4R/BMy2fAWDKxinsOL7DodvLteWy+ehm3vvjPb7b/R05No1zJSE9J533/3yfTl924p0t75Cek07DKg15+6a3+d/N/1PiJZdEV76KSb8uOh+bzeCtn/9h4k87sBlQO9iHN+9uzlVhmoYo4kx0/i2cjk3ZYxgGQ5cNZfmB5dStVJdPb/kUD5eSq1B55PQRVh5aycr4layOX01q1r8P2q4VUIuhVw/lhogbNNujGLJzs5m3cx5vb3mbo+lHgbxjOqTZEG684kYdU8lH0w4dTAOc81qz+xhDP9tIYmreNMTnb2tIzxYROomKOAmdfwunY1M2HUs/RvdvunM84zj3NbyPJ1o8Uex1ZeVmseHIBnvCtfPEznzf+7n70SK0BRuObCAlMwWAJsFNGHb1MFqGtbys/agocm25fL/ne97c9CYHTx0EINwnnEeiHuHWWrfq3j0pkJIvB9MA59yOncrk8c8388vfeb9k3dGsGi92a4SPh6ohipR1Ov8WTsem7Fq2fxlDlw3FgoX/xfyvyImQYRjsP7mfFYdWsCp+Fb8n/G6/3wjAgoXGQY1pW60t7cLb0SioEa5WV05mneSDPz/g420f2/u3q9aOYVcPo17leg7ZR2dnGAbLDixj6sap7EreBUAVzyo80OQB7rzyTtxd3E2OUMoyJV8OpgHO+dlsBjN++YeJP/1Nrs2gVrAPb959NfXC9N9TpCzT+bdwOjZl29hVY/ly55eE+YQx77Z5+LkXPO09LTuNtYfXsjJ+JSsOreDQqfzPqwz2CqZteFuuqXYNrau2JtAzsNBtJqUnMWPzDL78+0t7OfQuNbswuNlgIvwiSmzfnN2aw2uYsmEKW5K2AHlXEO9vdD996/XF283b5OjEGSj5cjANcOXH73uPM2T2RhJSM/BwzZuG2KulpiGKlFU6/xZOx6ZsO519mh7f9ODgqYN0rdWVl9u/DORdcdlxYof96tbGIxvzFctwtbrSPKQ57aq1o214W66sdOUlj1H7U/czbdM0ftjzQ946La7ceeWdPNj0QYK8gkpuJ53MlqNbmLJxCmsOrwHAy9WLe+rfQ/+G/QnwCDA5OnEmSr4cTANc+XI8LYvYzzexfEfeNMTbo8J56Y7G+GoaokiZo/Nv4XRsyr5NRzbRf2F/bIaNQY0HkXg6kZWHVnIs41i+flf4XWG/utUyrGWJXX3Zdmwbb2x4g5XxK4G8ZOPeBvdyX8P7Cr0SVx7tPLGTaRunsfTAUiAvwe15ZU8GNRlUoZNRKT4lXw6mAa78sdkM3v5lNxN+2pE3DTHIh+l3X039qvrvK1KW6PxbOB0b5zBlwxTe/ePdfG1erl5Eh0XTrlo72oW3I8LfsVMC1x5ey+QNk/kj6Q8AAj0C+b/G/0fver1LtBpjWXPg5AHe3PQm3+3+DgMDq8VK11pdeTjqYar5VjM7PHFiSr4cTANc+bVu73GGfLqRwyl50xDHdG1In1aahihSVuj8WzgdG+eQbcvm6Z+f5tCpQ7QOb8014dfQLKQZbi5upRqHYRgs2b+ENza8wd7UvQBU9anKI1GP0LVW13JV1e/o6aO8veXtfPe+3VTjJgZHDaZWYC2To5PyQMmXg2mAK9+Op2XxxOebWHZmGuJtTcN5ubumIYqUBTr/Fk7HRoojx5bDN/98w/RN0zly+ggAtQNqM/TqoVwfcb1T/viYa8tl+4nt/H74d9YmrGVtwloyczMBaBfejiHNhtAwqKHJUUp5ouTLwTTAlX82m8G7v+7mtR/zpiHWDPJhet+raRCu/94iZtL5t3A6NnI5MnIy+Gz7Z7z7x7v2hzU3DW7KsKuH0SKshcnRXZjNsLHzxE57orU+cT0ns07m69M0uCmPXf2YnncmDqHky8E0wFUc6/flVUOMT8nA3dXKmK4N6NvqCqf8JVCkPND5t3A6NlISUrNS854RtvVjMnIzAGhfrT2PXf0YV1W+yuTo8hiGwe6U3axNWMvvCb/ze8LvJGcm5+vj6+ZL89DmtAxrSXTVaK6qdJXGbnEYJV8OpgGuYjmRlsUTczezdHvedIxbm1Qlrntj/DxLd36+iOj8eyE6NlKSjp4+mveMsJ1fkmvkYsFCl1pd6FKzC0FeQQR7BVPZs3Kp3Bt29mHTaxPWsvZwXsL13wqRXq5eXB16Na3CWtEqrBX1KtfD1arbBaR0KPlyMA1wFY/NZvDeit28tnAHOTaDyCreTOt7NY2q6TkgIqVJ59/C6diII+xL3ce0jdNYuHfhed9ZLVaqeFbJS8a8gwn2CrYnZmc/B3sHU8WrCm7WS/vB8tCpQ6w9vNY+lfDs/Whnebh4EBUSZU+2GgY1vORtiJQUJV8OpgGu4lq/7wRDZm+wT0Mccn0d6ob64u/lRoCXG/6ebgR4u+Hr7orVqukNIiVN59/C6diII/117C9m/TmLval7OZp+lOMZx7EZtiIvX8mjEkHeQYR4hdiTtXMTNX93f7Ye22qfSnjo1KF8y7tZ3WgS3ITosGhahrWkSXAT3F3cS3o3RYpFyZeDaYCr2JJPZ/Hk3M0s3nak0D5WC/h55iVkAV5u+Hu5nvP+TJJ2zmf7e09X/L3ccHOxluIeiTgPnX8Lp2MjpSnHlsOJjBMcST9C0ukkjqYf5Wj6Ufv7pPQk++ez5d0vhavFlUZBjWgZ1pJWVVsRFRyFp6unA/ZE5PIV9fyribAixRDo7c67/VrwyZr9LN6WSGp6Ninp2aRm5JCSnk1Wjg2bASln2ovDx92Fyr7uVPX3omqgJ2EBnoQHeOX7s4qPu66uiYiIKVytrnlTC72DoUrh/WyGjeTMZI6ePichS0/i6Om8ZO3sn8mZydT0r0nLqi2JDoumWUgzvN28S2+HREqBki+RYrJYLNzTugb3tK5x3ncZ2bn2hCwvKTvzZ3rOv235vs8h9Uzbycy8XwfTsnJJO57OgePphcbg7mIlNMCDqgFeVA3wPOfPM+8DPansrQRNRETMY7VYqexZmcqelbmKslEtUcQsSr5EHMDTzQVPNxdC/C99ekROro2TZ66gHUvLJD45g8Mp6RxOyeBwcgaHUzM4nJzO0VOZZOXaOFCEBC0sIO/K2bkJWqi/JyH+HgT7ehDs54Gnm+OrVYmIiIhUZEq+RMoYVxcrlXzcqeTjTmSQD83Pv7AGQHaujcTUDBJSMohPySAhJZ345LzPZ5O1swna/uOn2X/89AW36+/pSoi/J8G+HvakLMQ/LzEL8fM886cHAV5uek6KiIiISDGUieRr+vTpjB8/noSEBJo2bcrUqVNp1apVof3nzp3LqFGj2Lt3L3Xr1uXVV1+lS5cu9u8Nw2DMmDG8++67JCcn065dO9566y3q1q1r73P8+HGGDBnCt99+i9VqpUePHrzxxhv4+vo6dF9FSoqbi5XqlbypXqnw+fBZOTaOnMzIu2qWknfF7PCZ5CwxNZOjJ/NeWbm2vKmPGafYdeTUBbfr7mIl2M/D/grxOz9B83J3wWoBq8WC1WLBxWrBYgEXq8XeZv/emvf+v9/lLVO8JM8wDGxG3p8GYBhgO1NbyDDA4D/fnynW5eFmxcPVquRSREREHML05GvOnDnExsYyY8YMoqOjmTx5MjExMezYsYOQkJDz+q9atYo+ffoQFxfHrbfeyuzZs+nWrRsbNmygUaNGALz22mtMmTKFWbNmUbNmTUaNGkVMTAxbt27F0zNvGtjdd9/N4cOHWbRoEdnZ2QwYMIAHHniA2bNnl+r+iziSu+vFEzTDMEhNz+HIyQyOnszkyJmE7PzPmXnFRHJtHEpO51By4VMdS5LLmeTMYrHgYrFgYJxJoM4kT2fe2868v1xWC3i7u+Ll7oK3uwtebnl/ntuW1+6Kl7s1r/1MH68z/f5974K3myueblasVgsW8vbDQl7iiQUsZ5LQvO/OtPPf9n+PgYiIiDgv00vNR0dH07JlS6ZNmwaAzWYjIiKCIUOG8Oyzz57Xv1evXqSlpbFgwQJ7W+vWrYmKimLGjBkYhkF4eDhPPPEETz75JAApKSmEhoYyc+ZMevfuzbZt22jQoAG///47LVq0AGDhwoV06dKFgwcPEh4eft52MzMzyczMtH9OTU0lIiJC5XylQsnMybVfLTvynz+Pnsywv8/MsWEzDGy2vCtMNsMg15aXHOUaRoklShXVfxM2y7nvseT7nrwc77zkz2KxnOn/73LnJniWM4mhYZDvv9XZq4V577Enw4D9KiOce5Uxb5lzvz/7+eenrqeSz6U/o0fl1P+lsUlEpGxwilLzWVlZrF+/nuHDh9vbrFYrHTt2ZPXq1QUus3r1amJjY/O1xcTEMH/+fAD27NlDQkICHTt2tH8fEBBAdHQ0q1evpnfv3qxevZrAwEB74gXQsWNHrFYra9as4Y477jhvu3FxcTz//POXs7siTs/D1eWiV9KK6uzUwIISs7NJW177mfdn2uGc5OOcpIECEg97MmLlvKtOZy8inX1vGJCRk0t6Vi6ns/L+TM/O4fQ5n/Pe5+S9zz7blpP/++xcMrJyOZ2d8++6snNLNNk8e6zOaSm5lZci54y6bNHYJCLiXExNvpKSksjNzSU0NDRfe2hoKNu3by9wmYSEhAL7JyQk2L8/23ahPv+d0ujq6krlypXtff5r+PDh+ZK+s78uikjx5E0jBBcslJVCi+6uVvw93Ry2/oKmSdqnUZ5JRM9eKbIZQAH3p9nOrCD/OrAnpvnWybnTMQvY9n+uXJ27vrPbOzdphfxX1/hPm/39fz+f+fNMb3ubv6fpM9+dnsYmERHnopGviDw8PPDw8DA7DBFxYpZzrri5YLlwZ5Ei0NgkIuJcrGZuPCgoCBcXFxITE/O1JyYmEhYWVuAyYWFhF+x/9s+L9Tly5Ei+73Nycjh+/Hih2xUREREREbkcpiZf7u7uNG/enCVLltjbbDYbS5YsoU2bNgUu06ZNm3z9ARYtWmTvX7NmTcLCwvL1SU1NZc2aNfY+bdq0ITk5mfXr19v7LF26FJvNRnR0dIntn4iIiIiIyFmmTzuMjY2lf//+tGjRglatWjF58mTS0tIYMGAAAP369aNatWrExcUB8Nhjj3HdddcxceJEbrnlFj777DPWrVvHO++8A+RN6xk2bBgvvvgidevWtZeaDw8Pp1u3bgDUr1+fTp06MWjQIGbMmEF2djaDBw+md+/eBVY6FBERERERuVymJ1+9evXi6NGjjB49moSEBKKioli4cKG9YMb+/fuxWv+9QNe2bVtmz57NyJEjGTFiBHXr1mX+/Pn2Z3wBPP3006SlpfHAAw+QnJzMNddcw8KFC+3P+AL45JNPGDx4MDfeeKP9IctTpkwpvR0XEREREZEKxfTnfDkrPWdGRMQcOv8WTsdGRMQcRT3/mnrPl4iIiIiISEVh+rRDZ3X2gmFqaqrJkYiIVCxnz7uauHE+jU0iIuYo6tik5KuYTp48CaCHWYqImOTkyZMEBASYHUaZorFJRMRcFxubdM9XMdlsNuLj4/Hz88NiKVsPS01NTSUiIoIDBw44zZx/Z4wZFHdpc8a4nTFmKNtxG4bByZMnCQ8Pz1eQScru2FSW/z5diOIuXc4YtzPGDIrbEYo6NunKVzFZrVaqV69udhgX5O/vX+b+Yl6MM8YMiru0OWPczhgzlN24dcWrYGV9bCqrf58uRnGXLmeM2xljBsVd0ooyNuknQxERERERkVKg5EtERERERKQUKPkqhzw8PBgzZgweHh5mh1JkzhgzKO7S5oxxO2PM4LxxS9nkrH+fFHfpcsa4nTFmUNxmUsENERERERGRUqArXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpUPIlIiIiIiJSCpR8iYiIiIiIlAIlXyIiIiIiIqVAyZeIiIiIiEgpcDU7AGdls9mIj4/Hz88Pi8VidjgiIhWGYRicPHmS8PBwrFb9hngujU0iIuYo6tik5KuY4uPjiYiIMDsMEZEK68CBA1SvXt3sMMoUjU0iIua62Nik5KuY/Pz8gLwD7O/vb3I0IiIVR2pqKhEREfbzsPxLY5OIiDmKOjYp+Sqms9M5/P39NcCJiJhA0+rOp7FJRMRcFxubNFleRERERESkFCj5EhERERERKQVlPvn65Zdf6Nq1K+Hh4VgsFubPn3/RZZYvX87VV1+Nh4cHderUYebMmef1mT59OpGRkXh6ehIdHc3atWtLPngREREREZEzynzylZaWRtOmTZk+fXqR+u/Zs4dbbrmF66+/nk2bNjFs2DD+7//+jx9//NHeZ86cOcTGxjJmzBg2bNhA06ZNiYmJ4ciRI47aDRERERERqeAshmEYZgdRVBaLha+++opu3boV2ueZZ57hu+++488//7S39e7dm+TkZBYuXAhAdHQ0LVu2ZNq0aUDec1EiIiIYMmQIzz77bJFiSU1NJSAggJSUFN3ULCJSinT+LZyOjYiIOYp6/i131Q5Xr15Nx44d87XFxMQwbNgwALKysli/fj3Dhw+3f2+1WunYsSOrV68udL2ZmZlkZmbaP6emppZs4OJ8Th+HFZNg/xqzIxFxTnfPBa9As6NwahqbHOv4hx+S+v0PZochJcDq60vwY0PxatzY7FCkgit3yVdCQgKhoaH52kJDQ0lNTSU9PZ0TJ06Qm5tbYJ/t27cXut64uDief/55h8QsTiYnE9a+A7+Mh4wUs6MRcV62XLMjcHoamxwnJymJxNfGQ06O2aFICcnYto2a8+bhFhpidihSgZW75MtRhg8fTmxsrP3z2QepSQViGPDXV7B4LCTvy2sLbQRth4CHHvYqcsk8fM2OwOlpbHKclG++hZwcPOrXJ3jwo2aHI5fp6JSpZO7YwaHYWGrM/ACLm5vZIUkFVe6Sr7CwMBITE/O1JSYm4u/vj5eXFy4uLri4uBTYJywsrND1enh44OHh4ZCYxQnsXwM/PQcHf8/77BsGN4yEqL5gdTE3NhGpsDQ2OYZhGCR/+SUAlfr0xu/GG02OSC6XR5067OlxJ+nr13Nk0uuEPvO02SFJBVXmqx1eqjZt2rBkyZJ8bYsWLaJNmzYAuLu707x583x9bDYbS5YssfcRsTu+Gz7vD+/fnJd4uXlDh+EwdANcfa8SLxGRcihj82ay/vkHi5cX/l26mB2OlAD3GjWoGvcyAMc/+IDUH38yOSKpqMp88nXq1Ck2bdrEpk2bgLxS8ps2bWL//v1A3pSLfv362fs/9NBD7N69m6effprt27fz5ptv8vnnn/P444/b+8TGxvLuu+8ya9Ystm3bxsMPP0xaWhoDBgwo1X2TMuz0cVg4Aqa1gq3zAQs0uxeGboQOz4K7j9kRioiIg5y96uUfE4OLr6bHlhf+N91E5fvvB+DwiBFk7tljckRSEZX5aYfr1q3j+uuvt38+O7e9f//+zJw5k8OHD9sTMYCaNWvy3Xff8fjjj/PGG29QvXp13nvvPWJiYux9evXqxdGjRxk9ejQJCQlERUWxcOHC84pwSAWUkwW/vws/vwYZyXlttW+Am16AsEamhiYiIo5nS0sj9bvvAQjs0d3kaKSkhcQ+TsaWLZxet45Djw0jcs5nWL28zA5LKhCnes5XWaJnqZQzhgFbv84rpnHizC9hIQ3g5hegTscLLioipUvn38Lp2Fy+5HlfcXjECNxr1KDWwh+wWCxmhyQlLPvIEfZ070FuUhIBt99O1Vfi9N9ZLltRz79lftqhiMMd+B3ej4G5/fMSL99Q6DoFHlqhxEtEpII5O+UwoEcP/YO8nHILCaHapIng4kLK11+T/Plcs0OSCkTJl1RcJ/bC3AHwv45wYA24esF1z8CQDdC8v4ppiIhUMJl79pC+fj1YrQTcfrvZ4YgD+bRqRcjjwwBIfPFF0v/409yApMJQ8iUVT/oJ+GkkTGsJf80DLBB1T14Fw+tH6NlDIiIVVMq8eQD4XnutHsRbAVQeOBDfG2/EyM7m0GOPkZucbHZIUgEo+ZKKIycLfpsBU5rBqqmQmwW1OsBDv0K36eAfbnaEIiJiEiMnh+T58wEIUKGNCsFisRAe9zJuV1xBdnw8h555BsNmMzssKefKfLVDqUCy08GW65h1714Gi0bnPbcLILge3Pxi3j1dmtMvIlLhnfrlV3KPJuFSpQp+HTqYHY6UEhd/f6q/MZm9vfuQ9vMvHHvnHYIeesjssKQcU/IlZcPKN/KSI0fzCYbrn8t7ZpeL/vqLiEgee6GN22/H4uZmcjRSmjzr1yds9GgOP/ccR6dMxatJE3zatjU7LCmnNO1QzHfsH1j6kmO34eYD7Z/Me0hyiwFKvERExC7n6FFOLV8O6NleFVVgj+4E3NkDbDYOPfEk2QkJZock5ZT+BSrmMgxY+CzkZkKt66H3bMdMA7S6KeESEZECpXzzDeTm4hUVhUft2maHIyYJGzmSjL+2krltG4eGPU6ND2dhcXc3OywpZ3TlS8y14wfY+VNectRlPLh7g5tXyb+UeImISAEMwyD5i7PP9tJVr4rM6ulJ9SlvYPXzI33TJhInTDA7JCmHlHyJebLTYeEzee/bDoaguubGIyIiFU76xk1k7dmDxdsb/85dzA5HTOYeEUH4q68AcOLDj0j94QeTI5LyRsmXmGfF65C8H/yrw7VPmR2NiIhUQMlffgGAf6dOuPj6mByNlAV+N9xAlUGDADj83Egyd+82OSIpT5R8iTmO74YVk/Pex7wE7hrwRESkdOWeSiP1h4WACm1IfsGPDcU7Ohrb6dMcHDoUW1qa2SFJOaHkS8yxcPiZIhsdoMHtZkcjIiIV0MkfF2KcPo17ZCReV19tdjhShlhcXak2cQKuwcFk7fqHw2PGYhiG2WFJOaDkS0rfjh/g74V5RTY6j9dDjkVExBTnFtqwaCyS/3ANCqLa65PAxYXUBQs48emnZock5YCSLyld2enww5kiG20eheArzY1HREQqpMzdu0nfuBFcXAjs1s3scKSM8m7RgpAnngAgMe4V0rdsMTkicXZKvqR0rXwDkveBfzUV2RAREdMkf5l31cv3uutwDQ42ORopyyoPuA+/m26C7GwODhtGzokTZockTkzJl5Se43vg10l572NeAg9fc+MREZEKycjOJmX+14AKbcjFWSwWqsa9jHuNGuTEHyb+qacxcnPNDkuclJIvKT1ni2zUvA4adDM7GhERqaBO/fILuceO4RIUhO+115odjjgBF19fqk2ZgsXTk7QVK0h6a4bZIYmTUvIlpWPHQvj7B7C6QhcV2RAREfPYC23cfhsWNzeToxFn4XnVlYSNHQNA0vTpnPp1hckRiTNS8iWOl50BC88tsnGVufGIiEiFlX3kCKd++QWAwB49TI5GnE1gt24E9uwJhkH8U0+RHR9vdkjiZJR8ieOtfANO7AW/cLj2abOjERGRCizl668hNxevZs3wqFXL7HDECYU+NwLPhg3JTU7m4LDHsWVlmR2SOBFXswOQcu7EXlihIhsiImI+wzBI+XIeAIF36qqXFI/Vw4Nqb7zBnh49yNiyhcS4OIKHDjU7LCkBFqsVl4AAh25DyZc41sLhkJMBNa+FhneYHY2IiFRg6Rs2kLV3LxZvb/w7dTI7HHFi7tWrEf7qKxx86GGSP/2M5E8/MzskKQEe9etT66t5Dt2Gph2K4/z9I+z4/kyRjQkqsiEiIqY6W2jDv3MnrD4+Jkcjzs6vQwdCnn4ai7u72aGIE9GVL3GM7Az44cz9Xa0fUZENERExVe6pU6QuXAhAYI87TY5Gyosq9w+g8n39zQ5DnIiSL3GMVVPOFNmoCtepyIaIiJgr9YcfMNLTca9ZE69mUWaHI+WIxaqJZFJ0+tsiJe/EXvh1Yt77m18EDz9TwxEREUk5M+Uw8M4eWDQNXkRMouRLSt7CEXlFNiLbQyNVkxIREXNl7tpF+ubN4OJCwO23mx2OiFRgSr6kZP39E+z4TkU2RESkzEg+U17et0MHXIOCTI5GRCoyJV9Scs4tshH9EITUMzceERGp8IysrLwHKwOBPTQbQ0TMpeRLSs6qqXBiT16RjQ7Pmh2NiIgIJ3/+mdzjx3EJDsL32vZmhyMiFZySLykZJ/apyIaIiJQ59kIb3bphcVWRZxExl5IvKRk/joCcdKhxjYpsiIhImZCdmMipX38FIKB7d5OjERFR8iUlYedi2L4ALC5wi4psiIhI2ZAy/2uw2fBq3hyPmjXNDkdExDmSr+nTpxMZGYmnpyfR0dGsXbu20L7Z2dmMGzeO2rVr4+npSdOmTVl45on2Z40dOxaLxZLvVa+eikMUS04m/PBU3vvWD0NIfXPjERERAQzDIHnemSmHKrQhImVEmU++5syZQ2xsLGPGjGHDhg00bdqUmJgYjhw5UmD/kSNH8vbbbzN16lS2bt3KQw89xB133MHGjRvz9WvYsCGHDx+2v1asWFEau1P+rJoCx3eDbxhc94zZ0YiIiACQvm4d2fv2Y/X2xj/mZrPDEREBnCD5mjRpEoMGDWLAgAE0aNCAGTNm4O3tzfvvv19g/48++ogRI0bQpUsXatWqxcMPP0yXLl2YOHFivn6urq6EhYXZX0F67selS94Pv5xTZMPT39x4REREzkg+U2jD/5YuWH18TI5GRCRPmU6+srKyWL9+PR07drS3Wa1WOnbsyOrVqwtcJjMzE09Pz3xtXl5e513Z2rlzJ+Hh4dSqVYu7776b/fv3XzCWzMxMUlNT870qvIXD/y2y0fhOs6MREalwNDYVLPfkSVJ//BHQlEMRKVvKdPKVlJREbm4uoaGh+dpDQ0NJSEgocJmYmBgmTZrEzp07sdlsLFq0iHnz5nH48GF7n+joaGbOnMnChQt566232LNnD+3bt+fkyZOFxhIXF0dAQID9FRERUTI76ax2nVNko8t4FdkQETGBxqaCpX7/A0ZGBu61a+PZtKnZ4YiI2JXp5Ks43njjDerWrUu9evVwd3dn8ODBDBgwAKv1313t3Lkzd911F02aNCEmJobvv/+e5ORkPv/880LXO3z4cFJSUuyvAwcOlMbulE05mfD903nvox+C0AbmxiMiUkFpbCpY8pf/Ftqw6MdBESlDyvTTBoOCgnBxcSExMTFfe2JiImFhYQUuExwczPz588nIyODYsWOEh4fz7LPPUqtWrUK3ExgYyJVXXsmuXbsK7ePh4YGHh0fxdqS8WTUVjv8DvqHQ4VmzoxERqbA0Np0v4++/ydiyBVxdCbj9NrPDERHJp0xf+XJ3d6d58+YsWbLE3maz2ViyZAlt2rS54LKenp5Uq1aNnJwcvvzyS26//fZC+546dYp//vmHqlWrlljs5VbyAfhlQt57FdkQEZEyJuXLeQD4Xd8B1ypVzA1GROQ/ynTyBRAbG8u7777LrFmz2LZtGw8//DBpaWkMGDAAgH79+jF8+HB7/zVr1jBv3jx2797Nr7/+SqdOnbDZbDz99NP2Pk8++SQ///wze/fuZdWqVdxxxx24uLjQp0+fUt8/p/PjiDNFNtpB47vMjkZERMTOyMoi5ZtvAAhQoQ0RKYPK9LRDgF69enH06FFGjx5NQkICUVFRLFy40F6EY//+/fnu58rIyGDkyJHs3r0bX19funTpwkcffURgYKC9z8GDB+nTpw/Hjh0jODiYa665ht9++43g4ODS3j3nkZMJv70J275RkQ0RESmTTi5bTu6JE7gGB+N7zTVmhyMich6LYRiG2UE4o9TUVAICAkhJScHfvxxPvTMM2Po1LB4DJ/bmtbUdkjflUETEBBXm/FsMFf3Y7H/gAdJ++ZUqDzxASOzjZocjIhVIUc+/Zf7Kl5jowO/w03NwYE3eZ99QuGEkRN1tblwiIiL/kZ2QQNqKlQAEdr/D5GhERAqm5EvOd2IvLB4Lf32V99nNG9oOzbvi5eFrZmQiIiIFSpk/H2w2vFu0wD0y0uxwREQKpORL/pV+Iq+S4dp3IDcLsECzu+H658A/3OzoRERECmTYbCSfqXKoQhsiUpYp+RLIyYJ1/4OfX81LwABqdci7ryussamhiYiIXMzp39eRfeAAVh8f/GNuNjscEZFCKfmqyAwDtn2bV0zj+O68tuD6eUlXnRtVzVBERJxC8pdfAOB/yy1Yvb1NjkZEpHBKviqqg+vzimnsX5332ScEbngOou4BF/21EBER55CbmsrJH38CILBHd5OjERG5MP0ru6I5sQ+WPA9/fpn32dUrr5BGu6Hg4WdubCIiIpco9fvvMTIz8ahbB88mTcwOR0TkgpR8VRTpyfDrRFgz499iGlF980rHq5iGiIg4odyUFI69+x4AAd17YNF0eREp45R8lXe52bDufVj+CqQfz2ureV3efV1V9QuhiIg4J8NmI/7Z4WQfOoRbtWoE3qkqhyJS9in5Kq8MA7Z/B4tGw/F/8tqCrspLuurepGIaIiLi1I699z9OLVuGxc2Nam+8gYufps6LSNmn5Ks8OrQefhoF+1bmffYJhutHQLN+KqYhIiJOL+23NRydPBmA0JEj8WrU0NyARESKSP8SL28ObYD3bgIjF1w9oc1gaPcYePqbHZmIiMhly05M5NATT4DNRkC3bgT2vMvskEREikzJV3nz9495iVdENNz5PgRUNzsiERGREmFkZ3Po8Vhyjx3D46qrCBszWkU2RMSpWM0OQEpY4p95fza8Q4mXiIiUK0cmTiJ9wwasvr5Uf2MyVi8vs0MSEbkkSr7Km8S/8v4M1fx3EREpP1J//InjM2cCUPXll3CPjDQ1HhGR4nBI8hUZGcm4cePYv3+/I1Yvhck8BSf25L0PUfIlIiLlQ+aePRweMQKAygMG4H/zzSZHJCJSPA5JvoYNG8a8efOoVasWN910E5999hmZmZmO2JSc68i2vD/9qoJPFXNjERERKQG206c5NPQxbGlpeLVoTkjs42aHJCJSbA5LvjZt2sTatWupX78+Q4YMoWrVqgwePJgNGzY4YpMC/97vpSmHIiJSDhiGQcLzz5O5cycuQUFUmzQJi5ub2WGJiBSbQ+/5uvrqq5kyZQrx8fGMGTOG9957j5YtWxIVFcX777+PYRiO3HzFo+RLRETKkeQ5n5Py9TdgtVJt4kTcQkLMDklE5LI4tNR8dnY2X331FR988AGLFi2idevWDBw4kIMHDzJixAgWL17M7NmzHRlCxWIvttHI3DhEREQuU/off5L40ksABD8+DJ/oViZHJCJy+RySfG3YsIEPPviATz/9FKvVSr9+/Xj99depV6+evc8dd9xBy5YtHbH5iskwVOlQRETKhdzkZA499hhGdja+N9xAlf/7P7NDEhEpEQ5Jvlq2bMlNN93EW2+9Rbdu3XArYH52zZo16d27tyM2XzGlHIDMVLC6QZW6ZkcjIiJSLIbNxqFnniE7Ph63iAjCX4nTg5RFpNxwSPK1e/duatSoccE+Pj4+fPDBB47YfMV09qpX8FXg6m5uLCIiIsV07O23Sfv5FyweHlSf8gYu/v5mhyQiUmIcUnDjyJEjrFmz5rz2NWvWsG7dOkdsUlRsQ0REnFzaqlUcnTIVgLDRo/CsX9/kiERESpZDkq9HH32UAwcOnNd+6NAhHn30UUdsUnS/l4iIOLHshAQOPfEkGAYBPboT2KOH2SGJiJQ4hyRfW7du5eqrrz6vvVmzZmzdutURmxQlXyIi4qSMrCwODXuc3BMn8Khfn7BRo8wOSUTEIRySfHl4eJCYmHhe++HDh3F1dWh1+4opOx2O7cp7rzLzIiLiZBInTCB90yasfn5Uf2MyVk9Ps0MSEXEIhyRfN998M8OHDyclJcXelpyczIgRI7jpppscscmK7eh2MGzgXQV8Q82ORkREpMhSf/iBEx9+BED4K3G4X3GFyRGJiDiOQy5DTZgwgWuvvZYaNWrQrFkzADZt2kRoaCgfffSRIzZZsZ075VDleEVExElk7t7N4edGAlBl0P/hd+ONJkckIuJYDkm+qlWrxpYtW/jkk0/YvHkzXl5eDBgwgD59+hT4zC+5TPbkS1MORUTEOdjS0jg4dCi206fxbtWK4MceMzskERGHc9gNWD4+PjzwwAOOWr2cS2XmRUTEiRiGweExY8na9Q+uwcFUmzgBi+4JF5EKwKFnuq1bt7J//36ysrLytd92222O3GzFYhiQoORLREScx4lPPyV1wQJwcaHa65NwDQ42OyQRkVLhkORr9+7d3HHHHfzxxx9YLBYMwwDAcuZ+pNzcXEdstmI6lQjpx8FiheB6ZkcjIiJyQelbtpAY9woAIbGxeLdoYXJEIiKlxyHVDh977DFq1qzJkSNH8Pb25q+//uKXX36hRYsWLF++/JLXN336dCIjI/H09CQ6Opq1a9cW2jc7O5tx48ZRu3ZtPD09adq0KQsXLrysdZZpZ6ccVqkDbl7mxiIiInIBOSdOcHDYMMjOxu+mm6h8/wCzQxIRKVUOSb5Wr17NuHHjCAoKwmq1YrVaueaaa4iLi2Po0KGXtK45c+YQGxvLmDFj2LBhA02bNiUmJoYjR44U2H/kyJG8/fbbTJ06la1bt/LQQw9xxx13sHHjxmKvs0zTw5VFRMQJGLm5xD/1NDnxh3GrcQVVX37JPiNGRKSicEjylZubi5+fHwBBQUHEx8cDUKNGDXbs2HFJ65o0aRKDBg1iwIABNGjQgBkzZuDt7c37779fYP+PPvqIESNG0KVLF2rVqsXDDz9Mly5dmDhxYrHXWaYp+RIRESeQ9NYM0laswOLpSfUpU3A58+8EEZGKxCH3fDVq1IjNmzdTs2ZNoqOjee2113B3d+edd96hVq1aRV5PVlYW69evZ/jw4fY2q9VKx44dWb16dYHLZGZm4unpma/Ny8uLFStWFHudZ9ebmZlp/5yamlrk/XAolZkXEamwyuzY9B+nVqwkafp0AMLGjMHzqqtMjkhExBwOufI1cuRIbDYbAOPGjWPPnj20b9+e77//nilTphR5PUlJSeTm5hIaGpqvPTQ0lISEhAKXiYmJYdKkSezcuRObzcaiRYuYN28ehw8fLvY6AeLi4ggICLC/IiIiirwfDpOTBUfPXEnUlS8RkQqnTI5N/5EdH0/8k0+CYRB4110E3tHN7JBEREzjkOQrJiaG7t27A1CnTh22b99OUlISR44c4YYbbnDEJu3eeOMN6tatS7169XB3d2fw4MEMGDAAq/XydnX48OGkpKTYXwcOHCihiC/DsZ1gywYPfwgoewOuiIg4Vpkcm85hy8ri4LDHyU1OxrNBA0JHPmd2SCIipirx5Cs7OxtXV1f+/PPPfO2VK1e+5Btrg4KCcHFxITExMV97YmIiYWFhBS4THBzM/PnzSUtLY9++fWzfvh1fX1/7dMfirBPAw8MDf3//fC/TnXu/l25aFhGpcMrk2HSOI6+8SsaWLVj9/ak25Q2sHh5mhyQiYqoST77c3Ny44oorSuRZXu7u7jRv3pwlS5bY22w2G0uWLKFNmzYXXNbT05Nq1aqRk5PDl19+ye23337Z6yxzEvVwZRERKZtSvl3AidmzAQh/7VXcq1c3OSIREfM5ZNrhc889x4gRIzh+/Phlrys2NpZ3332XWbNmsW3bNh5++GHS0tIYMCDv2SD9+vXLVzxjzZo1zJs3j927d/Prr7/SqVMnbDYbTz/9dJHX6TRU6VBERMqgzF27ODx6NABVHnwQvw4dzA1IRKSMcEi1w2nTprFr1y7Cw8OpUaMGPj4++b7fsGFDkdfVq1cvjh49yujRo0lISCAqKoqFCxfaC2bs378/3/1cGRkZjBw5kt27d+Pr60uXLl346KOPCAwMLPI6nYYqHYqISBmTeyqNg0Mfw0hPx7t1a4KHDjE7JBGRMsNiGIZR0it9/vnnL/j9mDFjSnqTpS41NZWAgABSUlLMmWOfdgzGnynbP/wgeOh5KSJSMZh+/i3DzD42hmEQ/8QTpH7/A64hIdT8ah6uVaqUehwiIqWtqOdfh1z5Kg/JVZl35MxVr0qRSrxERKRMOPHRx6R+/wO4ulJt8mQlXiIi/+GQe76kFGjKoYiIlCGnN24k8bXXAAh96km8r25mckQiImWPQ658Wa3WC5aVL4lKiBWeKh2KiEgZkXP8OIcej4WcHPw6daJSv35mhyQiUiY5JPn66quv8n3Ozs5m48aNzJo166L3g0kRqdKhiIiUAUZuLvFPPklOQgLukZFUffGFS36up4hIReGQ5OvsM7XOdeedd9KwYUPmzJnDwIEDHbHZisOWC0e25b3XtEMRETFR0vTppK1ajcXLi2pT3sDF19fskEREyqxSveerdevW+R5uLMV0fDfkZICbd17BDREREROc+vlnkt58C4Cq457H88orTY5IRKRsK7XkKz09nSlTplCtWrXS2mT5dfZ+r5D6YHUxNxYREamQsg4e4tDTzwAQ2Kc3AV27mhyRiEjZ55Bph5UqVco339swDE6ePIm3tzcff/yxIzZZseh+LxERMZEtK4tDw4ZhS0nBs3FjQocPNzskERGn4JDk6/XXX8+XfFmtVoKDg4mOjqZSpUqO2GTFojLzIiJiosSXXybjzz9xCQig+uTXsbq7mx2SiIhTcEjydd999zlitXKWysyLiIhJUr75huTP5oDFQviE8bjpdgIRkSJzyD1fH3zwAXPnzj2vfe7cucyaNcsRm6w4MlIgeX/e+5AG5sYiIiIVSsaOvzk8egwAQQ8/jG/79iZHJCLiXBySfMXFxREUFHRee0hICC+//LIjNllxnC0x718NvCubG4uIiFQYuadOcWjoUIyMDHzatSPo0UfMDklExOk4JPnav38/NWvWPK+9Ro0a7N+/3xGbrDg05VBEREqZYRgcHvEcWfv24RoWRviE8VhcVG1XRORSOST5CgkJYcuWLee1b968mSpVqjhikxWHKh2KiEgpOz5rFid/+gnc3Kg++XVcVTxLRKRYHJJ89enTh6FDh7Js2TJyc3PJzc1l6dKlPPbYY/Tu3dsRm6w4VOlQRERK0ekNGzgyYSIAoc88g1dUlLkBiYg4MYdUO3zhhRfYu3cvN954I66ueZuw2Wz069dP93xdDpsNErfmvdeVLxERcbCcpCQODXsccnLw79KFSnf3NTskERGn5pDky93dnTlz5vDiiy+yadMmvLy8aNy4MTVq1HDE5iqOlP2QdRJc3KFKHbOjERGRcszIyeHQE0+Sc+QI7rVrU/WFcfme4SkiIpfOIcnXWXXr1qVu3bqO3ETFcnbKYfBV4OJmbiwiIlKuHZ0yldNr1mDx9qb6lDew+viYHZKIiNNzyD1fPXr04NVXXz2v/bXXXuOuu+5yxCYrBt3vJSIipeDk0mUce+cdAMJffAGP2rVNjkhEpHxwSPL1yy+/0KVLl/PaO3fuzC+//OKITVYMKjMvIiIOlnXgAPHPPgtApXvuwb+A8VxERIrHIcnXqVOncHd3P6/dzc2N1NRUR2yyYlCZeRERcSBbZiYHH3sMW2oqXk2bEvr0U2aHJCJSrjgk+WrcuDFz5sw5r/2zzz6jQYMGjthk+Zd1Go79k/de0w5FRMQBEl98kcyt23CpVIlqk1/HUsAPqSIiUnwOKbgxatQounfvzj///MMNN9wAwJIlS5g9ezZffPGFIzZZ/h3dBhjgEwy+IWZHIyIi5UzyvK9InvsFWCyETxiPW9WqZockIlLuOCT56tq1K/Pnz+fll1/miy++wMvLi6ZNm7J06VIqV67siE2Wf5pyKCIiDpS5YwcAQUMG49uuncnRiIiUTw4rNX/LLbdwyy23AJCamsqnn37Kk08+yfr168nNzXXUZssvVToUEREHCh3+LD7XtsenTRuzQxERKbcccs/XWb/88gv9+/cnPDyciRMncsMNN/Dbb785cpPll658iYiIg/m2a4fF6tB/GoiIVGglfuUrISGBmTNn8r///Y/U1FR69uxJZmYm8+fPV7GN4jIMlZkXEREREXFyJfrzVteuXbnqqqvYsmULkydPJj4+nqlTp5bkJiqmk4ch/QRYXCDoKrOjERERERGRYijRK18//PADQ4cO5eGHH6Zu3bolueqK7eyUw6C64OZpbiwiIiIiIlIsJXrla8WKFZw8eZLmzZsTHR3NtGnTSEpKKslNVEyacigiIiIi4vRKNPlq3bo17777LocPH+bBBx/ks88+Izw8HJvNxqJFizh58mRJbq7iULENERERERGn55CSRj4+Ptx///2sWLGCP/74gyeeeIJXXnmFkJAQbrvtNkdssnxTmXkREREREafn8HqyV111Fa+99hoHDx7k008/dfTmyp+cTEj6O++9rnyJiIiIiDitUnuYh4uLC926deObb74prU2WD0l/gy0HPAPAv5rZ0YiIiIiISDE5xZMUp0+fTmRkJJ6enkRHR7N27doL9p88eTJXXXUVXl5eRERE8Pjjj5ORkWH/fuzYsVgslnyvevXqOXo3iufcKYcWi7mxiIiIiIhIsZX4Q5ZL2pw5c4iNjWXGjBlER0czefJkYmJi2LFjByEhIef1nz17Ns8++yzvv/8+bdu25e+//+a+++7DYrEwadIke7+GDRuyePFi+2dX1zJ6KFTpUERERESkXCjzV74mTZrEoEGDGDBgAA0aNGDGjBl4e3vz/vvvF9h/1apVtGvXjr59+xIZGcnNN99Mnz59zrta5urqSlhYmP0VFBRUGrtz6VTpUERERESkXCjTyVdWVhbr16+nY8eO9jar1UrHjh1ZvXp1gcu0bduW9evX25Ot3bt38/3339OlS5d8/Xbu3El4eDi1atXi7rvvZv/+/ReMJTMzk9TU1HyvUqFKhyIiUgjTxiYRESmWMp18JSUlkZubS2hoaL720NBQEhISClymb9++jBs3jmuuuQY3Nzdq165Nhw4dGDFihL1PdHQ0M2fOZOHChbz11lvs2bOH9u3bX/A5ZHFxcQQEBNhfERERJbOTF3LqKJxKBCwQXEbvSRMREdOYMjaJiEixlenkqziWL1/Oyy+/zJtvvsmGDRuYN28e3333HS+88IK9T+fOnbnrrrto0qQJMTExfP/99yQnJ/P5558Xut7hw4eTkpJifx04cMDxO3PkzFWvyjXBw9fx2xMREadiytgkIiLFVkarTOQJCgrCxcWFxMTEfO2JiYmEhYUVuMyoUaO49957+b//+z8AGjduTFpaGg888ADPPfccVuv5+WZgYCBXXnklu3btKjQWDw8PPDw8LmNvikH3e4mIyAWYMjaJiEixlekrX+7u7jRv3pwlS5bY22w2G0uWLKFNmzYFLnP69OnzEiwXFxcADMMocJlTp07xzz//ULVq1RKKvITofi8RERERkXKjTF/5AoiNjaV///60aNGCVq1aMXnyZNLS0hgwYAAA/fr1o1q1asTFxQHQtWtXJk2aRLNmzYiOjmbXrl2MGjWKrl272pOwJ598kq5du1KjRg3i4+MZM2YMLi4u9OnTx7T9LJDKzIuIiIiIlBtlPvnq1asXR48eZfTo0SQkJBAVFcXChQvtRTj279+f70rXyJEjsVgsjBw5kkOHDhEcHEzXrl156aWX7H0OHjxInz59OHbsGMHBwVxzzTX89ttvBAcHl/r+FSo3B45sz3uv5EtERERExOlZjMLm4skFpaamEhAQQEpKCv7+/iW/gaM7YHorcPOB4QehgHvVREQqIoeff52Yjo2IiDmKev7Vv+jLKvuUwwZKvEREREREygH9q76sUqVDEREREZFyRclXWaVKhyIiIiIi5YqSr7JKV75ERERERMoVJV9lUXoypBzIex/SwNRQRERERESkZCj5KouObM37MyACvAJNDUVEREREREqGkq+ySFMORURERETKHSVfZZG9zLySLxERERGR8kLJV1mkK18iIiIiIuWOkq+yxmaDxDP3fIU2NjcWEREREREpMUq+yprkvZCdBq6eULmW2dHI/7d398FR1fcexz+bhA0h5EEgJAQDQQRCYIUolpo4ypQMDGoqw1QQacAw1aGNlUC1UCViqyRChzSACMWplTrS0mlBW1rtpGmIYhVCIl7RXB4kkJSHhIjkiQvE7Ll/cNnLSkgCZM85u7xfMzsTTnbPfnYnyXc+e875AQAAAHQTypfdXDzlMCZJCg6xNgsAAACAbkP5shvP9V6jrc0BAAAAoFtRvuyGlQ4BAACAgET5shtWOgQAAAACEuXLTs41S6eqLnxN+QIAAAACCuXLTk7+tyRD6h0nhfezOg0AAACAbkT5shOu9wIAAAACFuXLTrjeCwAAAAhYlC87YZl5AAAAIGBRvuzCMDjtEAAAAAhglC+7aDwqnW2QgkKkfsOtTgMAAACgm1G+7OLiKYf9RkghTmuzAAAAAOh2lC+74JRDAAAAIKBRvuyClQ4BAACAgEb5sgtWOgQAAAACGuXLDlrPSvUHLnzNkS8AAAAgIFG+7KB+n2S0SWF9pIg4q9MAAAAA8AHKlx1cer2Xw2FtFgAAAAA+QfmyA673AgAAAAIe5csOWGYeAAAACHiULztgmXkAAAAg4FG+rNZcJ7WclBxBUkyS1WkAAAAA+Ajly2oXTznsM1Ry9rI2CwAAAACf8YvytXbtWiUmJqpnz54aP368du3a1eH9CwsLNWLECIWFhSkhIUELFizQ2bNnr2ufPsMphwAAAMANwfbla/PmzVq4cKGWLl2qiooKjRkzRpMnT1ZdXV2799+0aZMWL16spUuXqrKyUr/5zW+0efNmPfPMM9e8T59ipUMAAADghmD78lVQUKDHHntMWVlZSk5O1vr169WrVy+99tpr7d7/3//+t9LS0vTII48oMTFRkyZN0syZM72ObF3tPiXp3Llzamxs9Lp1C1Y6BABcI5/NJgCAT9i6fJ0/f17l5eVKT0/3bAsKClJ6ero+/PDDdh+Tmpqq8vJyT9k6dOiQ/v73v+u+++675n1KUn5+vqKiojy3hISE63+Bba3SyX0XvqZ8AQCukk9mEwDAZ2xdvurr69XW1qbY2Fiv7bGxsTpx4kS7j3nkkUf0i1/8Qnfffbd69OihoUOHasKECZ7TDq9ln5L0s5/9TA0NDZ5bTU3Ndb46SV8elNrOS84IKXrQ9e8PAHBD8clsAgD4jK3L17XYvn278vLy9Morr6iiokJbtmzR3/72N73wwgvXtd/Q0FBFRkZ63a7biUtOOXQ4rn9/AIAbik9mEwDAZ0KsDtCRfv36KTg4WLW1tV7ba2trFRcX1+5jcnNzlZmZqR/84AeSJJfLpZaWFj3++ON69tlnr2mfPsP1XgAAAMANw9ZHvpxOp+644w4VFxd7trndbhUXF+uuu+5q9zFnzpxRUJD3ywoODpYkGYZxTfv0GZaZBwAAAG4Ytj7yJUkLFy7UnDlzNG7cOH3rW99SYWGhWlpalJWVJUmaPXu2Bg4cqPz8fElSRkaGCgoKlJKSovHjx+vgwYPKzc1VRkaGp4R1tk/TsMw8AAAAcMOwffmaMWOGTp48qeeee04nTpzQ2LFj9e6773oWzKiurvY60rVkyRI5HA4tWbJER48eVUxMjDIyMrRs2bIu79MUZ05JTccufN1/pHnPCwAAAMASDsMwDKtD+KPGxkZFRUWpoaHh2i5wrnpf2viAFD1Yyvmv7g8IAAHquv/+BjDeGwCwRlf//tr6mq+AximHAAAAwA2F8mUVVjoEAAAAbiiUL6uw0iEAAABwQ6F8WcHdJtVVXvia0w4BAACAGwLlywqnqqSv/0cKCZP6DLE6DQAAAAATUL6scPF6r/4jpaBga7MAAAAAMIXt/5+vgDRiijRvh9T6P1YnAQAAAGASypcVQkKlOJfVKQAAAACYiNMOAQAAAMAElC8AAAAAMAHlCwAAAABMQPkCAAAAABNQvgAAAADABJQvAAAAADABS81fI8MwJEmNjY0WJwGAG8vFv7sX/w7j/zGbAMAaXZ1NlK9r1NTUJElKSEiwOAkA3JiampoUFRVldQxbYTYBgLU6m00Og48Or4nb7daxY8cUEREhh8NhdRwvjY2NSkhIUE1NjSIjI62O0yX+mFkit9n8Mbc/ZpbsndswDDU1NSk+Pl5BQZw9fym7ziY7/zx1hNzm8sfc/phZIrcvdHU2ceTrGgUFBenmm2+2OkaHIiMjbfeD2Rl/zCyR22z+mNsfM0v2zc0Rr/bZfTbZ9eepM+Q2lz/m9sfMErm7W1dmEx8ZAgAAAIAJKF8AAAAAYALKVwAKDQ3V0qVLFRoaanWULvPHzBK5zeaPuf0xs+S/uWFP/vrzRG5z+WNuf8wskdtKLLgBAAAAACbgyBcAAAAAmIDyBQAAAAAmoHwBAAAAgAkoXwAAAABgAspXgMjPz9edd96piIgI9e/fX1OnTtW+ffusjnXVXnrpJTkcDuXk5FgdpVNHjx7V97//ffXt21dhYWFyuVzavXu31bGuqK2tTbm5uRoyZIjCwsI0dOhQvfDCC7LbmjvvvfeeMjIyFB8fL4fDobfeesvr+4Zh6LnnntOAAQMUFham9PR0HThwwJqwl+god2trqxYtWiSXy6Xw8HDFx8dr9uzZOnbsmHWB/09n7/el5s2bJ4fDocLCQtPywb8FwmxiLvkWs8m3mE32Q/kKEKWlpcrOztZHH32koqIitba2atKkSWppabE6WpeVlZXp17/+tW677Taro3Tqq6++Ulpamnr06KF33nlHn3/+uVauXKmbbrrJ6mhXtHz5cq1bt04vv/yyKisrtXz5cq1YsUJr1qyxOpqXlpYWjRkzRmvXrm33+ytWrNDq1au1fv167dy5U+Hh4Zo8ebLOnj1rclJvHeU+c+aMKioqlJubq4qKCm3ZskX79u3Td7/7XQuSeuvs/b5o69at+uijjxQfH29SMgQCf59NzCXfYzb5FrPJhgwEpLq6OkOSUVpaanWULmlqajKGDRtmFBUVGffee68xf/58qyN1aNGiRcbdd99tdYyrcv/99xtz58712jZt2jRj1qxZFiXqnCRj69atnn+73W4jLi7O+OUvf+nZdvr0aSM0NNT4/e9/b0HC9n0zd3t27dplSDKOHDliTqguuFLu//znP8bAgQONvXv3GoMHDzZ+9atfmZ4NgcGfZhNzyRzMJvMwm+yBI18BqqGhQZLUp08fi5N0TXZ2tu6//36lp6dbHaVL/vKXv2jcuHF66KGH1L9/f6WkpOjVV1+1OlaHUlNTVVxcrP3790uSPvnkE+3YsUNTpkyxOFnXVVVV6cSJE14/J1FRURo/frw+/PBDC5NdvYaGBjkcDkVHR1sdpUNut1uZmZl6+umnNWrUKKvjwM/502xiLpmD2WQvzCbfC7E6ALqf2+1WTk6O0tLSNHr0aKvjdOoPf/iDKioqVFZWZnWULjt06JDWrVunhQsX6plnnlFZWZmefPJJOZ1OzZkzx+p47Vq8eLEaGxuVlJSk4OBgtbW1admyZZo1a5bV0brsxIkTkqTY2Fiv7bGxsZ7v+YOzZ89q0aJFmjlzpiIjI62O06Hly5crJCRETz75pNVR4Of8aTYxl8zDbLIPZpM5KF8BKDs7W3v37tWOHTusjtKpmpoazZ8/X0VFRerZs6fVcbrM7XZr3LhxysvLkySlpKRo7969Wr9+vW2H3B//+Ee9+eab2rRpk0aNGqU9e/YoJydH8fHxts0ciFpbWzV9+nQZhqF169ZZHadD5eXlWrVqlSoqKuRwOKyOAz/nL7OJuWQuZpM9MJvMw2mHAeaJJ57Qtm3bVFJSoptvvtnqOJ0qLy9XXV2dbr/9doWEhCgkJESlpaVavXq1QkJC1NbWZnXEdg0YMEDJycle20aOHKnq6mqLEnXu6aef1uLFi/Xwww/L5XIpMzNTCxYsUH5+vtXRuiwuLk6SVFtb67W9trbW8z07uzjcjhw5oqKiItt/svj++++rrq5OgwYN8vx+HjlyRD/5yU+UmJhodTz4EX+aTcwlczGbrMdsMhdHvgKEYRj68Y9/rK1bt2r79u0aMmSI1ZG6ZOLEifr000+9tmVlZSkpKUmLFi1ScHCwRck6lpaWdtlyyfv379fgwYMtStS5M2fOKCjI+/OW4OBgud1uixJdvSFDhiguLk7FxcUaO3asJKmxsVE7d+7UD3/4Q2vDdeLicDtw4IBKSkrUt29fqyN1KjMz87LrXSZPnqzMzExlZWVZlAr+xB9nE3PJXMwmazGbzEf5ChDZ2dnatGmT3n77bUVERHjOMY6KilJYWJjF6a4sIiLisnP/w8PD1bdvX1tfE7BgwQKlpqYqLy9P06dP165du7RhwwZt2LDB6mhXlJGRoWXLlmnQoEEaNWqUPv74YxUUFGju3LlWR/PS3NysgwcPev5dVVWlPXv2qE+fPho0aJBycnL04osvatiwYRoyZIhyc3MVHx+vqVOnWhdaHeceMGCAvve976miokLbtm1TW1ub53e0T58+cjqdVsXu9P3+5iDu0aOH4uLiNGLECLOjwg/542xiLpmL2eRbzCYbsnaxRXQXSe3efvvb31od7ar5w5K+hmEYf/3rX43Ro0cboaGhRlJSkrFhwwarI3WosbHRmD9/vjFo0CCjZ8+exi233GI8++yzxrlz56yO5qWkpKTdn+U5c+YYhnFhSd/c3FwjNjbWCA0NNSZOnGjs27fP2tBGx7mrqqqu+DtaUlJi29zt8aflfGG9QJlNzCXfYTb5FrPJfhyGYbP/QhwAAAAAAhALbgAAAACACShfAAAAAGACyhcAAAAAmIDyBQAAAAAmoHwBAAAAgAkoXwAAAABgAsoXAAAAAJiA8gUAAAAAJqB8AfBITExUYWGh1TEAAPBgNiGQUL4Aizz66KOaOnWqJGnChAnKyckx7blff/11RUdHX7a9rKxMjz/+uGk5AAD2wmwCfCvE6gAAus/58+fldDqv+fExMTHdmAYAAGYTcCmOfAEWe/TRR1VaWqpVq1bJ4XDI4XDo8OHDkqS9e/dqypQp6t27t2JjY5WZman6+nrPYydMmKAnnnhCOTk56tevnyZPnixJKigokMvlUnh4uBISEvSjH/1Izc3NkqTt27crKytLDQ0Nnud7/vnnJV1+akd1dbUefPBB9e7dW5GRkZo+fbpqa2s933/++ec1duxYvfHGG0pMTFRUVJQefvhhNTU1ee7zpz/9SS6XS2FhYerbt6/S09PV0tLio3cTANAdmE2Ab1C+AIutWrVKd911lx577DEdP35cx48fV0JCgk6fPq3vfOc7SklJ0e7du/Xuu++qtrZW06dP93r8xo0b5XQ69cEHH2j9+vWSpKCgIK1evVqfffaZNm7cqH/961/66U9/KklKTU1VYWGhIiMjPc/31FNPXZbL7XbrwQcf1KlTp1RaWqqioiIdOnRIM2bM8LrfF198obfeekvbtm3Ttm3bVFpaqpdeekmSdPz4cc2cOVNz585VZWWltm/frmnTpskwDF+8lQCAbsJsAnyD0w4Bi0VFRcnpdKpXr16Ki4vzbH/55ZeVkpKivLw8z7bXXntNCQkJ2r9/v4YPHy5JGjZsmFasWOG1z0vP0U9MTNSLL76oefPm6ZVXXpHT6VRUVJQcDofX831TcXGxPv30U1VVVSkhIUGS9Lvf/U6jRo1SWVmZ7rzzTkkXBuHrr7+uiIgISVJmZqaKi4u1bNkyHT9+XF9//bWmTZumwYMHS5JcLtd1vFsAADMwmwDf4MgXYFOffPKJSkpK1Lt3b88tKSlJ0oVP9C664447LnvsP//5T02cOFEDBw5URESEMjMz9eWXX+rMmTNdfv7KykolJCR4hpskJScnKzo6WpWVlZ5tiYmJnuEmSQMGDFBdXZ0kacyYMZo4caJcLpceeughvfrqq/rqq6+6/iYAAGyF2QRcH8oXYFPNzc3KyMjQnj17vG4HDhzQPffc47lfeHi41+MOHz6sBx54QLfddpv+/Oc/q7y8XGvXrpV04aLn7tajRw+vfzscDrndbklScHCwioqK9M477yg5OVlr1qzRiBEjVFVV1e05AAC+x2wCrg/lC7ABp9OptrY2r2233367PvvsMyUmJurWW2/1un1zqF2qvLxcbrdbK1eu1Le//W0NHz5cx44d6/T5vmnkyJGqqalRTU2NZ9vnn3+u06dPKzk5ucuvzeFwKC0tTT//+c/18ccfy+l0auvWrV1+PADAGswmoPtRvgAbSExM1M6dO3X48GHV19fL7XYrOztbp06d0syZM1VWVqYvvvhC//jHP5SVldXhcLr11lvV2tqqNWvW6NChQ3rjjTc8Fztf+nzNzc0qLi5WfX19u6d8pKeny+VyadasWaqoqNCuXbs0e/Zs3XvvvRo3blyXXtfOnTuVl5en3bt3q7q6Wlu2bNHJkyc1cuTIq3uDAACmYzYB3Y/yBdjAU089peDgYCUnJysmJkbV1dWKj4/XBx98oLa2Nk2aNEkul0s5OTmKjo5WUNCVf3XHjBmjgoICLV++XKNHj9abb76p/Px8r/ukpqZq3rx5mjFjhmJiYi67KFq68Kng22+/rZtuukn33HOP0tPTdcstt2jz5s1dfl2RkZF67733dN9992n48OFasmSJVq5cqSlTpnT9zQEAWILZBHQ/h8G6mgAAAADgcxz5AgAAAAATUL4AAAAAwASULwAAAAAwAeULAAAAAExA+QIAAAAAE1C+AAAAAMAElC8AAAAAMAHlCwAAAABMQPkCAAAAABNQvgAAAADABJQvAAAAADDB/wJkT1oAfoMpCgAAAABJRU5ErkJggg==", "text/plain": [ "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import matplotlib.pyplot as plt\n", "\n", "fig, ((ax_tl, ax_tr), (ax_bl, ax_br)) = plt.subplots(2, 2,\n", " sharex=True,\n", " sharey='row',\n", " figsize=(10, 6))\n", "ax_tl.set_title('Training set')\n", "ax_tr.set_title('Development set')\n", "ax_bl.set_xlabel('Iterations')\n", "ax_br.set_xlabel('Iterations')\n", "ax_bl.set_ylabel('Accuracy')\n", "ax_tl.set_ylabel('Loss')\n", "\n", "colours = iter(plt.rcParams['axes.prop_cycle'].by_key()['color'])\n", "range_ = np.arange(1, trainer.epochs+1)\n", "ax_tl.plot(range_, trainer.train_epoch_costs, color=next(colours))\n", "ax_bl.plot(range_, trainer.train_eval_results['acc'], color=next(colours))\n", "ax_tr.plot(range_, trainer.val_costs, color=next(colours))\n", "ax_br.plot(range_, trainer.val_eval_results['acc'], color=next(colours))\n", "\n", "# print test accuracy\n", "pred = model(test_circuits)\n", "labels = torch.tensor(test_labels)\n", "\n", "print('Final test accuracy: {}'.format(acc(pred, labels)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using standard PyTorch\n", "\n", "As we have a small dataset, we can use early stopping to prevent overfitting to the training data. In this case, we evaluate the performance of the model on the validation dataset every 5 epochs, and save a checkpoint if the validation accuracy has improved. If it does not improve for 10 epochs, we end the training, and load the model with the best validation accuracy." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [], "source": [ "def accuracy(circs, labels):\n", " probs = model(circs)\n", " return (torch.argmax(probs, dim=1) ==\n", " torch.argmax(torch.tensor(labels), dim=1)).sum().item()/len(circs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Training is the same as standard PyTorch. We initialize an optimizer, pass it the model parameters, and then run a training loop in which we compute the loss, run a backwards pass to compute the gradients, and then take an optimizer step." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch: 0\n", "Train loss: 1.835998997092247\n", "Dev acc: 0.5333333333333333\n", "Epoch: 5\n", "Train loss: 0.19097438035532832\n", "Dev acc: 0.9\n", "Epoch: 10\n", "Train loss: 0.05956625810358673\n", "Dev acc: 0.9666666666666667\n" ] } ], "source": [ "model = PennyLaneModel.from_diagrams(all_circuits)\n", "model.initialise_weights()\n", "optimizer = torch.optim.Adam(model.parameters(), lr=LEARNING_RATE)\n", "\n", "best = {'acc': 0, 'epoch': 0}\n", "\n", "for i in range(EPOCHS):\n", " epoch_loss = 0\n", " for circuits, labels in train_dataset:\n", " optimizer.zero_grad()\n", " probs = model(circuits)\n", " loss = torch.nn.functional.mse_loss(probs,\n", " torch.tensor(labels))\n", " epoch_loss += loss.item()\n", " loss.backward()\n", " optimizer.step()\n", "\n", " if i % 5 == 0:\n", " dev_acc = accuracy(dev_circuits, dev_labels)\n", "\n", " print('Epoch: {}'.format(i))\n", " print('Train loss: {}'.format(epoch_loss))\n", " print('Dev acc: {}'.format(dev_acc))\n", "\n", " if dev_acc > best['acc']:\n", " best['acc'] = dev_acc\n", " best['epoch'] = i\n", " model.save('model.lt')\n", " elif i - best['epoch'] >= 10:\n", " print('Early stopping')\n", " break\n", "\n", "if best['acc'] > accuracy(dev_circuits, dev_labels):\n", " model.load('model.lt')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Evaluate test accuracy" ] }, { "cell_type": "code", "execution_count": 22, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Final test accuracy: 0.9\n" ] } ], "source": [ "print('Final test accuracy: {}'.format(accuracy(test_circuits, test_labels)))" ] }, { "cell_type": "markdown", "metadata": { "raw_mimetype": "text/markdown" }, "source": [ "## Hybrid models" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "This model determines whether a pair of diagrams are about the same or different topics.\n", "\n", "It does this by first running the pair circuits to get a probability output for each, and then concatenating them together and passing them to a simple neural network.\n", "\n", "We expect the circuits to learn to output [0, 1] or [1, 0] depending on the topic they are referring to (cooking or computing), and the neural network to learn the XOR function to determine whether the topics are the same (output 0) or different (output 1).\n", "\n", ":term:`PennyLane` allows us to train both the circuits and the NN simultaneously using PyTorch autograd." ] }, { "cell_type": "code", "execution_count": 23, "metadata": {}, "outputs": [], "source": [ "BATCH_SIZE = 50\n", "EPOCHS = 100\n", "LEARNING_RATE = 0.1\n", "SEED = 2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "As the probability outputs from our circuits are guaranteed to be positive, we transform these outputs `x` by `2 * (x - 0.5)`, giving inputs to the neural network in the range [-1, 1]. \n", "\n", "This helps us to avoid \"dying ReLUs\", which could otherwise occur if all the input weights to a given hidden neuron were negative; in this case, the overall input to the neuron would be negative, and ReLU would set the output of it to 0, leading to the gradient of all these weights being 0 for all samples, causing the neuron to never learn. \n", "\n", "(A couple of alternative approaches could also involve initialising all the neural network weights to be positive, or using `LeakyReLU` as the activation function)." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [], "source": [ "from torch import nn\n", "\n", "class XORSentenceModel(PennyLaneModel):\n", " def __init__(self, **kwargs):\n", " PennyLaneModel.__init__(self, **kwargs)\n", "\n", " self.xor_net = nn.Sequential(nn.Linear(4, 10),\n", " nn.ReLU(),\n", " nn.Linear(10, 1),\n", " nn.Sigmoid())\n", "\n", " def forward(self, diagram_pairs):\n", " first_d, second_d = zip(*diagram_pairs)\n", " evaluated_pairs = torch.cat((self.get_diagram_output(first_d),\n", " self.get_diagram_output(second_d)),\n", " dim=1)\n", " evaluated_pairs = 2 * (evaluated_pairs - 0.5)\n", " return self.xor_net(evaluated_pairs)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Make paired dataset\n", "\n", "Our model is going to determine whether a given pair of sentences are talking about different topics, so we need to construct a dataset of pairs of diagrams for the train, dev, and test data." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [], "source": [ "from itertools import combinations\n", "\n", "def make_pair_data(diagrams, labels):\n", " pair_diags = list(combinations(diagrams, 2))\n", " pair_labels = [int(x[0] == y[0]) for x, y in combinations(labels, 2)]\n", " return pair_diags, pair_labels\n", "\n", "train_pair_circuits, train_pair_labels = make_pair_data(train_circuits,\n", " train_labels)\n", "dev_pair_circuits, dev_pair_labels = make_pair_data(dev_circuits,\n", " dev_labels)\n", "test_pair_circuits, test_pair_labels = make_pair_data(test_circuits,\n", " test_labels)" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "There are lots of pairs (2415 train pairs), so we'll sample a subset to make this example train more quickly." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [], "source": [ "TRAIN_SAMPLES, DEV_SAMPLES, TEST_SAMPLES = 300, 200, 200" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "if TESTING:\n", " TRAIN_SAMPLES, DEV_SAMPLES, TEST_SAMPLES = 1, 1, 1" ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [], "source": [ "train_pair_circuits, train_pair_labels = (\n", " zip(*random.sample(list(zip(train_pair_circuits, train_pair_labels)), \n", " TRAIN_SAMPLES)))\n", "dev_pair_circuits, dev_pair_labels = (\n", " zip(*random.sample(list(zip(dev_pair_circuits, dev_pair_labels)), DEV_SAMPLES)))\n", "test_pair_circuits, test_pair_labels = (\n", " zip(*random.sample(list(zip(test_pair_circuits, test_pair_labels)), TEST_SAMPLES)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Initialise model" ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ "As :py:class:`XORSentenceModel` inherits from :py:class:`.PennyLaneModel`, we can again pass in `probabilities=True` and `normalize=True` to :py:meth:`~XORSentenceModel.from_diagrams`." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [], "source": [ "all_pair_circuits = (train_pair_circuits +\n", " dev_pair_circuits +\n", " test_pair_circuits)\n", "a, b = zip(*all_pair_circuits)\n", "\n", "model = XORSentenceModel.from_diagrams(a + b)\n", "model.initialise_weights()\n", "model = model\n", "\n", "train_pair_dataset = Dataset(train_pair_circuits,\n", " train_pair_labels,\n", " batch_size=BATCH_SIZE)\n", "\n", "optimizer = torch.optim.Adam(model.parameters(), lr=LEARNING_RATE)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train and log accuracies\n", "\n", "We train the model using pure PyTorch in the exact same way as above." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [], "source": [ "def accuracy(circs, labels):\n", " predicted = model(circs)\n", " return (torch.round(torch.flatten(predicted)) ==\n", " torch.Tensor(labels)).sum().item()/len(circs)" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Epoch: 0\n", "Train loss: 4.291878283023834\n", "Dev acc: 0.53\n", "Epoch: 5\n", "Train loss: 3.321199357509613\n", "Dev acc: 0.55\n", "Epoch: 10\n", "Train loss: 0.38510115444660187\n", "Dev acc: 0.955\n", "Epoch: 15\n", "Train loss: 0.9513051249086857\n", "Dev acc: 0.77\n", "Epoch: 20\n", "Train loss: 4.628978729248047\n", "Dev acc: 0.525\n", "Early stopping\n" ] } ], "source": [ "best = {'acc': 0, 'epoch': 0}\n", "\n", "for i in range(EPOCHS):\n", " epoch_loss = 0\n", " for circuits, labels in train_pair_dataset:\n", " optimizer.zero_grad()\n", " predicted = model(circuits)\n", " loss = torch.nn.functional.binary_cross_entropy(\n", " torch.flatten(predicted), torch.Tensor(labels))\n", " epoch_loss += loss.item()\n", " loss.backward()\n", " optimizer.step()\n", "\n", " if i % 5 == 0:\n", " dev_acc = accuracy(dev_pair_circuits, dev_pair_labels)\n", "\n", " print('Epoch: {}'.format(i))\n", " print('Train loss: {}'.format(epoch_loss))\n", " print('Dev acc: {}'.format(dev_acc))\n", "\n", " if dev_acc > best['acc']:\n", " best['acc'] = dev_acc\n", " best['epoch'] = i\n", " model.save('xor_model.lt')\n", " elif i - best['epoch'] >= 10:\n", " print('Early stopping')\n", " break\n", "\n", "if best['acc'] > accuracy(dev_pair_circuits, dev_pair_labels):\n", " model.load('xor_model.lt')\n", " model = model" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Final test accuracy: 0.945\n" ] } ], "source": [ "print('Final test accuracy: {}'.format(accuracy(test_pair_circuits,\n", " test_pair_labels)))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Analysing the internal representations of the model\n", "\n", "We hypothesised that the quantum circuits would be able to separate the representations of sentences about food and cooking, and that the classical NN would learn to XOR these representations to give the model output. Here we can look at parts of the model separately to determine whether this hypothesis was accurate.\n", "\n", "First, we can look at the output of the NN when given the 4 possible binary inputs to XOR." ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([[0.9993979 ],\n", " [0.65196735],\n", " [0.00569755],\n", " [0.1350544 ]], dtype=float32)" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "xor_labels = [[1, 0, 1, 0], [0, 1, 0, 1], [1, 0, 0, 1], [0, 1, 1, 0]]\n", "# the first two entries correspond to the same label for both sentences, the last two to different labels\n", "xor_tensors = torch.tensor(xor_labels).float()\n", "\n", "model.xor_net(xor_tensors).detach().numpy()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We can see that in the case that the labels are the same, the outputs are significantly greater than 0.5, and in the case that the labels are different, the outputs are significantly less than 0.5, and so the NN seems to have learned the XOR function.\n", "\n", "We can also look at the outputs of some of the test circuits to determine whether they have been able to seperate the two classes of sentences." ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [], "source": [ "FOOD_IDX, IT_IDX = 0, 6\n", "symbol_weight_map = dict(zip(model.symbols, model.weights))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "nbsphinx": "hidden" }, "outputs": [], "source": [ "if TESTING: \n", " FOOD_IDX, IT_IDX = 0, 0" ] }, { "cell_type": "code", "execution_count": 34, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "woman prepares tasty dinner .\n" ] }, { "data": { "text/plain": [ "array([0.42397027, 0.57602973])" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(test_data[FOOD_IDX])\n", "\n", "p_circ = test_circuits[FOOD_IDX].to_pennylane(probabilities=True)\n", "p_circ.initialise_concrete_params(symbol_weight_map)\n", "unnorm = p_circ.eval().detach().numpy()\n", "\n", "unnorm / np.sum(unnorm)" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "skillful person runs software .\n" ] }, { "data": { "text/plain": [ "array([0.95847886, 0.04152114])" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "print(test_data[IT_IDX])\n", "\n", "p_circ = test_circuits[IT_IDX].to_pennylane(probabilities=True)\n", "p_circ.initialise_concrete_params(symbol_weight_map)\n", "unnorm = p_circ.eval().detach().numpy()\n", "\n", "unnorm / np.sum(unnorm)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "From these examples, it seems that the circuits are able to strongly differentiate between the two topics, assigning approximately [0, 1] to the sentence about food, and [1, 0] to the sentence about computing." ] }, { "cell_type": "raw", "metadata": { "raw_mimetype": "text/restructuredtext" }, "source": [ ".. rubric:: See also:\n", "\n", "- `Training: Classical case <./trainer-classical.ipynb>`_\n", "- `Training: Quantum case <./trainer-quantum.ipynb>`_\n", "- `Advanced: Manual training <../manual-training.rst>`_" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 4 }