{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "5a26d281",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       ".output_png {\n",
       "    display: table-cell;\n",
       "    text-align: center;\n",
       "    vertical-align: middle;\n",
       "}\n",
       "</style>\n"
      ],
      "text/plain": [
       "<IPython.core.display.HTML object>"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from IPython.core.display import HTML\n",
    "\n",
    "HTML(\n",
    "    \"\"\"\n",
    "<style>\n",
    ".output_png {\n",
    "    display: table-cell;\n",
    "    text-align: center;\n",
    "    vertical-align: middle;\n",
    "}\n",
    "</style>\n",
    "\"\"\"\n",
    ")\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "d4ab1a0f",
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "import seaborn as sns\n",
    "\n",
    "plt.rcParams[\"figure.figsize\"] = (16, 8)\n",
    "sns.set_theme()\n",
    "sns.set_context(\"talk\")\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "92b42e41",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# A partioning-based clustering algorithm: K-means\n",
    "\n",
    "## Outline\n",
    "\n",
    "- Algorithm\n",
    "- Implementation\n",
    "- scikit-learn implementation\n",
    "- Choose K\n",
    "- Data preprocessing"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68ece225",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Toy data generation"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "a327ab4d",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 45612 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.datasets import make_blobs\n",
    "\n",
    "n_points = 250\n",
    "data, labels = make_blobs(\n",
    "    n_samples=n_points, n_features=2, centers=3, cluster_std=1, center_box=[-10, 10]\n",
    ")\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4ad865a5",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## K-means algorithm\n",
    "\n",
    "__Objective__: Find _cluster centers_ or _centroids_ representing the different clusters"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "5c1cba55",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 71936 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a3de7f5e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Formulation for M data points and K desired clusters. \n",
    "\n",
    "We call $c_k$ the centroid of cluster $k$, $a_{ik}$ the membership variable and $d(. ,.)$ the distance metric used.\n",
    "\n",
    "$$\\text{Minimize}_{A, c} \\sum_{i=1}^{M} \\sum_{k=1}^K a_{ik} \\times d(x_i, c_k)$$\n",
    "$$\\text{subject to} \\sum_{k=1}^K a_{ik} = 1, \\forall i \\in \\{1, ..., M\\}$$\n",
    "$$a_{ik} \\in \\{0, 1\\}, \\forall i \\forall k$$"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0054eea7",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Solving with the EM-algorithm\n",
    "\n",
    "- Fix K, the desired number of clusters\n",
    "- Initialize the K centroids as random points from the dataset\n",
    "- Repeat:\n",
    "    - Allocation update: Choose the new allocation as the closest centroid obtained at previous step\n",
    "    - Centroid update: Compute the centroid of the new class (mean), defined by the new allocation obtained\n",
    "at stage 2\n",
    "- Stop when the centroids are not moving anymore\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "888d9a4d",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### Demonstration K-means\n",
    "\n",
    "K=3\n",
    "\n",
    "#### Initialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "87f3cd19",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 46384 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "centroids_idx = np.random.randint(0, n_points, 3)\n",
    "centroids = data[centroids_idx]\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1])\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "833cc340",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Allocation update"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "28e72b56",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 71264 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = np.array([-1] * n_points)\n",
    "for i_d in range(len(data)):\n",
    "    dist = 1000\n",
    "    for i in range(3):\n",
    "        new_dist = np.linalg.norm(data[i_d] - centroids[i])\n",
    "        if new_dist < dist:\n",
    "            dist = new_dist\n",
    "            labels[i_d] = i\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6180bd63",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### Centroids update"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "ceb75ec6",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 71576 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    centroids[i] = np.mean(data[labels == i], axis=0)\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3203bdaa",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "#### More steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "03e38e93",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Allocation update\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 71484 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = np.array([-1] * n_points)\n",
    "for i_d in range(len(data)):\n",
    "    dist = 1000\n",
    "    for i in range(3):\n",
    "        new_dist = np.linalg.norm(data[i_d] - centroids[i])\n",
    "        if new_dist < dist:\n",
    "            dist = new_dist\n",
    "            labels[i_d] = i\n",
    "\n",
    "print(\"Allocation update\")\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "39a9d09e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "centroids update\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 71564 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    centroids[i] = np.mean(data[labels == i], axis=0)\n",
    "\n",
    "\n",
    "print(\"centroids update\")\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "d47ec2b3",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Allocation update\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 71536 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "labels = np.array([-1] * n_points)\n",
    "for i_d in range(len(data)):\n",
    "    dist = 1000\n",
    "    for i in range(3):\n",
    "        new_dist = np.linalg.norm(data[i_d] - centroids[i])\n",
    "        if new_dist < dist:\n",
    "            dist = new_dist\n",
    "            labels[i_d] = i\n",
    "\n",
    "print(\"Allocation update\")\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "32dc752f",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Centroids update\n"
     ]
    },
    {
     "data": {
      "image/png": "[encoded data, 71564 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "for i in range(3):\n",
    "    centroids[i] = np.mean(data[labels == i], axis=0)\n",
    "\n",
    "print(\"Centroids update\")\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=labels, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "707f9581",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## K-means using scikit-learn"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "8c28a2e2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.cluster import KMeans\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "77415efb",
   "metadata": {
    "slideshow": {
     "slide_type": "-"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 72968 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "K = 3\n",
    "kmeans = KMeans(n_clusters=K)\n",
    "kmeans.fit(data)\n",
    "\n",
    "clusters = kmeans.predict(data)\n",
    "centroids = kmeans.cluster_centers_\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=clusters, palette=\"tab10\")\n",
    "plt.scatter(centroids[:, 0], centroids[:, 1], c=\"r\", marker=\"x\", s=100, linewidths=3)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d40bd57b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Selecting the optimal number of clusters K\n",
    "\n",
    "- The Elbow method\n",
    "- The Silhouette method"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60fd5659",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "### The Elbow method\n",
    "\n",
    "Plot the inertia $I$ as a function of the number of clusters.\n",
    "$$I=\\sum_{i=0}^n \\min_{k \\in \\{1, ..., K\\}} (||x_i - c_k||^2)$$\n",
    "Inertia is always decreasing when more clusters are added (clusters get tighter).\\ \n",
    "However, when good groups are formed, inertia start decreasing _less_.\\\n",
    "Hence, __choose the number of clusters corresponding to the last big drop in I (the _elbow_ of the curve)__.\n",
    "\n",
    "__Exercise: Try to compute the inertia curve for the toy example. Make K vary from 1 to 10.__\\\n",
    "*Hint: The K-means in sklearn has a kmeans.inertia_ parameter.*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "826af521",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 18644 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "inertia = []\n",
    "for k in range(1, 11):\n",
    "    kmeans = KMeans(n_clusters=k)\n",
    "    kmeans.fit(data)\n",
    "    inertia.append(kmeans.inertia_)\n",
    "inertia = np.array(inertia)\n",
    "\n",
    "plt.bar(range(1, 11), inertia)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2e6ef3cd",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "## The silhouette method\n",
    "\n",
    "The silhouette was seen in the previous slides. It takes into account both the points within a cluster but also from other clusters, Hence, it should not be always decreasing when K increases.\n",
    "\n",
    "By plotting the silhouette curve for different values of K, we can simply choose the number of clusters with the highest silhouette.\n",
    "\n",
    "_Exercise: Try to compute the silhouette curve for the toy example. Make K vary from 2 to 10._"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "65ca9744",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 29764 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.metrics import silhouette_score\n",
    "\n",
    "silhouette = []\n",
    "for k in range(2, 11):\n",
    "    kmeans = KMeans(n_clusters=k)\n",
    "    clusters = kmeans.fit_predict(data)\n",
    "    silhouette.append(silhouette_score(data, clusters))\n",
    "silhouette = np.array(silhouette)\n",
    "\n",
    "plt.plot(range(2, 11), silhouette)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e350864",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Data preprocessing\n",
    "\n",
    "**Example:** We want to identify groups of people that are collected from a population.\\\n",
    "The data measured are the **weight (in kg)** and the **age (in days)** of the individuals.\\\n",
    "The objective is to identify the groups corresponding to men and women.\\\n",
    "The average weight of women is 60 kg and for men it is 80 kg. The variance is about 5 kg for both groups.\\\n",
    "The average age of both groups is 50 years and the variance is 10 years."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "06543a11",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 116664 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "means = [np.array([60, 50 * 365]), np.array([80, 50 * 365])]\n",
    "covariance = np.array([[5, 0], [0, 10 * 365]])\n",
    "n_points = 500\n",
    "\n",
    "data = [\n",
    "    np.random.multivariate_normal(mean=means[i], cov=covariance, size=n_points)\n",
    "    for i in range(2)\n",
    "]\n",
    "data = np.r_[data[0], data[1]]\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "ad0a54b6",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 208388 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "kmeans = KMeans(n_clusters=2)\n",
    "clusters = kmeans.fit_predict(data)\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=clusters, palette=\"tab10\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "a8eee5b3",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 12824 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 15376 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(data[:, 0])\n",
    "plt.show()\n",
    "plt.hist(data[:, 1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "a6e48898",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 16460 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.boxplot(data)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "9063c55e",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 103856 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from sklearn.preprocessing import StandardScaler\n",
    "\n",
    "ss = StandardScaler()\n",
    "data_scaled = ss.fit_transform(data)\n",
    "\n",
    "sns.scatterplot(x=data_scaled[:, 0], y=data_scaled[:, 1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "9466769c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 13184 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "image/png": "[encoded data, 14044 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.hist(data_scaled[:, 0])\n",
    "plt.show()\n",
    "plt.hist(data_scaled[:, 1])\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "ecfc52e0",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 11044 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "plt.boxplot(data_scaled)\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "86bdc3d8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "[encoded data, 211052 characters]",
      "text/plain": [
       "<Figure size 1152x576 with 1 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "kmeans = KMeans(n_clusters=2)\n",
    "clusters = kmeans.fit_predict(data_scaled)\n",
    "\n",
    "sns.scatterplot(x=data[:, 0], y=data[:, 1], hue=clusters, palette=\"tab10\")\n",
    "plt.show()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f9e257a6",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
