neural-network/nnetwork.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "9b3f1635",
   "metadata": {},
   "source": [
    "# Neural Network"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "478651c8",
   "metadata": {},
   "source": [
    "## What is a *Neuron* (artifical)\n",
    "\n",
    "> **disclaimer**: I'm not an Neurologist. This is only based on online research.\n",
    "\n",
    "An **artificial neuron** works *similary* to a **biological neron** in the way it process information.\n",
    "\n",
    "In a brain, like yours, a neuron receive **electrical signals** from others, process them and sends an output signal.\n",
    "\n",
    "An **artifical neuron** countrary to biological ones:\n",
    "1. **Takes inputs** (usually numbers between 0 and 1).\n",
    "2. **Multiplies** each by a corresponding **weight** (importance of that input).\n",
    "3. **Adds a bias**, which shifts the result up or down.\n",
    "4. **Applies an activation function**, which normalizes or squashes the output (commonly: **sigmoid**, **ReLU**, etc.).\n",
    "5. **Returns the final output**, often a value between 0 and 1. \n",
    "\n",
    "---\n",
    "\n",
    "## Vocabulary / key components\n",
    "1. **inputs**: inputs are usually a unique list of numbers, they are simply values sent to a neuron, which then process them.\n",
    "2. **weights**: weights are also a list of numbers that has the same size of inputs. The weight determines how important de the number of the input is. If it's high, the input matters. Else, if the weight is low, the number matters less.\n",
    "3. **bias**: the bias are constant that are added after all the inputs are multiplied by the weight. it helps shift the resultat up or down.\n",
    "\n",
    "---\n",
    "\n",
    "## 🔑 Vocabulary / Key Components\n",
    "\n",
    "| Term     | Meaning |\n",
    "|----------|---------|\n",
    "| **inputs**  | List of input values (e.g., 8-bit binary numbers like `01001010`) |\n",
    "| **weights** | Values associated with each input, controlling how much influence each input has |\n",
    "| **bias**    | A constant added to the weighted sum to adjust the output |\n",
    "| **activation function** | A function like `sigmoid` that transforms the output into a bounded range |\n",
    "\n",
    "---\n",
    "\n",
    "## ⚙️ Minimal Neuron Implementation\n",
    "\n",
    "### Step 1 – Initialization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7d9d6072",
   "metadata": {},
   "outputs": [],
   "source": [
    "import random\n",
    "\n",
    "# neuron class\n",
    "class Neuron:\n",
    "    \"\"\"\n",
    "    z                   : linear combination of inputs and weights plus bias (pre-activation)\n",
    "    y                   : output of the activation function (sigmoid(z))\n",
    "    w                   : list of weights, one for each input\n",
    "    \"\"\"\n",
    "    def __init__(self, isize):\n",
    "        # number of inputs to this neuron\n",
    "        self.isize = isize\n",
    "        # importance to each input\n",
    "        self.weight = [random.uniform(-1, 1) for _ in range(self.isize)]\n",
    "        # importance of the neuron\n",
    "        self.bias = random.uniform(-1, 1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6dd28c51",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "id": "1aff9ee6",
   "metadata": {},
   "source": [
    "# 2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7ca39a42",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import random\n",
    "\n",
    "# Neuron 2\n",
    "class Neuron:\n",
    "    def __init__(self, input_size: int) -> None:\n",
    "        self.input_size = input_size\n",
    "        self.weight = [random.uniform(0, 1) for _ in range(self.input_size)]\n",
    "        self.bias = random.uniform(0, 1)\n",
    "\n",
    "    def sigmoid(x: float) -> float:\n",
    "        return 1/(1 + math.exp(-x))\n",
    "    \n",
    "    def forward(self, inputs: list) -> float:\n",
    "        assert len(inputs) == self.input_size, \"error: misnumber inputs number\"\n",
    "        total = sum(self.weight[i] * inputs[i] for i in range(self.isize)) + self.bias\n",
    "        return self.sigmoid(total)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6709c5c7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Neuron output : 0.9001175686881125\n"
     ]
    }
   ],
   "source": [
    "# 8 for 8 bits (1 Byte)\n",
    "nbits: int = 8\n",
    "neuron = Neuron(nbits)\n",
    "inputs: list = [1, 0, 1, 0, 0, 1, 1, 0] \n",
    "\n",
    "output = neuron.forward(inputs)\n",
    "print(\"Neuron output :\", output)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aa57ae8e",
   "metadata": {},
   "source": [
    "# 3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f6de25ea",
   "metadata": {},
   "outputs": [],
   "source": [
    "import math\n",
    "import random\n",
    "\n",
    "# Neuron 3\n",
    "class Neuron:\n",
    "    def __init__(self, isize: int) -> None:\n",
    "        self.isize = isize\n",
    "        self.weight = [random.uniform(0, 1) for _ in range(self.isize)]\n",
    "        self.bias = random.uniform(0, 1)\n",
    "\n",
    "    def forward(self, inputs: list) -> float:\n",
    "        assert len(inputs) == self.isize, \"error: incorrect inputs number\"\n",
    "        total = sum(self.weight[i] * inputs[i] for i in range(self.isize)) + self.bias\n",
    "        return self.sigmoid(total)\n",
    "    \n",
    "    def sigmoid(x: float) -> float:\n",
    "        return 1/(1 + math.exp(-x))\n",
    "\n",
    "    # target needs to be between 0 and 1\n",
    "    def train(self, inputs: list, target: float, learning_rate: float = 0.1):\n",
    "        z = sum(self.weight[i] * inputs[i] for i in range(self.isize)) + self.bias\n",
    "        output = self.sigmoid(z)\n",
    "\n",
    "        error = output - target\n",
    "        d_sigmoid = output * (1 - output)\n",
    "        dz = error * d_sigmoid\n",
    "\n",
    "        for i in range(self.isize):\n",
    "            self.weight[i] -= learning_rate * dz * inputs[i]\n",
    "\n",
    "        self.bias -= learning_rate * dz\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".venv",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}