That feeling of "something missing" sent me looking backwards, not forwards. If transformers felt like magic, I wanted to know where the trick actually began.

The trail went back further than I expected — through decades of ideas, one built on top of another, until it landed on something surprisingly small: a single neuron. Not a transformer, not even a proper network. Just one unit, one formula, sitting at the very bottom of everything I'd been studying.

That's where I decided to start. No libraries, no shortcuts — just pure Python, and Claude as my tutor: asking me questions, making me derive the math myself, flagging it when something was wrong — while I wrote every line of code and worked through every formula on my own.

A neuron, at its simplest, is just a small decision-maker: it takes some inputs, weighs how much each one matters, and produces one answer. That's it. No magic, no mystery — just a formula.

But knowing what it is didn't answer the question I actually cared about: why does something like this need to exist at all?

We can't use pure if-else logic everywhere. Yes, rule-based logic can handle a lot — but sometimes what's needed isn't a fixed rule; it's a decision.

Take the rain prediction—two inputs: temperature and humidity. On the surface, this feels easy — just write a rule. If the temperature is above some threshold and the humidity is above another threshold, predict rain.

But think again. Where do those numbers come from? I'd have to guess them myself. And once, I guess, what's my actual success rate? I'd have no real way to know if my rule was any good — or if slightly different numbers would have worked better.

The answer isn't a smarter rule. It's not writing a better if-else. It's replacing the guesswork entirely — building something that finds its own numbers, from real data, instead of me handing it mine.

That's what I set out to build.

So I built one: a single neuron, in pure Python, that takes temperature and humidity and predicts whether it will rain.

Two stages

A neuron like this really has two separate stages: after training and training itself.

Everything I'm about to walk through first is the after-training stage. It assumes w1, w2, and b already exist, already have specific values, and just uses them to answer a question.

How it predicts

The first thing I had to build was a way to connect two inputs — temperature and humidity — to one output: will it rain?

I started with a weighted sum:

$$z = w_1x_1 + w_2x_2 + b$$

x₁ is temperature, x₂ is humidity. w₁ and w₂ are numbers that decide how much each input matters, and b is a baseline lean that shifts the result regardless of the inputs.

Here's what that looks like:

Graph 1: The weighted sum is a flat plane — it can tilt and shift, but never bend. (Desmos 3D, with my trained weights)

The xy-plane here is every pair of temperature and humidity. For every pair, z gives us one point on that surface — and as you can see, the plane extends without limit. z can be any number at all — which is exactly the problem the next step has to solve.

z on its own isn't a prediction. It's just a raw number. In machine learning, this raw number has its own name: a logit, the log-odds of rain, before it becomes an actual probability.

To turn that raw number into something meaningful, I passed it through sigmoid:

$$\hat{y} = \sigma(z) = \frac{1}{1 + e^{-z}}$$

Here's what that does to the same idea:

Graph 2: Sigmoid bends the same surface into an S, trapped between 0 and 1. This is the prediction.

The flat, unlimited surface becomes bounded — squeezed between 0 and 1. This is the actual prediction: a number telling me how confident the model is that it will rain.

I chose a weighted sum instead of something more complex on purpose. It's the simplest way to combine two inputs while still letting each one carry its own importance. It keeps the math behind training clean, and it scales easily — adding a third input later, like wind speed, would just mean one more term.

And despite the name, this technique — logistic regression — isn't really regression in the usual sense. It's classification: the final answer is one of two categories, rain or no rain. "Regression" only describes what happens on the way there — fitting the logit with a weighted sum, before sigmoid turns it into a decision.

That's the whole prediction mechanism — two formulas, two graphs, one confidence score at the end.

But here's the big thing: that S-curve has to be shaped to match real rain pairs and real no-rain pairs. Every point where it actually rained, every point where it didn't — the curve has to bend and sit exactly right over all of them.

And the only way to reshape that curve is by changing w1, w2, and b. Nothing else can move it.

That's what training actually does.

How it trains

Training isn't one calculation. It's a loop — the same few steps, repeated hundreds of times, each pass making the neuron a little less wrong than the last.

For training, we need a real dataset — real examples of what actually happened. For version one, I used four data points I could check by hand, so I could verify every step against my own arithmetic.

To know if the neuron is right or wrong, we need a way to measure it. That's what a loss function does.

For version one, I used MSE — mean squared error:

$$L = \tfrac{1}{2}(\hat{y} - y)^2$$

Later, I learned there's a better option for this kind of problem — BCE, binary cross-entropy:

$$L = -\big[,y\log \hat{y} + (1-y)\log(1-\hat{y}),\big]$$

But before either one, there's a real question worth asking: what actually makes a loss function good?

Three things. It should be zero when the prediction is perfect, and grow bigger the more wrong it gets. It has to be differentiable — smooth enough for calculus to work on it. And its slope should push hardest exactly where correction matters most, not fade out right when the neuron needs the biggest nudge. (MSE, I'd later discover, quietly fails that third test when paired with sigmoid — a story for the next post.)

Once we have loss, the only things left to control are w1, w2, and b. Those are the only knobs training is allowed to touch. So loss becomes a function of them, through the prediction: loss(sigmoid(z)).

To know which way to turn those knobs, we need the gradient. A gradient always points in the direction a function increases fastest — so to make loss smaller, we do the opposite of whatever it points toward:

$$w ;\leftarrow; w - \eta,\frac{\partial L}{\partial w}$$

Here η is the learning rate — how big a step we take each time.

That's the whole idea training runs on: find the gradient, step the other way, repeat.

So what actually happens when training runs?

For every row in the data, the neuron makes a guess, checks how wrong that guess was, finds the gradient, and nudges w1, w2, and b a little in the direction that lowers the loss. Then it moves to the next row, and the next, until it's been through the whole dataset once. That's one epoch. Then it starts over, and does it again — hundreds of times.

Each pass, the weights shift a little less than before. Eventually, they settle and stop moving in any meaningful way. That's the model finished learning.

And here's the part that matters most: once that's done, the dataset itself isn't needed anymore. To predict rain on any new day, all the neuron ever needs again are those three numbers — w1, w2, and b. Nothing else. Not the four rows I trained on, not any of the math that found them. Just three numbers, doing all the work.

Then I gave it real data

The toy version worked, so I felt ready for something real: a Kaggle weather dataset — 2,500 days of temperature, humidity, and whether it rained.

It broke. Not with an error message — worse. It looked fine. 87% accuracy. But when I checked closer, it had caught only 79 of the 314 rainy days. It was mostly just saying "no rain" — and since 87% of days were dry, saying no made it look smart while learning almost nothing.

Three things were quietly wrong. My loss function's push faded exactly where the model was most wrong — the worst place to stop pushing. The rare rainy days were being ignored because ignoring them was the cheapest way to lower the loss. And accuracy, the number I trusted, was hiding all of it.

Fixing those three things became version two. It isn't code I typed alone — it's what my first version became after each failure was put in front of me, and I had to understand why before I was allowed to fix it. That version catches 281 of the 314 rainy days.

The finished model over real data: blue dots are 2,186 dry days, red dots are 314 rainy days, and the surface is the neuron's P(rain) — with the decision boundary drawn where it crosses 0.5. But each of those three failures is a story of its own — and they're coming in the next post.

Where this leaves me

That's a single neuron, start to finish. Two formulas, one loss function, one loop that repeats until the weights stop moving.

It's not magic. It never was. It's arithmetic, repeated patiently until it's right.

But I also know this is just one neuron — the smallest piece, not the whole picture. Real networks stack thousands of these together, layer after layer, each one this simple on its own. That's where I'm headed next: not a fancier neuron, but many simple ones, working together.

For now, though, I finally have what I came looking for — not a diagram of how AI learns, but something I built myself, wrong turns and all, until it actually worked.