The open-model world

What exactly is an open language model? Let's try to understand. In a simple way. Here is the key idea.

An AI model is really just a giant file of numbers. Those numbers are called the weights. Everything the model knows lives inside them. So a company that builds one has a choice.

Keep that file locked on their servers. Or publish it for anyone to download. A closed model stays hidden. You never touch the file.

You send it text, and it sends text back. That is an API. Think of GPT four, or Claude. An open model is the opposite.

The company puts the weights file online. You download it, and the model sits on your own machine. Llama and Mistral work this way. Holding the actual file changes everything.

You can run it with no company in the loop, look inside it, and reshape it for your own needs. How big is this file? Often many gigabytes of numbers. Downloading an open model is not copying an answer.

It is copying the whole brain. So the real test is simple. Can you get the weights? If the numbers are yours to download, the model is open.

If they stay behind an API, it is closed. And a deeper point. A downloaded model is yours to keep. A closed one can change, or vanish, the day the company decides.

So, an open model is one whose weights you can download and run yourself. The file is the model, and that file is in your hands. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a parameter? Let's try to understand. In a simple way. Picture a model as a giant wall of little dials.

Billions of them. Each dial is just a number the model can adjust. And each of those numbers is one parameter. So a parameter is a single tunable number inside the network.

During training, the model turns each one a tiny bit, until its guesses get good. Engineers call it a weight. Now, how do you say how big a model is? You count these numbers.

When someone says a model has seven billion parameters, they mean seven billion of those dials. Notice what size is not. It is not the file on disk. It is not how much text the model read.

Size means one thing. The count of parameters. Why count them? Because that number hints at capacity.

More parameters means more room to store patterns, facts, and skills picked up from the data. And this count is baked in. It comes straight from the design, the number of layers and how wide each one is. It is fixed before training even starts.

That is why you see names like the seven B model, or the seventy B one. The B just stands for billion parameters. It gives everyone one simple ruler to line models up. Bigger is not always better, but the parameter count is the first thing people compare.

So, model size is really a head count of tunable numbers. Count the parameters, and you have named the model's scale. Quick check now. One question is coming up.

Let's see if it clicked.

How does a model's memory work? Let's try to understand. In a simple way. Here is the strange part.

A model does not truly remember you between messages. Each reply, it re-reads everything in front of it. So its only memory is the text sitting in front of it right now. That space has a name.

The context window. Picture a whiteboard of fixed size. Everything the model can think about must fit on that board. And it is measured in tokens.

One board, shared by everything. The hidden instructions, the whole conversation so far, your question, and the answer it is about to write. What happens when the board fills up? The oldest text slides off the edge.

The model does not choose. The earliest tokens simply fall out. That is why a long chat can seem to forget how it began. The detail did not fade.

It dropped off the window's edge. Careful. This is not the knowledge baked in during training. That is permanent.

The window is temporary, wiped clean for every new chat. Working memory, not long-term. So why cap it? The model weighs every token against every other one.

Twice the text, four times the work. Capped by design, to stay fast. Newer models push the board wider. A whole book, a full codebase, held at once.

But a bigger board costs more to fill. So, the context window is the model's working memory. Measured in tokens, shared by everything. Fill the board, and the oldest words fall off.

Quick check now. One question is coming up. Let's see if it clicked.

How does an AI leaderboard work? Let's try to understand. In a simple way. Every week a new model claims to be the best.

Claims are cheap. To compare them fairly, you need one shared test every model must sit. That shared test is called a benchmark. It is a fixed set of questions with known answers, the same exam handed to every model.

Each model answers, and you count how many it got right. That count becomes a single score, usually a percent. One model, one number. Sort those scores from high to low, and you get a leaderboard.

Just a ranked table. The best score sits on top, the rest below in order. But there is no single exam. One benchmark tests math, another tests coding, another tests general knowledge.

A model can top one board and sink on another. Some leaderboards have no answer key. In an arena, two anonymous models answer the same prompt, and a person picks the better reply. Those votes become a rating.

Now the big catch. If the test questions leaked into a model's training data, it can memorize the answers. The score jumps, but nothing was learned. That is contamination.

So treat a leaderboard as a hint, not a verdict. Pick the benchmark that matches what you need, and stay suspicious when a score looks too good to be true. So, a leaderboard ranks models by their score on a shared test. Same exam, one number each, sorted top to bottom.

Just remember what the test measured. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a model family? Let's try to understand. In a simple way. Here is the surprise.

Under the hood, Llama, Mistral, and the rest are nearly the same machine. All of them are transformers. So what really makes one different from another? A family is all the models one lab builds from one recipe.

Llama comes from Meta, Mistral from a startup of the same name, Gemma from Google. Each name is really a lab's bloodline. Inside a family, that recipe is poured into different sizes. A small seven billion, a larger seventy billion.

Each size ships as a raw base model, and one tuned to chat. Siblings. So where do the real differences come from? First, the data.

Same design, but each lab feeds it a different pile of text, cleaned its own way. That shapes how the model talks. Second, small architecture tweaks. Mistral added sliding window attention to stretch its memory.

Others use a mixture of experts, waking only part of the model per word. Little changes chasing speed. Third, the tokenizer. Each family chops text into pieces with its own menu.

A different menu can handle code, or other languages, far more smoothly. Fourth, the finishing. After the base learns, each lab tunes it its own way, with instructions and human feedback. That polish sets the model's manners.

Put it together, and a family is a bundle. One lab's design, its data, its tuning, under a single name. That package is what separates Llama from Mistral. So, the peers all share one transformer skeleton.

What sets a family apart is its lab, its data, its tweaks, and its tuning. Same bones, different upbringing. Quick check now. One question is coming up.

Let's see if it clicked.

How does matching a model to your job work? Let's try to understand. In a simple way. Start with a trap.

There is no single best model. The one sitting at the top of a list may be wrong for you. Best always means best fit for your job. So begin with the job, not the model.

What are you actually asking it to do? Write code. Summarize reports. Chat with users.

Translate. Name the task first. Now match a specialist to it. Some families are tuned for code, some for chat, some for math, some for many languages.

A model trained on your kind of work usually wins. Next, size it against your machine. A bigger model is smarter, but slower and hungrier. The right size is the smallest one that still clears your quality bar.

Then weigh speed and cost. A live chatbot needs a fast reply. A nightly batch can wait. Match the model's speed and price to how the job is used.

Watch the variant too. A base model just predicts text. An instruct or chat version was tuned to follow directions. For most jobs, you want the instruct one.

Then the real decider. Take three or four of your own real tasks and run the finalists on them. A leaderboard is a hint. Your own examples give the answer.

So, there is no best model, only the best fit. Start from the job, pick a specialist, size it to your machine, budget its speed, then test it on your own tasks. Quick check now. One question is coming up.

Let's see if it clicked.

How does running an AI on your own laptop work? Let's try to understand. In a simple way. Start with what an open model really is.

Just a file of numbers, its weights. You download it once, and it sits on your disk. Once it is on your disk, the whole model runs on your machine. No server, no cloud, no internet.

That buys three things. Your words never leave the laptop, so it is private. It works offline, with no internet. And each run is free, with no bill.

But a file alone does nothing. A small program called a runtime loads the weights and runs the same next word loop. Ollama is one popular example. Where does the math run?

Your CPU can do it. A GPU, a graphics card, does it far faster. Even a laptop can manage a smaller model. Here is the one real catch.

The whole model must fit inside your memory at once. A model too big for your laptop will not load. So you pick a laptop sized model. A smaller sibling, a few billion parameters, fits comfortably.

Clever tricks can shrink one further, but that is the next lesson. What do you give up? A laptop model is smaller, so a bit less sharp than the giants in the cloud. But it is fully yours.

So, an open model is a file you download and run yourself. A runtime does the work, the whole model must fit in memory, and you get privacy and no bills. Quick check now. One question is coming up.

Let's see if it clicked.

How does shrinking a model to fit your memory work? Let's try to understand. In a simple way. Start with a single weight.

It is just a number, and a number has to be stored using some bits. By default, each weight uses sixteen bits. Now multiply. A seven billion weight model, at two bytes each, needs about fourteen gigabytes of memory.

That is too big for most laptops to load. So here is the trick. Store each weight using fewer bits. This is called quantization, and it shrinks the whole file at once.

How does that work? Take the range of weight values and mark off a small set of levels. Then round every weight to its nearest level. Four bits gives you just sixteen levels, so each weight fits in half a byte.

That same model drops from fourteen gigabytes down to about three and a half. Notice what did not change. The model still has all seven billion weights. You did not remove any.

Each one is simply stored with less detail. The cost is a little rounding error. Each weight is now slightly off, so the model is a touch less sharp. At four bits, most people barely notice.

This is why you see labels like Q four or Q eight. The number is the bits per weight. Lower means smaller and lighter, but a bit rougher. So, quantization stores each weight with fewer bits.

Same number of weights, less detail each. The model shrinks to fit your memory, for a tiny drop in sharpness. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is video memory? Let's try to understand. In a simple way. Video memory, or VRAM, is the memory that lives on your graphics card.

It is separate from your normal RAM, and much faster. To run fast, the model must live in here. So how much do you need? Start with the weights, the model's numbers.

A seven billion model at four bits is about four gigabytes. That is the floor. But the weights are not the whole bill. While the model runs, it needs extra room to think, a working scratchpad.

That room lives in VRAM too. The biggest extra is the KV cache. As your conversation grows, the model keeps a memory of every token so far. Longer chats mean a bigger cache.

So the real bill is the weights, plus the cache, plus a little overhead. Add them up, and that total must fit inside your VRAM. A quick rule of thumb. Take the weight size, and add twenty to thirty percent for a short chat.

Very long context can need far more. What if it does not fit? The extra spills over into your slow system RAM. It still runs, but it crawls.

This is called offloading. So two people running the same model can need very different memory. A short question is cheap. A long document in the context is not.

So, video memory holds the model while it runs. Budget the weights, plus the cache that grows with context, plus a little headroom. Fit it all, and it flies. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is an inference engine? Let's try to understand. In a simple way. An open model is just a file of weights, and a file cannot run itself.

The program that runs it, feeding tokens in and words out, is the inference engine. Here is the key. The engine does not touch the weights or the answers. Same model, same replies.

It only changes how fast they come out. So where does the speed hide? A graphics card can do enormous math at once. Feed it one token at a time, and most of the chip sits idle.

The first trick is batching. Rather than serve people one by one, the engine pushes many requests through together, and refills the batch as each one finishes. The card stays busy. The second trick skips repeated work.

Normally each new word makes the model re-read the whole chat. The engine caches that work and reuses it, keeping every token cheap. The third trick is leaner math. The same calculations, written as tighter code, make fewer trips to memory.

These routines are called kernels, and faster kernels mean more tokens per second. Put it together and the payoff is real. More tokens per second for you, and many more users served on one card. The model is identical.

The engine just wastes less. So, an inference engine is the software that runs the weights, without changing a single answer. It keeps the card busy, reuses past work, and runs leaner code. Same model, far faster.

Quick check now. One question is coming up. Let's see if it clicked.

What exactly is a cloud API? Let's try to understand. In a simple way. You have an open model, a big file of weights.

To actually use it, there are two very different routes. Route one is a cloud API. You download nothing. You send your text over the internet to a company's servers, their cards run the model, and the answer returns.

Route two is your own machine. You download the weights once, and run them right on your computer. Nothing you type ever leaves your desk. Now the tradeoffs.

Cost. The API charges a small fee per token, every time, forever. Running locally is free per use, but you buy the hardware upfront. Next, privacy.

With the API, your words travel to someone else's computer, so you must trust them. Local keeps every word on your machine, and even works offline. Then power and setup. The cloud rents you giant cards you could never buy, and the biggest models, with nothing to install.

Your machine is capped by its own memory. And control. The API can raise its price, rate limit you, or retire the model. Your local copy is yours, and keeps working regardless.

Here is the key. It is the same open weights either way. Only the delivery differs. Many teams even mix both, prototype on the API, then move private work local.

So, one open model, two routes. The cloud API is instant and scalable, but you rent and trust. Your own machine is private and free to run, if it fits. Quick check now.

One question is coming up. Let's see if it clicked.

What exactly is the difference between open weights and open source? Let's try to understand. In a simple way. Lots of models get called open.

But open weights and open source are not the same. One gives you the finished product. The other, the whole recipe. Open weights means you get the trained file itself, the final numbers.

You download it, run it, even fine tune it. But how was that file made? It took three things. The training data, the training code, and the recipe of choices.

Open weights gives you none of them. Picture a baked cake. Open weights is the finished cake, ready to eat. Open source is the full recipe, so you could bake it yourself.

In normal software, open source means the code is public. Anyone can read it, rebuild it, and change it from scratch. For a model, the source is the data plus training code. Why does this gap matter?

Without the data, you cannot see what it learned from, cannot check it for bias, cannot rebuild it. You inherit only the result. So most famous open models, like Llama and Mistral, are open weights, not open source. They ship the file, but keep the data private.

A few projects go all the way, releasing the weights, data, and code together. That is truly open source. It is rare, because the data is the hardest to share. So, open weights gives you the model to use.

Open source gives you everything to rebuild it. Same word, two very different promises. Quick check now. One question is coming up.

Let's see if it clicked.

What exactly is a usage restriction on an open model? Let's try to understand. In a simple way. So the weights are open, you can download them.

But open means available, not anything goes. Attached to the file is a set of rules you agree to. Those rules live in a license. When you download the model, you accept it.

It is a real legal contract, not a friendly suggestion. First, an acceptable use policy. It lists what you may never do with the model. Things like fraud, weapons, or clear harm.

Owning the file does not lift that ban. Some models are gated. Before you can download, you must accept the terms, and sometimes request access and wait. A gate at the door, not a locked wall.

A few licenses add a cap on scale. One famous clause says, if your app grows past seven hundred million users a month, you need a separate deal. That only touches giants. Another common rule.

You may not use this model's answers to train a rival model. So no quietly copying it into your own competitor. Many licenses also ask for credit. You keep the license text with any copy you share, and label your product, built with that model.

And the rules vary a lot. Some models use a wide open license like Apache. Others use a custom community license, with all these strings attached. So, open weights hand you the file, but the license sets the rules.

Read it before you build, because open does not mean unrestricted. Quick check now. One question is coming up. Let's see if it clicked.

What exactly is commercial use of an open model? Let's try to understand. In a simple way. You want to build a product on an open model and sell it.

Downloading the file is easy. Selling what you make is a legal question. It turns on two words. Commercial use.

Does the license grant it? That answer splits open models into two camps. The first camp is permissive. Licenses like Apache two point zero.

They say, do almost anything, including sell it. Just keep the notice. Safest for business. The second camp is non commercial.

Research and personal use only. You may download and tinker, but never put it in a paid product. Here is the trap. Open does not mean commercial.

A model can be fully downloadable and still be off limits for money. Licenses also inherit. A fine tuned model can only pass on the freedom of its base. Research only base, research only child.

A subtler trap. If a model was trained on another model's answers, it can inherit those terms. Tainted by the giant it copied. And even when selling is allowed, the model ships as is.

No warranty. If it does harm, the risk is yours, not the lab. So before you build, open the license file. Search for non commercial, or research only.

Check what it was trained on. In doubt, ask a lawyer. So, can you use it for business? Find the license, confirm commercial use, and trace where it came from.

Read the fine print first. Quick check now. One question is coming up. Let's see if it clicked.

What exactly are the hidden risks in a model's training data? Let's try to understand. In a simple way. A model learns all it knows from a huge pile of training data.

Once training ends, that pile is gone, baked into the numbers. But it still shapes every answer. Think of a model as a mirror of its data. Whatever went in comes back out.

The good and the bad, both get absorbed. Risk one is bias. If the data leaned one way, the model leans that way too. It repeats stereotypes, or stays silent on groups the data barely mentioned.

Risk two is harmful content. The open internet is full of hate, lies, and junk. Some of that gets learned, and can resurface in the model's answers. Risk three is memorization.

A model can memorize rare lines word for word. Private details, or copyrighted text, can leak back out. Risk four is data poisoning. Anyone can post text on the web.

A bad actor can plant traps, so the model learns a false fact or a hidden trigger. Risk five is a frozen clock. The data stops at a cutoff date. The model knows nothing newer, and can keep repeating old, outdated mistakes.

And here is the hard part. You cannot see any of this. The weights hide the data, so every risk comes along, silent and unseen. So the gist.

A model inherits the flaws of its data. Bias, harmful content, leaks, poison, and a frozen clock. All baked in, all invisible. Quick check now.

One question is coming up. Let's see if it clicked.