How Intelligent Systems Are Built
Before Machine Learning won, the field spent forty years trying to build intelligence by hand — first as search, then as logic and rules. To understand why learning-from-data took over, you have to feel exactly where those two approaches broke.
In Lesson 1 you learned there are four broad approaches to AI (Search, Symbolic logic, Statistical learning, Bio-inspired). This lesson zooms into the first two — the "build it by hand" approaches — because understanding what Machine Learning replaced is the fastest way to understand what Machine Learning is. Every design choice in the rest of this course is a reaction to a wall that search or logic ran into. See the wall, and the whole ML paradigm stops looking arbitrary and starts looking inevitable.
What this lesson delivers
- How the search & planning approach frames a problem, and the exact mechanism by which it explodes.
- How symbolic AI (knowledge + logic) represents and reasons — and why encoding the world by hand fails.
- The two named walls — combinatorial explosion and the knowledge-acquisition bottleneck — that ended two AI eras.
- Why "learn the rules from data" is the natural escape from both walls, not just one.
- A durable mental checklist for deciding, on any new problem, whether to write rules or learn them.
1 · Two questions every builder must answer
Strip away the jargon and building an intelligent system comes down to answering two questions:
The four approaches are really just four different pairs of answers to Representation and Inference. This lesson examines the first two pairs. Keep the two questions in mind — we'll use them again to see precisely where each approach cracks.
2 · The search & planning approach
The oldest idea in AI is beautifully simple: many problems can be framed as finding a path from where you are to where you want to be. If you can describe the situations you might be in and the moves that connect them, then "being intelligent" reduces to exploring that space of possibilities until you find a path to the goal.
Formulate a problem as four things: a set of states (possible configurations of the world), a start state, a set of actions that move you from one state to another, and a goal test that recognizes when you've arrived. Solving the problem = finding a sequence of actions from start to a goal state.
This single template covers a startling range of tasks. A GPS route: states are intersections, actions are roads, the goal is your destination. A puzzle like the 8-puzzle: states are tile arrangements, actions slide a tile. A chess game: states are board positions, actions are legal moves. In each case, the "intelligence" is not knowledge about the domain — it's the systematic exploration.
Representation: the state space (a graph of situations connected by actions). Inference: a search algorithm that walks that graph looking for a goal. That's the whole approach — and notice it needs almost no domain knowledge, just the rules of the game.
Blind search vs informed search
The earliest algorithms searched blindly — breadth-first, depth-first — trying options in a fixed order with no sense of which looked promising. They are correct but hopelessly slow on real problems. The key improvement was informed search: use a heuristic, a cheap estimate of "how close is this state to the goal?", to explore promising directions first.
A heuristic h(state) estimates the remaining cost to the goal. The famous A* algorithm expands states in order of f = g + h, where g is the cost already paid to reach the state and h is the estimated cost still to go. A good h is the difference between searching a whole city and driving almost straight to the destination.
Hold onto the word evaluation: when a chess engine scores a position, or later when a model outputs a prediction, they are descendants of this same idea — a cheap number that guides a decision.
Where search shines — and where it stops
Search is still the right tool whenever the rules are known, exact, and the state space is navigable: route finding, logistics scheduling, theorem proving, and game engines all lean on it today. It is precise, it can be proven correct, and it needs no training data.
The state space grows exponentially with the size of the problem. Chess has roughly 1040 reachable positions; Go has more than 10170. No heuristic makes brute-force exploration of such spaces tractable. And crucially, search assumes the world is clean and known: discrete states, deterministic actions, a crisp goal test. The real world — a blurry photo, an ambiguous sentence, a noisy sensor — offers none of that. Search has no idea what to do when it can't even tell which state it's in.
3 · The symbolic (knowledge & logic) approach
If search's weakness is that it has no knowledge, the obvious fix is to give the machine knowledge. This is the second grand approach, often called symbolic AI or GOFAI ("Good Old-Fashioned AI"): represent facts about the world as symbols and logical statements, then derive new conclusions by inference.
Encode what you know as explicit statements — facts and if–then rules — in a formal language (often logic). A reasoning engine then applies the rules to the facts to deduce answers the programmer never wrote down directly. The knowledge is declarative: you state what is true, not how to compute the answer.
The textbook illustration is a chain of deduction from a rule and a fact:
Scale that up and you get an expert system: hundreds or thousands of hand-written rules capturing a specialist's knowledge in a narrow domain. MYCIN (1970s) diagnosed bacterial infections this way; other systems configured computers or approved loans. For a while they genuinely worked and genuinely made money.
Representation: explicit symbols, facts, and if–then rules written by humans. Inference: logical deduction (chaining rules over facts). Compare to search: search knew the rules of the game but nothing about the domain; symbolic AI is the opposite — it's stuffed with domain knowledge but that knowledge had to be poured in by hand.
The genuine strengths
- Transparency. Every conclusion traces back through named rules — the system can explain itself ("I concluded X because rules 12 and 47 fired"). This is priceless in medicine, law, and finance.
- Precision with sparse data. A single correct rule handles infinitely many cases; you don't need thousands of examples to encode "a triangle has three sides."
- Guarantees. If the facts and rules are correct, the deductions are provably correct.
Every rule must be elicited from a human and hand-coded. That runs into three fatal problems:
- Tacit knowledge can't be written down. (Recall Lesson 1: you recognize a face but can't state the rule.) Whole categories of intelligence resist being turned into if–then statements at all.
- Brittleness. Rules cover the cases their author imagined. Feed the system a situation just outside the rulebook and it fails silently or absurdly — it has no notion of "similar to."
- Maintenance collapse. As rules multiply into the thousands, they interact and contradict each other. Adding rule 5,000 can break rule 12 in ways no one can trace. The system becomes unmaintainable long before it becomes complete.
4 · Two walls, one root cause
Step back and look at the two failures side by side. They look different but share a single root cause.
| Approach | What it needs from a human | The wall it hits | Fails on |
|---|---|---|---|
| Search & planning | the exact rules / state space | combinatorial explosion | huge spaces; noisy, uncertain, or unknown states |
| Symbolic / logic | all the knowledge, as explicit rules | knowledge-acquisition bottleneck | tacit knowledge; novel cases; scale |
Both approaches demand that a human specify the intelligence up front — either as the complete rules of the space, or as the complete body of knowledge. That works only when the problem is small enough and clean enough for a person to fully articulate. The messy, high-variety, tacit-knowledge-laden real world is neither. The bottleneck was never the computer — it was the human in the loop, trying to write down what they cannot fully articulate and cannot exhaustively enumerate.
The escape: invert the pipeline
If the human can't write the rules, remove the human from that job. Instead of programmer → rules → program, do data + desired outcomes → learning procedure → rules. Let the machine infer the rules from examples. This is exactly the pivotal insight from Lesson 1 — and now you can see it solves both walls at once:
The learning approach fills in the <T, P, E> template from Lesson 1: the task T stays the same, but the rules now come from experience E (data) instead of a programmer, and we tune them to maximize a performance measure P. Where symbolic AI asked a human "what are the rules?", ML asks the data "what rules make P as high as possible?"
5 · Reality is a composite — don't over-learn this lesson
It would be a mistake to walk away thinking "search and logic are obsolete." They aren't. The honest picture is that modern systems combine all the approaches, each where it's strongest:
- A self-driving car learns to perceive pedestrians (statistical learning), searches for a safe trajectory (planning), and obeys hand-written rules for hard legal constraints ("never cross a solid line").
- A modern chess engine searches millions of positions but ranks them with a learned evaluation function — the two approaches fused.
- Even large language models are wrapped in search-like decoding and rule-based guardrails.
On any new problem, ask: can I write the rules down completely and correctly? If yes and the space is tractable — use search or logic; they're exact, explainable, and need no data. If no — because the knowledge is tacit, the cases are too varied, or the input is noisy — then learn the rules from data. Most valuable modern problems fall in the second bucket, which is why this course lives there.
6 · Limitations — what this reframing does not claim
- Learning is not magic. It trades the knowledge bottleneck for a data bottleneck — you now need many good examples, and their quality decides everything (Lesson 1's "data quality is destiny").
- You lose transparency by default. Symbolic systems explained themselves; many learned models don't. Interpretability becomes something you must deliberately choose or engineer.
- Learned rules can be wrong in new ways. A model happily absorbs biases and spurious correlations in the data — failures symbolic systems never had.
- "Learn it" isn't always the answer. For small, crisp, rule-expressible problems, hand-written logic is still simpler, cheaper, and safer than training a model.
How this connects to the next lesson
You now understand the approaches and why the learning approach won. But the vocabulary around it is a mess — people say "AI", "ML", "Deep Learning", and "Data Science" almost interchangeably, and they are not the same thing. In Lesson 3 — AI vs ML vs Deep Learning vs Data Science, we draw the precise containment map so you can place any tool, headline, or job description in its correct box. After that, Lesson 4 opens the ML machine itself and we finally start filling in the <T, P, E> triple with real mechanics.
Exercises & reflection
Still no code — these sharpen the judgment you'll use constantly. A sentence or two each is enough.
For each task, decide whether you'd write the rules (search/logic) or learn them from data (ML), and say which wall the other approach would hit:
- Validating that a credit-card number has the correct check digit.
- Deciding whether a photo contains a cat.
- Finding the cheapest flight itinerary between two cities.
- Detecting sarcasm in a tweet.
Show discussion
(1) Rules. The check-digit algorithm is exact and tiny — learning it from data would be absurd. (2) Learning. "Cat" is tacit and infinitely varied; a rulebook hits the knowledge-acquisition bottleneck instantly. (3) Search. The rules (routes, prices, connections) are known and exact — a classic state-space search, though pruning matters to fight explosion. (4) Learning. Sarcasm depends on context and tone no one can enumerate as rules; symbolic AI would be hopelessly brittle.
A team spends two years hand-coding a 4,000-rule expert system for medical triage. It works in demos but keeps giving dangerous advice on unusual patients, and every new rule seems to break an old one. Name the wall they hit and explain, in one sentence each, the two symptoms described.
Pick one everyday system (a maps app, a spam filter, a smart speaker) and identify at least two of the four approaches working together inside it. This trains the "intelligence is a composite" instinct you'll need all course.
Intelligent systems are built by answering two questions — how to represent the problem and how to infer an answer. The first two historical answers were search (explore a space of states with heuristics) and symbolic logic (deduce over hand-written rules). Both demand that a human specify the intelligence up front, so both hit walls: search explodes combinatorially and drowns in noise, while logic chokes on the knowledge-acquisition bottleneck. Machine Learning escapes both by inverting the pipeline — inferring the rules from data instead of from a person — which is exactly why it became the dominant approach and the subject of this course.