Before Large Language Models (LLMs), which is what people now just call "AI", chatbots relied on pattern matching tricks, decision trees and regex. This article covers many of the techniques, a lot of which are still being used in the "modern" AI tools.
ELIZA was built by Joseph Weizenbaum in 1966. ELIZA pretended to be a therapist. It would match keywords against a set of rules and build replies with no real comprehension involved.
The main trick was called reflection: swap "I" for "you" and "my" for "your," then hand part of the sentence back as a question.
It's the oldest trick in the book, and still extremely effective. Any bot that echoes a word or phrase back at you is using it.
PARRY (Kenneth Colby, 1972) simulated a person with paranoia. It had a bank of "moods" like fear or anger, and would increase or decrease the likelihood of those responses using pattern matching.
Same question, different reply, depending on where the hidden "mood" happened to be.
A.L.I.C.E. stored its rules in AIML (Artificial Intelligence Markup Language): patterns with wildcards, where more specific matches win over the general ones, and a way to reference the bot's last reply.
| Pattern | Template |
|---|---|
| MY NAME IS * | Nice to meet you, ⭐. |
| I LIKE * | Why do you like ⭐? |
| * | (fallback, when nothing else matches) |
The ELIZA pattern could scale to thousands of hand-written rules.
A A Markov chain will track which words will follow other words in a body of text. To generate a reply, the bot will start with a word which could be pulled from your message, and then it will keep picking the next word by probability.
MegaHAL is a famous example. the grammar holds up fine; the sentence as a whole often does not. This created the genre's reputation for weird fricken replies.
Instead of generating text, a retrieval-based bot will index a large text of real sentences (movie subtitles / chats) and return the closest match to what you sent.
Has good grammar because a human wrote every line, but it is only as good as the amount of predefined text. If there aint a lot of text you get a bunch of off-topic replies.
Monte Carlo methods are named after the Monte Carlo casino and coined by physicists Stanislaw Ulam and John von Neumann, probably a couple of degenerate gamblers, while they were working on nuclear weapons research at the Los Alamos research facility in the late 1940s. The core idea was that when a problem was too complex to solve cleanly, or too random, lets run it many times with random inputs and use the spread of outcomes as an estimate. It has since become a major tool for physics, finance, and computing wherever random sampling can be used. Think sports scores.
Chatbots could use the same methodology. Keep a pool of replies and roll the dice to pick among them, roll the dice on whether to reply at all, even roll dice on how many characters pile into a conversation. Yatzee Baby.
Obviously this isnt generating real speech, but it can be the glue that will stop a rule-based bot from repeating the same line every time, breaking the roleplay.
Loebner Prize bots learned that "I don't understand" will break the (this is a human) illusion. So they decided to turn the question back on the user; change the subject with a joke, or be distracted/confused.
This one aint about the language, more about the timing and small imperfections.
This is absolutely huge for creating "real, half-distracted human." behaviour.
Modern AI models language are way better at generating fluent, context-aware text instead of picking from rules or canned pools. But a lot of these tricks are still used by all of these "modern" AI tools.
*) meaning "match anything," which would be used inside a pattern.