How My First Big Project Failed Because I Didn’t Know Better
It was the year of 1986, people (including me) were wearing weird jeans jackets and hair gel was in high demand. Just the year before I had bullied my wider family into giving me enough money to get my first computer (my first experience with a funding round, if you will).
I was fascinated by text-adventure games (I will write more about that another time). Somehow they connected my love for computers with reading, and — slightly autistic as I was — here was finally someone (or something) you could actually talk to.
Quick explainer for those who did not live through that time: Text adventures worked like a digital “choose your own story” game, but (at least in the more advanced ones) by not just by having you select from a fixed list of options, but by actually telling the computer what you wanted to do in (almost) natural language.
However, they were all written in English and (in Germany) hard to get hold of. So I did what any ambitious 15 year old would do, I started to write my own.
I created a fictional universe, drafted a story. I even sketched a fictional map of the town the adventure would take place in (my love of maps came right after my love for computers). Soon I arrived at what I dreamed to do in the first place: Writing my own natural language parser and write code to handle the dialogue between user and machine.
There was no world wide web around to get instructions on how to go about it, no code was available as open source. So reverse engineering was the only way: I played Zork (published by the legendary company Infocom) at the computer belonging to the father of a friend, and more some games which were on display in the local department store (I had to bribe the store clerk with a coffee several times to let me play for hours). I played after school until store closure, and sometimes at times when I was supposed to be in school.
I started to create myself a dictionary of common verbs, nouns, articles and learned how sentences are structured, made long lists of possible word combinations. Tokenization felt natural, though I didn’t know at the term it was called that.
I got far, handling common verb-article-noun combinations and even various ways to arrange them. However, I simply did not understand how to handle all possibilities I expected as input. Far too often I ended up with “I do not understand what you are saying”. It was to easy to break and be thrown out of the game experience. Attempting to solve that I ran into other issues, such as my complete disregard for memory management, having exhausted the 32 kilobytes of my machine pretty quickly. Frustrated, I abandoned the project after a few months.
Looking back years later, I saw clearly the flaw in my approach. I did far too well. My parser was in (some) parts more advanced than Infocom’s. I tried to predict every possible input the interpreter could receive. Looking back at Zork (now available in emulators), I see how Infocom did not care about handling distracting attempts and focused on moving the main story forward.
It was only much later that I learned about the 80–20 rule (the “Pareto principle”), applied to my case meaning the importance to focus on satisfying a maximum 80% (or even much less) of possible situations and ignoring the rest which would require easily 1000% of work time to handle.
Infocom’s developers obviously had an extremely good feeling for what was needed to create their immersive game experience, most likely because they were able to take on the role of a game player, not the one of a hacker who tries to break things.
I never really got back to writing a game like that, mainly because they fell out of fashion with the rise of computer graphics. My obsession with human-machine interaction however has followed me for the next decades. I have written a lot of NLP code since then (and my fight with perfection continues) and every time I start a project those months pop up in my head.
