I used to be suspicious of claims that supervised learning is the wrong way towards general artificial intelligence, like this DeepMind post claiming that humans actually learn by unsupervised learning more than supervised. In their words, the distinction is one of learning with (supervised) or without (unsupervised) a specific task in mind. The authors cite toddlers and Ludwig Wittgenstein’s “language games” to bolster their arguments, but on both accounts I would disagree.

In the case of Wittgenstein, he develops his theory of “language games” in the book Philosophical Investigations, where his examples are task-oriented, i.e. more like supervised learning than unsupervised. The prototypical example is of two builders lacking a common language, where one tries to get the other to pass him a brick (Section 2, p. 15 of the Bibliothek Suhrkamp edition). As for toddlers, sure there is a lot of exploration, but with my daughter at least, I felt like her exploration of the world and development were guided by task-oriented feedback involving comfort vs. discomfort, warmth, hunger, etc.. Note that in both of these situations, the type of learning fits within the framework of reinforcement learning, rather than classical supervised learning, as the learning process features interactions with delayed feedback.

But a coincidence of Covid-19 induced boredom (I am learning to hackysack) and social-distancing (I caught up with a friend over the phone and without beer, so we actually sent each other relevant links post-chat) has changed my mind. Christopher Prohm directed me to this tutorial from ICML 2019 on open-ended algorithms by Jeff Clune, Joel Lehman and Ken Stanley. One contribution of open-ended algorithm work is that sometimes you can find global optima better if your goal is novelty rather than finding the optimum. In terms of the near-sighted hiker methaphor I used for my first research talk on work with Harald Scheraga’s protein folding group at Cornell University, the promise of open-ended algorithms is that a near-sighted hiker seeking the lowest elevation of a landscape can have better success by optimizing new scenery than actually looking for a lowpoint.

Now for ingredient two in my change of heart on supervised learning. When I was in middle school, the cool kids were good at hackysack. Back then I was still fighting my inner geek, so I learned to be OK, though never good, with a hackysack. But with the Covid-19 restrictions, a hackysack is just about a perfect fit for our small lawn. As I have been “hacking” (that’s what we called it in suburban Philadelphia in the late 80’s), I have been thinking what the feedback loop is that leads to improvement. I have a hard time believing that each time my shoe hits the hackysack in a good way that I have some dopamine burst that reinforces this foot-eye coordination. While I do have some abstract task in mind (several, in fact, such as getting better, staving off boredom and just moving around), on the brain-foot interface it feels more like fun-oriented learning, rather than some physical analogue of supervised learning.

Perhaps, however, the issue is more about the origin of different tasks and goals, and which kind of learning is best suited for each. Toddler and language development have been formed by the goal of survival over millenia of evolution, whereas hackysacks have nothing to do with survival, except maybe by some farfetched analogy to bird mating rituals translated into middle-school social dynamics. Rather, with hackysacks and its many variations, we see humans both pursuing some immediate task (keeping the hackysack in the air) and creating new problems at the same time (see e.g. the POET paper).

A big thanks to Christopher Prohm for suggestions and pointing out less-than-rigorous statements in an earlier version of this post. All remaining nonsense is my own.