Hitting the Books: How to build a music recommendation ‘information-space-beast’

As of October, singers, songwriters and music makers are uploading 100,000 new songs every day to streaming services like Spotify. That is too much music. There’s no reality, alternate or otherwise, wherein someone could conceivably listen to all that even in a thousand lifetimes. Whether you’re into Japanese noise, Russian hardcore, Senegalese afro-house, Swedish doom metal, or Bay Area hip hop, the sheer scale of available listening options is paralyzing. It’s a monumental problem that data scientist Glenn McDonald is working to solve. In the excerpt below from Computing Taste: Algorithms and the Makers of Music Recommendation, author and Tuft’s University anthropologist Nick Seaver explores McDonald’s unique landscape-based methodology for surfacing all the tracks you never knew you couldn’t live without.

Reprinted with permission from Computing Taste: Algorithms and the Makers of Music Recommendation by Nick Seaver, published by The University of Chicago Press. © 2022 by The University of Chicago. All rights reserved.

The World of Music

“We are now at the dawn of the age of infinitely connected music,” the data alchemist announced from beneath the Space Needle. Glenn McDonald had chosen his title himself, preferring “alchemy,” with its esoteric associations, over the now-ordinary “data science.” His job, as he described it from the stage, was “to use math and typing and computers to help people understand and discover music.”

McDonald practiced his alchemy for the music streaming service Spotify, where he worked to transmute the base stuff of big data — logs of listener interactions, bits of digital audio files, and whatever else he could get his hands on — into valuable gold: products that might attract and retain paying customers. The mysterious power of McDonald’s alchemy lay in the way that ordinary data, if processed correctly, appeared to transform from thin interactional traces into thick cultural significance.

It was 2014, and McDonald was presenting at the Pop Conference, an annual gathering of music critics and academics held in a crumpled, Frank Gehry–designed heap of a building in the center of Seattle. I was on the other side of the country, and I followed along online. That year, the conference’s theme was “Music and Mobility,” and Mc Donald started his talk by narrating his personal musical journey, playing samples as he went. “When I was a kid,” he began, “you discovered music by holding still and waiting.” As a child at home, he listened to the folk music his parents played on the stereo. But as he grew up, his listening expanded: the car radio offered heavy metal and new wave; the internet revealed a world of new and obscure genres to explore. Where once he had been stuck in place, a passive observer of music that happened to go by, he would eventually measure the progress of his life by his ever broadening musical horizons. McDonald had managed to turn this passion into a profession, working to help others explore what he called “the world of music,” which on-demand streaming services had made more accessible than ever before.

Elsewhere, McDonald (2013) would describe the world of music as though it were a landscape: “Follow any path, no matter how unlikely and untrodden it appears, and you’ll find a hidden valley with a hundred bands who’ve lived there for years, reconstructing the music world in methodically- and idiosyncratically-altered miniature, as in Australian hip hop, Hungarian pop, microhouse or Viking metal.”

Travelers through the world of music would find familiarity and surprise — sounds they never would have imagined and songs they adored. McDonald marveled at this new ability to hear music from around the world, from Scotland, Australia, or Malawi. “The perfect music for you may come from the other side of the planet,” he said, but this was not a problem: “in music, we have the teleporter.” On-demand streaming provided a kind of musical mobility, which allowed listeners to travel across the world of music instantaneously.

However, he suggested, repeating the common refrain, the scale of this world could be overwhelming and hard to navigate. “For this new world to actually be appreciable,” McDonald said, “we have to find ways to map this space and then build machines to take you through it along interesting paths.” The recommender systems offered by companies like Spotify were the machines. McDonald’s recent work had focused on the maps, or as he described them in another talk: a “kind of thin layer of vaguely intelligible order over the writhing, surging, insatiably expanding information-space-beast of all the world’s music.”

Although his language may have been unusually poetic, McDonald was expressing an understanding of musical variety that is widely shared among the makers of music recommendation: Music exists in a kind of space. That space is, in one sense, fairly ordinary — like a landscape that you might walk through, encountering new things as you go. But in another sense, this space is deeply weird: behind the valleys and hills, there is a writhing, surging beast, constantly growing and tying points in the space together, infinitely connected. The music space can seem as natural as the mountains visible from the top of the Space Needle; but it can also seem like the man-made topological jumble at its base. It is organic and intuitive; it is technological and chaotic.

Spatial metaphors provide a dominant language for thinking about differences among the makers of music recommendation, as they do in machine learning and among Euro-American cultures more generally. Within these contexts, it is easy to imagine certain, similar things as gathered over here, while other, different things cluster over there. In conversations with engineers, it is very common to find the music space summoned into existence through gestures, which envelop the speakers in an imaginary environment populated by brief pinches in the air and organized by waves of the hand. One genre is on your left, another on your right. On whiteboards and windows scattered around the office, you might find the music space rendered in two dimensions, containing an array of points that cluster and spread across the plane.

In the music space, music that is similar is nearby. If you find yourself within such a space, you should be surrounded by music that you like. To find more of it, you need only to look around you and move. In the music space, genres are like regions, playlists are like pathways, and tastes are like drifting, archipelagic territories. Your new favorite song may lie just over the horizon.

But despite their familiarity, spaces like these are strange: similarities can be found anywhere, and points that seemed far apart might suddenly become adjacent. If you ask, you will learn that all of these spatial representations are mere reductions of something much more complex, of a space comprising not two or three dimensions but potentially thousands of them. This is McDonald’s information-space-beast, a mathematical abstraction that stretches human spatial intuitions past their breaking point.

Spaces like these, generically called “similarity spaces,” are the symbolic terrain on which most machine learning works. To classify data points or recommend items, machine-learning systems typically locate them in spaces, gather them into clusters, measure distances among them, and draw boundaries between them. Machine learning, as the cultural theorist Adrian Mackenzie (2017, 63) has argued, “renders all differences as distances and directions of movement.” So while the music space is in one sense an informal metaphor (the landscape of musical variation) in another sense it is a highly technical formal object (the mathematical substrate of algorithmic recommendation).

Spatial understandings of data travel through technical infrastructures and everyday conversation; they are at once a form of metaphorical expression and a concrete computational practice. In other words, “space” here is both a formalism — a restricted, technical concept that facilitates precision through abstraction — and what the anthropologist Stefan Helmreich (2016, 468) calls an informalism — a less disciplined metaphor that travels alongside formal techniques. In practice, it is often hard or impossible to separate technical specificity from its metaphorical accompaniment. When the makers of music recommendation speak of space, they speak at once figuratively and technically.

For many critics, this “geometric rationality” (Blanke 2018) of machine learning makes it anathema to “culture” per se: it quantifies qualities, rationalizes passions, and plucks cultural objects from their everyday social contexts to relocate them in the sterile isolation of a computational grid. Mainstream cultural anthropology, for instance, has long defined itself in opposition to formalisms like these, which seem to lack the thickness, sensitivity, or adequacy to lived experience that we seek through ethnography. As the political theorists Louise Amoore and Volha Piotukh (2015, 361) suggest, such analytics “reduce heterogeneous forms of life and data to homogeneous spaces of calculation.”

To use the geographer Henri Lefebvre’s (1992) terms, similarity spaces are clear examples of “abstract space” — a kind of representational space in which everything is measurable and quantified, controlled by central authorities in the service of capital. The media theorist Robert Prey (2015, 16), applying Lefebvre’s framework to streaming music, suggests that people like McDonald — “data analysts, programmers and engineers” — are primarily concerned with the abstract, conceived space of calculation and measurement. Conceived space, in Lefebvrian thought, is parasitic on social, lived space, which Prey associates with the listeners who resist and reinterpret the work of technologists. The spread of abstract space under capitalism portends, in this framework, “the devastating conquest of the lived by the conceived” (Wilson 2013).

But for the people who work with it, the music space does not feel like a sterile grid, even at its most mathematical. The makers of music recommendation do not limit themselves to the refined abstractions of conceived space. Over the course of their training, they learn to experience the music space as ordinary and inhabitable, despite its underlying strangeness. The music space is as intuitive as a landscape to be walked across and as alien as a complex, highly dimensional object of engineering. To use an often- problematized distinction from cultural geography, they treat “space” like “place,” as though the abstract, homogeneous grid were a kind of livable local environment.

Similarity spaces are the result of many decisions; they are by no means “natural,” and people like McDonald are aware that the choices they make can profoundly rearrange them. Yet spatial metaphorizing, moving across speech, gesture, illustration, and computation, helps make the patterns in cultural data feel real. A confusion between maps and territories— between malleable representations and objective terrains— is productive for people who are at once interested in creating objective knowledge and concerned with accounting for their own subjective influence on the process. These spatial understandings alter the meaning of musical concepts like genre or social phenomena like taste, rendering them as forms of clustering.