Kata Kovács and Tom O’Doherty

Self-surveillance and Pervasive Data

June 9, 2017
Filed under: · ·

The text below was originally presented as a talk at Scope Sessions #66, in Panke, Berlin, on Thursday, June 8th, 2017. This talk was based around some observations drawn from the first year of Minute/Year and some related artistic strategies.

Minute/Year — Spectrograms, 2016, Day 1 — Day 366
Minute/Year — Spectrograms, 2016, Day 1–366

Hi. We are Kata Kovács and Tom O’Doherty. We are an artist duo, and we live and work in Berlin. We have worked together since 2011.

We tend to work with artistic ideas related to time and duration, repetition and processes, the relationship between sound and architecture, and the combinations of time and movement. We have created works for indoor and outdoor spaces, and they have lasted anywhere from fifteen minutes to multiple years.

We’d like to talk through some ideas about technology — specifically, about the newly-emergent phenomenon of smart speakers, such as the Amazon Echo, Apple HomePod, or Google Home, and the strange overlap that these innovations have had with some of our own work, and the work of other artists.

But firstly, we would like to talk about an ongoing installation work of ours, called Minute/Year.

Minute/Year at G23, Berlin
Minute/Year, as installed in G23 in 2016

Minute/Year is an automated, durational work, in which sound is automatically layered in a resonant space, for one minute each day.

Picture it like this: in an ordinary living room, there are two microphones installed on the wall on one side, and there are two speakers installed on the other side.

Every day, at the same time, the microphones record the sound that occurs in the space, during one specific minute. At the same time, the speakers play back the previous day’s recording — the sound that was recorded in the same space 24 hours before. So, each recording includes traces of its predecessor, which includes traces of its predecessor in turn, and so on, in a potentially infinite regression.

Each day’s minute-long recording is then converted into a spectrogram — an image of the frequencies of the audio — and the two parts are then published online. This entire process is automated — it happens without any human involvement. The minute runs each day at 20:17 (in 2017). In 2016, it ran each day at 20:16.

The grid above shows all the spectrogram images from the whole of 2016 in a single glance — 366 images, one for each day (2016 was a leap year).

Here are some individual images of the spectrograms generated by the installation, over the course of the last 18 months.

Minute/Year, Day 9 — Day 16, 2016
Minute/Year, Day 9 — Day 16, 2016

Minute/Year, Day 183— Day 190, 2016
Minute/Year, Day 183— Day 190, 2016

Minute/Year, Day 14, 2016
Minute/Year, Day 14, 2016

Spectrogram screenprints, as presented at ‘Minute/Year (2016)’ at Lite-Haus Galerie, Berlin, January 2017
Spectrogram screenprints, as presented at ‘Minute/Year (2016)’ at Lite-Haus Galerie, Berlin, January 2017

In January of this year, we had an [exhibition–broken link], presenting hand-made screen-printed images of selected individual spectrograms generated by the installation over the course of 2016.

During the first year of the piece, it was installed in the living room of an ordinary private apartment.

In 2017 — the second year — the piece has been installed in a large open area in Lütticher 5, an apartment and shared living space in Wedding, Berlin, which is is also occasionally used as a gallery and performance venue.

With the new year, we changed the colour of the spectrograms.

Minute/Year, Day 2, 2017
Minute/Year, Day 2, 2017

Minute/Year, Day 1–32, 2017
Minute/Year, Day 1–32, 2017

So, practically speaking, the daily outcome of Minute/Year is a one-minute-long audio file, and accompanying spectrogram image, which are both automatically posted online each day. This means that they go on the website, and also on Vimeo, Mastodon, and our current favourite, Telegram, which gives us a notification on our phone each day as soon as that day’s minute has been posted online.

At this point, we should mention that this piece owes a very distinct debt to two previous works.

Record cover of 'I am sitting in a room' by Alvin Lucier

The first is Alvin Lucier’s 1969 work I am sitting in a room. In this work, a piece of spoken text is recorded, re-played, and re-recorded, over and over, in a room, as it slowly transforms into a fascinating and beautiful loop of room resonances. This work might be quite familiar to some, as it has been an influence for hundreds of other works, ours included.

The second obvious debt that we owe is to On Kawara’s Today series of paintings, which was begun in 1966.

On Kawara's Today series

These paintings are a conceptual investigation of duration and repetition, and they simply show the date on which they were painted, plainly rendered in white on a coloured background.

On Kawara's Today series

They were painted by the artist from January 1966 right up until his death in 2014.

We want to be clear about the influence of these two antecedents in particular on Minute/Year. However, the piece is also motivated by many other overlapping themes.

Can surveillance be turned inside-out? What happens when public and private contexts are collapsed, for one minute each day? Is it possible to simultaneously be a listener and a performer? What ghosts lurk in the constant repetition and layering of sound in a single space? Who, and where, are the audience, in a work that is mainly experienced online?

These questions, and the discussions arising from them, frame the interests that have motivated the creation of the work.

Minute/Year has been created with the technical collaboration of Kris Slyka.

Minute/Year began on January 1st, 2016. It has run every single day since then, for one minute each time. It has no set end date.

Minute/Year — all spectrograms from 2016 in series, in 49 seconds.

Ok, time for a little change of focus.

This is the Amazon Echo:

Amazon Echo

And this is the Google Home:

Google Home

And the newly-announced Apple HomePod:

Apple HomePod

These objects, and similar ones from various other companies, are usually referred to by the generic category name of smart speakers.

They are internet-connected boxes, which contain both speakers and microphones, and which allow people to do everyday internet stuff by just saying it out loud — “hey, internet, what’s the weather like in Tokyo?” or “hey, Amazon, order me a new box of cornflakes” or “hey, magic tube, play the new Horse Lords album” — and having the box then announce the answer, or order the stuff, or play the music.

Smart speakers
The future is tubular

In their implementation, smart speakers have a lot of practical parallels with an artwork like Minute/Year. Both involve recording what’s happening in a space, playing something back, and putting data online as a result.

However, they are also, obviously and practically speaking, extremely different to Minute/Year.

Part of what motivated us to begin Minute/Year was the possibility to create a sort of daily moment of absurdist self-surveillance. We were interested in the idea that we could deliberately turn a private space into a public one, for one minute every day.

When we began, we never thought that the whole notion could be turned on its head, and that people would start to embrace the idea of being permanently surveilled by corporations inside their own homes.

What is even stranger to think about is that, over the course of 2016, when Minute/Year began, this new innovation in consumerism has become much more widespread. At the beginning of 2016, these sorts of smart speakers were very much a tech-nerd thing. However, as time has passed, they seem to be slowly becoming ‘normal’ — and the big software companies are pushing them, hard.

Amazon Echo advertisement, Postdamer Platz, Berlin, June 8., 2017
Amazon Echo advertisement, Postdamer Platz, Berlin, June 8., 2017

Compare it to our phones. In 2007, only nerds and rich people had iPhones or other smartphones. However, by late-2008-ish, or early-2009-ish, more-or-less everyone had them. By 2010 or 2011, your grandparents had them.

These new smart-speaker boxes seem to be undergoing a similar process.

Pinboard tweet
The ever-reliable, ever-snarky Maciej Cegłowski, owner of Pinboard (link)

Some numbers: the Amazon Echo was first released (to Amazon Prime members only, and in the United States only) in November of 2014. It was then generally released in the United States in June 2015. It was released in the UK in September 2016, and in Germany in October 2016. Amazon doesn’t disclose sales figures, but by most industry estimates, they have sold over ten million of these devices in just the last 12 months, from June 2016 to June 2017. Some estimates claim that the combined total of smart speakers of all kinds sold to date in the United States is 35 million.

In March of this year, just two months ago, audio that was recorded by an Amazon Echo was used, for the first time, as evidence in a murder case, in Bentonville, Arkansas.

Amazon Echo in murder case
The world is a strange place.

Clearly, this is an emerging mass product and social phenomenon, just like previous ones such as smartphones in the 2010s, or before that, television in the 1980s, or before that, private cars in the 1950s and 1960s.

We started Minute/Year at the beginning of 2016. In the time since then, one of the central ideas involved in it — the gentle absurdity of having a computer listening in to a domestic space — has undergone a sort of social transformation. At the beginning of 2016, humans were not being listened to by boxes in their homes, generally speaking. But humans in 2018 or 2019 will possibly feel like this is a perfectly normal way to exist. It feels a little strange, but it’s certainly one way to gain some durational perspective on your own work. There is an odd kind of irony in having the ocean of late-capitalist efficiency engulf your artwork in unexpected ways.

So we have this newly-emergent reality. However, we would like to hope that in this changed technological landscape, our little open-ended artistic project retains, or even strengthens, its social or aesthetic charge.

Here are some of the questions that we have found ourselves asking:

If everyone slowly becomes used to the idea that they are always being listened to, how does this change the act of listening?

If everyone is always acoustically observed, does that inevitably turn domestic spaces into a site of performed activity?

Can self-surveillance be a method of reacting to, or transforming, the habits and culture of this strange new era?

Any one of these questions could be the jumping-off point for a whole essay or talk — even a book-length investigation. But we would like to stay, for now, with our focus on the creative use of automated data and observation, and mention some related works by other artists.

There are certainly many other artistically-motivated projects that have been based around ideas of data and surveillance (indeed, in some ways, surveillance as a theme is an emerging artistic cliché). There are also many projects based around transforming certain unpredictable, machine-generated inputs into other, artistically-inclined outputs. However, there are a certain group of works that specifically attempt to combine self-surveillance with datasets, records, or indexes. We’d like to mention a couple of them here.

Many of these kinds of works have started from the perspective of using video or photography. One of the first of these efforts was undertaken by Steve Mann, who also coined the term ‘sousveillance’ to refer to the act of deliberately attempting to turn surveillance on its head.

Steve Mann, Visual Filter (1995)
Steve Mann, Visual Filter (1995)

Here’s a picture of Mann with his Visual Filter, used for continuous live webcast, in 1995. (We can now all do the same thing with the phones in our pockets.)

Mann’s daughter illustrated very well the difference between surveillance and sousveillance in a drawing, made when she was six years old.

Surveillance vs. sousveillance, as explained by Steve Mann’s six year-old daughter.
Surveillance vs. sousveillance, as explained by Steve Mann’s six year-old daughter.

Closer to home, there are the works that have been created by Sophia New and Dan Belasco, an artist duo who live and work in Berlin. They work together under the name plan b performance.

All My Journeys in Berlin 2003–2012 (smaller scale) — Daniel Belasco Rogers
All My Journeys in Berlin 2003–2012 (smaller scale) — Daniel Belasco Rogers

Dan and Sophia have both, since 2003 and 2007 respectively, been carrying GPS tracking devices, recording everywhere they go, so they have a permanent record of their movements in time and space. They have used this information as material to create various works.

All Our Journeys in Berlin 2007–2012 — Daniel Belasco Rogers and Sophia New (2013)
All Our Journeys in Berlin 2007–2012 — Daniel Belasco Rogers and Sophia New (2013)

Ironically, Dan’s and Sophia’s work is another durational project that has been, in a sense, overtaken by capitalism. When they started doing this process, they were the only people in the world doing this. But we all carry GPS devices now.

More recently, there have been a series of works by the American artist Michael Mandiberg (who is probably best-known for his ongoing efforts to attempt to print out the entirety of Wikipedia).

Print Wikipedia — Michael Mandiberg
Print Wikipedia — Michael Mandiberg

One of Michael’s more recent works is Quantified Self Portrait (Rhythms), which is part of his ‘Workflow’ series of works. Quantified Self Portrait (Rhythms) works by taking the sounds of a year of the artist’s heart rate, alongside the sound of a year of his email alerts, and playing them back in real-time, the following year. He describes the project as a way to use himself as “a proxy to hold a mirror to a pathologically overworked and increasingly quantified society, revealing a personal political economy of data.” His self-surveillance, of both his body and his work in parallel, results in a subtle, durationally-inclined outcome.

Video documentation of ‘Quantified Self Portrait (Rhythms)’, as installed at LACMA, Los Angeles

All of these undertakings — by Steve Mann, by Plan B Performance, by Michael Mandiberg, and our own Minute/Year installation — start with the implication, or the reality, of ambient surveillance and pervasive data, and stretch their artistic possibilities.

We’re curious to see where these projects will lead. Likewise, we’re curious to see what other similarly-motivated works by other artists will emerge in a new era when it slowly becomes normal to presume that you are being listened to, or otherwise observed.