0.0 Context Setting
I started writing this on Saturday 31 October 2020, which was Halloween (if you’re in a culture that does halloween), and two days until the U.S. Presidential Election. I say two days but it was actually several million years ago.
I’m writing this paragraph on Wednesday 4 November 2020.
I’m not going to write about how both Halloween and the U.S. Presidential Election are their own horrific existential reflection of our times because, well, gestures at everything.
Stuff I’ve done lately:
bitten the bullet and bought a whole set of clamps, articulated arms, double ball joints, camera mounts, pass-through AC adapters so that I can join all the other oooh look at you with your fancy camera video conferencers and hook up my SLR to my computer to use as ridiculously expensive webcam.
Currently listening to: Mint Royale’s 1999 album, On The Ropes. This album came out during my first year of college. It’s fair to say it has memories attached to it.
1.0 Something That Caught My Attention, And Then Something That I’m Thinking About As A Result
There’s two parts to this one:
1.1 Shot: Are you there, Alexa? It’s me, Dan.
It started when I saw a tweet from Lilly Irani a professor at the University of California San Diego’s Department of Communication. Irani had pointed to a Google patent [US9037455B1, 2015, Limiting notification interruptions] that in her words was “for a device that delays notifications by listening to whether there is human speech and waiting until the human is done talking”, with the question that “while seemingly polite, isn’t this also your phone always listening to you?”.
(Aside: there’s a certain deliciousness here: Irani points out that Tristan Harris, now of the Center for Humane Technology, is a coauthor).
Anyway, first some specifics: first, the phone isn’t always listening, the claim is about when a navigation app is running, and a set period of time during which it wants to give a direction notification.
In private conversation with a friend, there’s a lot of detail to tease apart here, which is part of the reason why society is finding it difficult to deal and engage with ethical technology issues like these. First: is it the quality of the sensor, i.e. what’s being sensed, that’s at issue here? There are other continuous sensors, like heart rate sensors that are “constantly” on the look out for whether you might have a particular heart condition. My point there was that there is a difference in degree: a heart rate sensor in the configuration of a watch that works on a wrist can only sense one heart rate at a time, whereas a hypothetical phone that’s “listening” can “hear” everything (in the case people are concerned about, “everyone”) around it.
But so far, when we talk about devices that are always listening, what they’re generally doing is the equivalent of this Gary Larson’s infamous What We Say To Dogs / What They Hear cartoon from The Far Side:
My understanding is that most (not all!) wake-word implementations rely on “listening” that happens on a digital signal processor very close to the microphone, and it’s only when that DSP trips on the wake word does any sort of digitized audio get further into a system. So what should be happening is that at an autonomic level, what a device hears is blah blah blah blah hey siri and then the device wakes and starts sending audio to the next part of the chain.
Lots of computing works this way now. One way is through computational photography. Last year’s iPhone 11 had a fantastic explanation of how taking a photo on your phone works now:
For a while now, you haven’t been the one taking your photos. That’s not a slight at you, dear reader: When your finger touches the shutter button, to reduce perceived latency, the iPhone grabs a photo it has already taken before you even touched the screen.
This is done by starting a sort of rolling buffer of shots as soon as you open the Camera app. Once you tap the shutter, the iPhone picks the sharpest shot from that buffer. It saves a shot you, the user unaware of this skullduggery, assumes you have taken. Nope. You merely provided a hint, to help the camera pick from the many shots it had taken on its own. [from the very smart people at Halide]
I bet that most people would say, on learning this about how phones take pictures, that this is different from feeling like an Amazon Echo is always listening to you. At the very least when you open a camera app, you’ve made an intentional choice to at least be open to be taking a photograph. And there’s a sort of expectation that you know that this rolling buffer is going to be thrown away. I’d argue that you’d know, if only that keeping that buffer around at a good enough resolution would quickly show up, either in your phone’s available storage, or in your phone’s network traffic.
But anyway, this isn’t the interesting part, I don’t think.
(At least not to me, the armchair enthusiast).
Disclaimer: if you’re a lawyer, or you have more than passing knowledge of intellectual property law, then apologies ahead of time for what I’m painfully aware might sound like mansplaining.
What would be interesting is if there were some way to use existing institutions and societal systems as a tool to move the use of technology in a direction more oriented toward society, rather than short-term, damn-the-externalities capitalism.
In other words: what would it look like if, an entity were to file a patent application for a non-contact biometric identification system that you could use to pay for goods in a store, and refuse to license it until sufficient societal protections were in place to prevent it from being used to cause significant harm, or until sufficient studies of potential harm or societal failure modes had been completed?
What would it look like if a patent for a recommendation engine were to be filed and granted, and then licensed only under the same conditions?
In other words, what if patents were used defensively for collective societal needs as a bulwark against The Numbers Must Go Up And To The Right?
This type of thing already exists in a different form, but because we live in a market capitalism world, they’re known as defensive patent pools, or defensive patent aggregation which, because this is an essay, I shall describe by citing wikipedia:
Defensive patent aggregation (DPA) is the practice of purchasing patents or patent rights to keep such patents out of the hands of entities that would assert them against operating companies. [wikipedia]
Just look at that. War by any other means.
This idea - that you could use patents in a way that’s a bit sideways to the usual practice of using them defensively or offensively (ugh) - has come up before in a different way, I think. The Electronic Frontier Foundation wrote about this back in 2012, in The Defensive Patent License and Other Ways to Beat the Patent System and covering the Defensive Patent License. Here’s how the EFF describes how the DPL would work:
DPL patent holders must offer a nonexclusive, royalty-free license for any patent they own to anyone who requests one, as long as the licensee agrees not to sue the licensor or any other member of the DPL community for patent infringement.
The licensee must offer its patents under the DPL with the same conditions to anyone who requests one.
The licenses remain in effect throught the patent's life, even if it is later sold.
The licenses can only be revoked if an offensive patent suit is filed.
The goal of the Defensive Patent License is to bring in the free/open source community to the benefits (such as they are) of the patent system, on the (reasonable) belief that the patent system isn’t going away any time soon, and one option at least would be to improve how it works. So it uses legally binding restrictions in how the patent is licensed to live up to free/open source goals. Anyway, you can read the paper.
Okay, fine. Where am I going with this?
You can do pretty much anything you want with a license. You could write and use, say, a hippocratic license that prohibits the use of software to violate universal standards of human rights, and embodies the principles of Ethical Source Software.
(You could also be an overgrown manchild with more money, influence and power than sense and write into the terms of service (not a license!) of your low earth orbit mesh satellite internet service provider that you accept that no Earth-based government has authority or sovereignty over Martian activities [The Independent] which, TO BE CLEAR, IS A FUCKING PUBLICITY STUNT])
In the real world, though, words and contracts can have consequences, but they have consequences only because they are backed by the force of law. The operative word being “force”. So as Cory Doctorow points out, you’d have to defend the patent, which means money, time and effort. Licenses like this only work so long as you have the ability to enforce them.
Shobita Parthasarathy showed me that there’s been attempts in biotech:
But! I’d love to see what other people have thought about this and where the holes are. Clearly one of the holes is having the resources to actually enforce and defend the patent. That said, even if it’s unrealistic, I hereby propose the Ian Malcom Patent License, that prevents use if you haven’t stopped to think if you should.
Surprise. It took me a week to write this episode. I’m finishing it now on Saturday, November 7 2020, just a few minutes after President-elect and Vice President-elect Biden and Harris just finished their speeches.
It’s day one.
How are you?