Shhhhhh, Facebook Actually Is Listening: Surveillance Marketing Explained

You’ve heard the questions and conspiracy theories, but we’ve done the research, and guess what? Facebook really is listening. So are your other devices. Here’s how that affects advertising, why you should care — and how to lock your devices down. Short of moving all conversations to a cone of silence, that is. 

In a recent Estipona team meeting, our VP of Creative, Paige Galeoto, was sharing a seemingly inconsequential tidbit about her life: Namely, the fact that she had recently purchased recyclable produce bags for her trips to Raley’s. 

Random story, I know. But hang tight: There’s a reason I’m revealing such a compelling anecdote from the chatty portion of our work meetings. 

Because I, a consummate shopper but clearly remarkably unwoke when it comes to how to be eco-friendly with my butter lettuce, piped up with a burning question: Where does one find and procure these recyclable produce bags? 

“Amazon, duh,” she retorted. We laughed, talked about how of course Amazon is the destination for something so niche, then lamented the irony of how the bags were shipped in a big box with loads of packing material. At that point it was back to our regularly scheduled advertising strategy meeting. 

I’m embarrassed to admit this, but I did not follow up immediately to purchase said recyclable produce bags. I got distracted (so many shiny objects in strat-comm meetings!), so I did not think about it again: no searching, no notes left to myself, nothing added to my to-do list about visiting Amazon, nothing. So this would have been end of story — until this happened. 

Within two hours, I was browsing Facebook and staring in disbelief at this: 

Creepy much? But I get it. Mark Zuckerberg and his team of covert minions totally listened to our conversation, right? 

According to them, nope: 

Facebook does not use your phone’s microphone to inform ads or to change what you see in News Feed. Some recent articles have suggested that we must be listening to people’s conversations in order to show them relevant ads. This is not true. We show ads based on people’s interests and other profile information – not what you’re talking out loud about.

I mean, come on, Facebook: So I’m just supposed to chalk this up to random coincidence? They’re just so good at anticipating my behavior, that they predicted the exact moment I became a my-Raley’s-bags-of-butter-lettuce-are-killing-the-earth level of woke?

I can say with confidence that I do have an open mind, but I’m no sucker.

Literally No One Believes This is a Coincidence

And I’m not alone in my suspicions. Recently, people shared similar stories of unbelievable advertising "coincidences" with the BBC — from a newly engaged couple who spontaneously bought their engagement rings and told absolutely no one but almost immediately started getting wedding ads, to the friends who decided to blatantly test the concept and started talking proactively about a random product and indeed were served up ads about that product within 10 minutes. 

So let’s get down to it. Being in this industry, I am not naïve to the tactics Facebook and other similar online resources are using in order to target advertising. “Personalization” is the name of the tactic, and overwhelmingly, this is a very good thing: I do not, for example, want to be served up ads for a toque blanche (white hat) because I am, in fact, not a professional chef (shocking, I know). I do, however, desperately want to get ads for a Disney Cruise, because I’m a rabid Disney fan who loves to daydream about spending an exorbitant amount of money on my next vacation surrounded by oversized mice and insidiously happy music. 

But sometimes this personalization can cross a line — one that can be deemed “surveillance marketing.” We as advertising strategists often talk about the uncomfortable reality that such targeting can be seen as creepy. 

According to this analysis: “… as consumers learn more about how advertising personalization works, they like it less. A recent YouGov study found that 45 percent of UK consumers are against their data being used for personalization of information, services, and advertising, and 54 percent find personalized advertising creepy.”

I’ll admit, I’m one of those people. Which is why, in my Facebook settings, my microphone is set to “off.” 

The Loophole: How Facebook Still Hears You When Your Mic is Off

So then how exactly did the Recyclable Produce Bag Incident of 2019 happen? Surely it’s not a coincidence that an ad for recyclable produce bags that I’ve never seen in my life is served up to me the exact day I ask where one finds such a thing?

In a Vice article evocatively titled “Your Phone is Listening and it’s Not Paranoia,” cybersecurity consultant and researcher Dr. Peter Hannay suggests there’s a strong possibility that Facebook can be triggered to record us occasionally — but that the trigger itself is key. Just as Siri requires a “Hey Siri” or Alexa waits to hear its name, triggers inspire the active listening part for the device. 

According to him:

“From time to time, snippets of audio do go back to [other apps like Facebook’s] servers but there’s no official understanding what the triggers for that are,” explains Peter. “Whether it’s timing or location-based or usage of certain functions, [apps] are certainly pulling those microphone permissions and using those periodically. All the internals of the applications send this data in encrypted form, so it’s very difficult to define the exact trigger.”

He goes on to explain that apps like Facebook or Instagram could have thousands of triggers. 

So in my example, was the “where would I find (fill in the blank)” be the trigger? It’s very possible. But how do you explain this, given that my microphone was off? Well, I was sitting in a room with seven other people — presumably not all of whom have taken such stringent measures. And we’re sharing a network, and thus an IP address. 

Aha! Mystery solved. 

But alas, there are other ways to connect the dots. Here are a few other possible explanations:

• According to this story, there are apps that you download for other purposes (like game-playing) that specifically include data-tracking software. Of course you “give permission” when you download them, but how many of us really read those permissions? Most of them are TL;DR anyhow, so we dismissively click “Accept” without another thought. With that simple action, it’s possible your phone just became a microphone. 

• Similarly, in my example, as I mentioned we were all in the same room, on the same network, sharing an IP address. Paige has obviously searched for recyclable produce bags and purchased them on Amazon. Therefore, because she’s interested in recyclable produce bags, and I hang out with her frequently, I, too, must be interested in recyclable produce bags. “Interest by proxy” is the technical term, so if you frequently hang out with people who have recently purchased something unusual, there’s a good chance you’ll be served up an ad because they’re you’re friend, so clearly you like the same things they do. 

Facebook Rolls Out Something New and Slightly Less Stalkery!

In fairness, Facebook understands your concerns (and seems a bit miffed that you keep devising nefarious plots, you conspiracy theorist you). And now it’s pulling back the curtain just a wee bit to help you understand why you see what you see when you see it, having just rolled out a new feature that explains to users how it picks your personalized ads. 

This, people, is very big news! Well, kinda.

According to this Guardian story, here is what happens when you click the “Why am I seeing this?” link that appears with suggested posts: 

“The resulting information screen will show generally obvious information about the source of the post, like the fact that the two accounts are friends, or that the user follows the page that posted the content. But it will also show more detailed information about what caused a given post to be placed high up on a user’s news feed. Examples shared by Facebook include ‘you’ve liked [this friend’s] posts more than posts from others’, ‘you’ve commented on posts with photos more than other media types’ or ‘this post in [this group] is popular compared to other posts you’ve seen’.”

However, the following quote offers some insight into how this is simply a peek behind the curtain — and not a big reveal: 

“But the vast majority of the ‘tens of thousands’ of signals the company analyses to decide what content to show users will remain hidden, Facebook said, to avoid overwhelming people with potentially irrelevant information.

Oh wow. Thank you, Facebook! So really, you’re doing this for us: You’re so concerned for our well-being, so committed to watching out for us and not overwhelming us with all the “potentially irrelevant information,” that you’re going to bite the bullet and not share all the information. That’s awesome! (Where is the sarcasm font when you need it?) 

Who Else is Listening, Exactly? 

So obviously, we know Facebook isn’t the only one doing this, right? In fact, Amazon recently came under fire because it has been reported that not only is Alexa listening when you speak to an Echo, a literal Amazon employee is potentially listening, too — all in the name of helping the software improve its grasp of human speech. 

Yep. According to this report: 

“Amazon employs thousands of full-time workers in several countries, including the United States, Costa Rica and Romania, to listen to as many as 1,000 audio clips in shifts that last up to nine hours. The audio clips they listen to were described as ‘mundane’ and even sometimes ‘possibly criminal,’ including listening to a potential sexual assault.”

Awesome. 

How to Lock Down Your Devices to the Best of Your Ability

So, after all of this: How interested are you in making sure your spoken personal property — aka your words — are as private as much as humanly possible? Here’s the way with Facebook: 

• If you are an iPhone or iPad user, go to Settings >> Facebook and slide the Microphone switch to the left, so it turns from green to white. That turns it off. Alternatively, you can go to Settings >> Privacy >> Microphone, then look for Facebook and do the same. Note that you can toggle the mic on and off for other apps, too.

• For Android users: Try Settings >> Applications >> Application Manager >> look for Facebook >> Permissions >> Turn off the mic.

The important thing to keep in mind with Facebook is that your microphone needs to be on to record a video. So if you’re commonly cutting into with a Facebook Live, you’ll need to remember to turn the microphone on before you record. And here’s the tough part: You’ll also need to remember to turn it OFF again AFTER you record. 

And you know, just for good measure: Here’s how to lock down microphones for other devices/apps: 

• Here’s how to turn on/off “Hey Siri” on your iPhone and iPad.
• Here’s how to listen to (and delete) everything you’ve ever said to Google.
• Here’s how to listen to and potentially delete all the stuff your Amazon Echo has heard you say.

And now, if you’ll excuse me, I’m off to finish construction on the microphone-free cone of silence that I’m adding to my home.

 

Oh yeah, and probably buying some recyclable produce bags, too. 

Mikalee Byerman is the Vice President of Strategy for the Estipona Group, a humor writer and is more than slightly annoyed by the “Divorced mom seeking connection?” Facebook ads that follow her everywhere. Contact her here.