Recommendation Algorithms Are a Necessary Evil (Sort Of)

Megan was brought into the embrace of the good lord Jesus Christ through the power of YouTube. She started with mommy bloggers who had, as she described it, a “really positive energy.” From there, she noticed that they frequented the same Targets and drank Diet Coke from the same drive-throughs and had the same bleached blonde hair and went to the same church—i.e., they were all from Utah. 

Her investigation into their personal lives surfaced a video series entitled “I’m a Mormon.” She dove into the deep end of the baptismal font (metaphorically speaking), watching dozens of hours of sermons on YouTube. Eventually, she requested a Book of Mormon to be dropped off at her house. I would know, I was the zitty 20-year-old missionary YouTube put on her doorstep to deliver it. Shortly thereafter, she got dunked in a baptismal font (not metaphorically speaking) and joined the LDS Church. On that day, she reported feeling “hopeful and free for the first time in a long time.” 

Jake escaped the grips of the same organization through YouTube. He had recently returned home from a mission to a far-off country and was watching the same “I’m a Mormon” videos. The system then recommended a new series: “I’m an Ex-Mormon.” Jake was sucked in—dozens of hours of videos were consumed. From there, Google directed him to various blogs where people questioned the tenets of the faith he had just spent two years preaching. After several years of questioning and doubting, he left the LDS church. I should know, Jake is my friend. When I asked him how he felt after leaving, he reported, “Hopeful and free for the first time in a long time.” Note: Both names have been changed to protect privacy. 

You may or may not like religion, but that is irrelevant. What matters is this: Did the AI recommendation do good? The emotional outcome was identical for the individuals. To the best of my knowledge, neither person regrets the choice they made. And, still, neither person would’ve made the change they did without YouTube’s recommendation engine surfacing just the right video at just the right time. 

The challenge is that “good” is stakeholder dependent. If you’re a devout Mormon, Jake’s choice was bad, potentially dooming his soul. If you’re a committed atheist, Megan was a fool, suckered into a cult. In either case, YouTube finds both outcomes good because the two consumed dozens of hours of ad-supported videos before making this decision. Other stakeholders—like society at large, content producers, governments, or advertisers—may have different perspectives on the relative good of YouTube’s AI-powered conversions. 

To further muddy the waters, how much good is even attributable to YouTube is debatable. Ask yourself: What percentage of these two individuals' actions can be credited to the information they received versus their own free will? To what degree do you believe in individual agency?

This isn’t some mere philosophical debate. Over one billion hours of video are consumed by YouTube’s users every day. Over 70% of the videos consumed are surfaced by algorithmic feeds. It is the second most visited website in the world. And the beating heart of its success is a recommendation engine. 

Recommendation engines, sometimes called recommendation algorithms, have been blamed for Trump’s election, Biden’s election, mass shootings, the proliferation of vegan restaurants, and TikTok’s dominance. The tech is responsible for you reading this very article. Whether Gmail put this in your “main” inbox, spam, or social tab, the destination was determined by some type of recommendation engine. They permeate e-commerce sites and travel websites. Anywhere there is more information than the eye can scan, algorithms are at work surfacing results. 

AI has made that math far more potent. The same scientific advances powering products like ChatGPT or autonomous vehicles are also telling you to watch the new trailer for a Marvel movie. These algorithms don’t operate in a vacuum. They are battling head to head, formula to formula, vying for dominance in the market for eyeballs. I wrote about this phenomenon last May, arguing that “addiction is becoming the blood sacrifice required of consumers to allow businesses to win.” In the war zone of the internet, the most time spent wins. 

In the meeting of these two concerns, of commercial interests and ethical conduct, the recent AI boom has me concerned. While we are all still debating the ethical implications of this technology, the algorithms keep getting better. Services are becoming more addicting and there isn’t much an individual can do about it. Tech companies will often defend recommendation engines by pointing out that when they are deployed, use time and customer ratings increase, thus proving that these algorithms are “good.” This is a circular argument—of course these things go up when the customer has access to them, that is what they are designed to do. 

It feels like everyone has an opinion on this tech. Some people believe that algorithms are terrible and cause harm, while others believe that they are morally neutral or even helpful in giving people what they want. However, in my opinion, this debate is far too black-and-white and oversimplified. Instead, it would be more productive to examine how algorithms impact our decision-making and culture by considering three rules that are often overlooked by one or both sides. 

1. Tech companies need to acknowledge that algorithms are deeply editorial, meaning they have the power to shape opinions and perspectives. 
2. Critics must recognize that design sets the boundaries of influence. 
3. Third, monetization sets the boundaries for what companies are incentivized to do with these algorithms. 

By examining these overlooked factors, we can have more nuanced discussions about the role of AI amplification in society.

Think of these rules as playing a similar role to what physics does during a game of basketball—gravity doesn’t determine who wins the game, but it does determine if the ball goes in the hoop. My three rules don’t determine which system is “good,” but they do allow us to understand how that goodness comes to be.

Now is the time to figure this out. Society stands in the center of an AI cage match. In the left-hand corner, are incredibly powerful AI recommendation engines, and in the right, are new Generative AI tools that are exponentially increasing the volume of content to filter. If we don’t get this right, the past few years of misinformation, election interference, and false panics will look quaint by comparison. 

Editorial algorithms 

As we discussed earlier, there is no universal "good" when it comes to recommendation engines because you have to balance stakeholder priorities. Editorial algorithms are one such way this phenomena manifests itself—the creators of a product get to determine what values they want to protect or dismantle in their app, by choosing what types of content and topics users will see. Technology isn’t created in an amoral vacuum. It is personal opinion forced upon the world. To see it in practice, we return to YouTube.

When the service was in its infancy, it had two simultaneous methods of recommending features. The first, a delightfully analog attempt, was a team of "cool hunters" whose job was to scour the website for good videos to feature on the homepage. This was paired with a simple algorithm that recommended related videos based on co-visitation; e.g., if you liked this video, another user liked one just like it. But even in those early days, there were editorial choices made with the algorithm’s design. For example, early recommendation experiments in the autos and vehicles category just surfaced a bunch of fetish videos of women’s feet revving engines in luxury cars. The choice was made by the cool hunter in charge of cars to designate this as “bad.” While a group of rather horny dudes who loved cars and feet might disagree with that call, the broader population was probably pleased. Around the same time, another tweak to the system accidentally made the related sections full of ”boobs and asses basically.”