What Is Wrong about Recommendation System

Update: check this article to see how to use a RSS reader I wrote to take back the control of your information consumption.

I remember when I was in high school, the courses were so boring that I was always trying to find some other things to read on class. It was very hard because smart phones were very expensive. And we don’t even have a good library in our city. But just more than 10 years later, it’s a totally different world. The information is so easy to get and so overwhelming that people can easily get lost. So I’m trying to use mobile phone and social network platforms less often since 2 years ago. Recently, I feel I’ve made some progress. I want to share some of my experience by writing a series of articles (hopefully :)). This time, I want to talk about one of the reasons that I want to involve less with the digital world: the recommendation system.

When I started my software engineer job as an intern, I worked on a recommendation system. At first, I was so fascinated that the computer can learn people’s interest and recommend things for them. It’s a kind of machine learning system. The way it works is you define a goal and let the program to reach it. That’s also how you evaluate whether the recommendation system is good enough. Usually the goal is to find the content that user will click. There are good reasons to choose such a goal: For ads platform, a click on an ads means money from advertisers. For social network platform, a click means user is spending more time on the platform. For retailer website, a click means the user is more likely to buy one more item. Basically, for a commercial company, the goal is how it can earn more money from you.

In my case, I was also training the system to filter out content that users most likely to click. But after I trained the system, I found it recommended lots of erotic contents. That’s when I realized something is wrong. Logically, the program is right: users are likely to click on the erotic content. That’s when I felt something bad may happen if we rely purely on algorithms.

Not long after that, my worry became true. Recommendation systems became more popular and more powerful. Companies are founded based on the idea of recommendation algorithms. The most successful one is ByteDance, the company behind Tiktok. It created an app called Jin Ri Tou Tiao (which means Today’s Headlines in English) to recommend news based on algorithm. The news are usually low quality, often has erotic content, exaggerated title and partial fact. It’s not good news, but combined with its aggressive notification system, people just cannot stop reading them.

The reason behind this is a simple fact: what the brain wants may not be what the body really needs. Human beings have evolved for a long time to fit the environment, but the environment has changed so much and so fast in recent years because of the technology, and our brain just cannot catch up. That’s why overweight is so normal in developed countries: it makes sense to eat food that has high calorie at old days when food was not guaranteed. The brain never had a chance to evolve in an environment that calorie is more than needed. Another extreme example is the drug: it’s very bad for healthy but it’s so hard to get rid of it once you are addicted.

Maybe recommendation system is better than drug, but it’s definitely worse than high-calorie food. It uses power of computer and the information collected from you. It’s brain hacking with modern technology. Not only our body hasn’t evolved to fit it, our society is also falling behind. There is no regulation about that. It’s basically wild west. The companies are doing whatever they want in this field to meet their interests.

Indeed, as human beings, the brain has higher function that can sometimes recognize what’s bad even it looks temping at first. But that function needs lots of energy and focus, it’s nearly impossible to recognize all of them if you get the information all day. And the energy and focus has limit. Once you spent too much energy to resist the temping recommended content, the less energy you have for real important tasks.

After you addicted to the recommended contents, the best case is you will waste a lot of time. You may also feel anxiety because watching the contents doesn’t really fulfill your needs. The more anxiety you are, the harder to stop. And worst case, you may get manipulated by other people. Cambridge Analytica is a very good example.

That’s why I’m very alert to black-box recommendation systems. And that’s why I was very disappointed when Twitter order the timeline based on recommendation. I also had bad time when watching recommended videos and news endlessly that even lost sleep. The solution is simple: just leave them. It’s easy to say but very hard to do. I’ll share some of my tactics in the future.