Dear Analyst #118: Uncovering trends and insights behind Facebook News Feed, Reels, and Recommendations using data science with Akos Lada

No, this isn’t an episode about how Facebook’s algorithm and feed works. The data science function is popping up in companies small and large given the amount of data swimming around. No other company understand the power and influence that data science can have on the customer experience than Facebook (Meta, to be exact). Akos Lada is Facebook’s Director of Data Science for Feed Ranking and Recommendations. Akos has always been interested in the intersection of social science and data, so this role at Facebook seems fitting. In this episode, Akos discusses what the analytics team does at Facebook, an analytics framework his team developed and open-sourced, A/B testing, and more.

What does the data science team do at Facebook?

I know the company is called Meta, but I grew up calling it Facebook, so I’m just going to stick with Facebook for now. The data science team actually consists of two teams: Analytics and Core Applied Data Science.

The Analytics team partners with product managers and engineers and their focus is on delivering long-term value for users (you’ll hear a lot about this during this episode). There is also another data science team Akos used to work on, called Central Applied Science (formerly known as Core Data Science), which is a smaller team that focuses on scientific problems and research that every product team at Facebook might be able to benefit from. One of the frameworks the Central Applied science team created and open-sourced is called Ax. This framework helps optimize any kind of experiment including machine learning experiments, A/B tests, and simulations.

Making better decisions with the GTMF model

Akos’ team published a blog post on four analytics best practices at Facebook which is worth a read. The impetus for this blog post was one question: how does Facebook drive more long-term value for users?

There are many different lenses you can put on to answer this question. Of course, Akos’ team treats this question as a data science question. The Ground Truth Maturity Framework (GTMF) improves ground truth data–the data that powers Facebook’s machine learning models. In a sense, the GTMF model ensures your data is clean. One place where GTMF is used is News Feed ranking. The team’s ultimate goal with News Feed is trying to figure out if a post is something you would want to click on. You can read more about how machine learning is used in the News Feed algorithm here.

Running A/B experiments to figure out the right number of notifications to send to Facebook users

Akos discusses at length his team’s experimentation frameworks. One interesting insight is that the longer his team kept experiments running (say one year) then the outcome of the experiment would change. One of the more surprising results from a long-term experiment his team ran was that if you send less notifications to users, it led to better long-term value for users (e.g. clicking on more posts). In the short-term, sending less notifications would naturally lead to less people engaging with posts.

At the end of the day, this is a behavioral science challenge. Given the amount of data Akos’ team can analyze, they suggested that the product team drastically reduce the number of notifications being sent to Facebook users. You can read more about this experiment and the results here on the Facebook Analytics team’s blog.

While the data science team has so much data at their disposal to make data-driven decisions, Akos talks a bit about how the team also uses intuition for making decisions as well. In an organization as large as Facebook, you can run multiple experiments at a time, evaluate the results, and then ensure the knowledge and insights are spread between product teams. While the results from an experiment on News Feed may not necessarily apply to other product teams, other products at Facebook like Instagram and WhatsApp can benefit from the institutional knowledge.

What the future holds for data science at Facebook

There is a saying at Facebook that the work is only 1% done. Akos talks about how the data science field in general is a relatively new field that really began in the last decade. Compared to other fields like economics, data science is still in its infancy.

Akos’ team is investing more time in machine learning systems, neural networks, reinforcement learning, and all the new and sexy data science topics you’ve been reading about in the last few years. Akos’ interests in data science goes beyond Facebook as he’s published academic papers such as this one about heterogenous causal effects. Akos talks about his fascination with how activity can change when nodes are connected to each other (referring to Facebook’s social graph). If someone sees a post and they find it interesting, they will share that post with their friends. Then those friends share that same post with their friends. Given the connected nature of the social graph, how can Akos’ team help suggest posts that you might like? Facebook’s Recommendation system is built on this concept called collaborative filtering.

Advice for aspiring data scientists

It seems like a tradition now to ask people on the podcast about advice they have for upcoming data analysts, engineers, and scientists. Akos’ advice was a bit sobering but exactly what aspiring data scientists should keep in mind as they find their next role. It’s a tough time in the tech world, but don’t be discouraged. Akos believe that despite the downturn, data science will continue to grow as technology becomes ever more prevalent in our lives. Now is the time to double-down on building your skills. One of the reasons Facebook has their Analytics blog is to share their insights with the community in the hopes that data scientists can build off of Facebook’s work. Akos talks a bit about the generative AI trend, but he’s still focused on how regular “generic” AI can still help people around the world.