Dear Analyst #89: Leading high performing data teams and deciphering the data stack with David Jayatillake

Most episodes I have the privilege of speaking with analysts who are in the trenches using tools and doing analyses. In this episode, we look at the role of data from a manager/director of data’s perspective. David Jayatillake is currently the Chief Product & Strategy Officer at Avora, an augmented analytics solution that helps companies make better decision. Not to say David doesn’t get into the weeds when he has to (his GitHub profile tells that story), but it was interesting hearing how to build high performing data teams. We get into topics relating to data strategy, bundling and unbundling of data tools, and ways to be an effective manager.

Moving from data to product

For most of David’s career, he’s led business intelligence and analytics teams in various industries including retail, credit, and e-commerce. That’s why I thought it was interesting when he moved into a product role at Avora.

As a leader of data teams, David was using various data tools to analyze, store, and transform data. Over time, you might start to see some of the deficiencies in these tools, and even have ideas for new product features. As an end user of some of these tools in my organization, I’m quite familiar with the little quirks of these tools that just make you shrug a little bit when you have to open the tool up in your browser.

David had been advising Avora on building various product features before moving full-time into the Chief Product Officer role at the company. In terms of transitioning to a product role, David had various PMs he could observe to learn the rituals of leading a product team like leading a standup and creating a product roadmap.

The great bundling debate

A few months ago, David wrote this blog post about the bundling and unbundling of tools for the data engineering profession. For those new to this world in data engineering (like me), this analogy sums it up nicely from the Analytics Engineering Roundup:

Unbundling means taking a platform that aggregates many different verticals, such as Craigslist, and splitting off one or more services into stand-alone businesses that are more focused on the experience of that niche customer base. Sometimes, as was the case with Airbnb, that experience is so much better it eclipses the market cap of the original business.
Anna Filippova, Analytics Engineering Roundup

Taking this concept to the modern data stack, David’s blog post discusses the unbundling of Airflow into many off-the-shelf and open-source tools. I’ve never worked in a data engineering role and was worried that this part of the discussion would go way over my head. I hear about ETL, streaming, and automation from being in the space but have only lightly touched some of these tools in this data stack:

David’s blog post also discusses the role the data engineer plays in a world of unbundled software:

For most companies, the cost of hiring and keeping data engineers working on pipelines that could be built with specialist “unbundled” tooling is unwarranted. The real value is in the next steps of data-enabled decision making, whether human or automated: Is this sneaker actually the same as this other sneaker? Should we lend to this customer? Should we deploy this product feature? Should we spend more on this marketing channel than the next? The curation of clean incrementally loaded data into well-defined data models and feature stores, to enable this decision making, is still a key value of analytics engineering that is not easily automated.

Cloud vendors to the rescue?

David recently spoke on the Analytics Engineering podcast with Benn Stancil, founder at Mode (listen to episode #71 with Benn) about the bundling vs. unbundling debate. He brought up companies like Oracle which were these all-in-one platforms for enterprises that weren’t necessarily that great for analytics teams.

Public cloud companies like AWS, Azure, and Google Cloud are kind of becoming these all-in-one platforms that are interacting with different parts of the data stack. These cloud vendors have data warehouse solutions that solve the most common use cases for companies, but they are still missing marketing leading data tools around ETL, data transformation, and business intelligence. For instance, if you’ve ever used AWS QuickSight or Google Data Studio, they functionally get the job done but are not best in class, according to David. Data teams have to adapt to changes in their businesses so newer needs around observability and data cataloguing have emerged with complementing tools to serve these needs.

As I mentioned earlier, I’m a neophyte to this world of data engineering tools. Like many of you, I’m part of the group that utilizes the tools built by data engineers to get our work done. To learn more about this big and beautiful world of data engineering and data pipelines, I’d recommend listening to episode 34 with Priyanka Somrah of Work-Bench (she also runs a great newsletter called The Data Source) and episode 58 with Krisha Naidu, a data engineer at Canva.

Unbundling the manager

We moved from data tools to David’s ideas on how to be a great manager. Consider this structure which is common at most hierarchical organizations: The line manager acts as the project manager to figure out what work needs to be done by the team. These managers also are also responsible for the career development and mentoring of their direct reports.

These days, many organizations have already stripped the “project manager” role from the line manager’s responsibilities. The product manager, for instance, is in charge of product prioritization. People managers are still responsible for career coaching and mentorship. David sees value in splitting these responsibilities apart so that you have people who specialize in your career development and people who specialize in mentorship:

It allows individual contributors who don’t want to do career coaching to invest in other colleagues via mentoring. They often do this regardless, but it recognises the value of it and also allows access to more of these individual contributors for mentoring roles. It allows career coaching to be taken on by non-technical staff when engineering resource is scarce and expensive. Often HR specialists could do this role really well, as they are trained in how employee life cycle processes should happen: 360 feedback, career ladders, pay review, promotion, probation, redundancy, learning… it’s a long list!

Specialized data analyst roles

We spoke about the rise of the analytics engineer role in the last few years and what future roles today’s data and business analysts should consider. From David’s perspective, analysts are becoming more specialized. When I first started as a financial analyst, just knowing Excel meant you were “good enough” to be an analyst in HR, sales, and other departments.

Today, you need to understand the tools and business logic for these different departments. That’s why you’ll see a specific skill sets required for HR data analysts who can crank through a river of employee data and for sales analysts who know how to work with Salesforce and help set quotas for a sales team.

Data analyst career path (Source: CareerFoundry)

During David’s time at at Lyst, there were teams focused on building out Lyst’s checkout product. There were data analytics engineers focused on serving just this specific product at Lyst. You have product analysts who know how to use tools like Amplitude and Mixpanel, and then analysts on the marketing side who specialize in using Google’s various ad tools.

Learning through writing

David starting writing his newsletter after Benn Stancil inspired a group of people to start writing at the end of 2021. David’s newsletter was originally going to be about career transitions as he moved from organization to organization. He realized he has benefited greatly from being in the data space, and his newsletter is a way to share his data leadership experience with the broader community. Like him, I believe publishing a newsletter or creating a podcast helps with organizing your thoughts and learning about new and unfamiliar topics (like this episode clearly shows). Check out David’s response to this post about people with a side business: