Podcast: Play in new window | Download

Subscribe: Spotify | TuneIn | RSS

One of the best feelings is knowing you’ve streamlined and automated a report such that your colleague doesn’t have to spend hours creating the report every week. Whether you automate the report through a bunch of Excel formulas or setting up some sort of data pipeline, analysts are always thinking about driving internal efficiencies. This was one of my first jobs as an analyst, and also what Nadja Jury, a data scientist at Education Perfect, is doing at her company. In this episode, Nadja discusses how she went about automating weekly reports for the head of customer support, communicating data with internal stakeholders, and setting up data definitions so the whole company is on the same page (spoiler: it involves “ski passes”).

From customer support to backend junior developer

Education Perfect supports students in an online learning environment by providing assessment and collection of student feedback for teachers. Nadja started in customer support, then moved to the enrollment team, and ultimately became a backend junior developer. You don’t come across many people who take that kind of career trajectory, so I asked Nadja how she found herself as a backend developer:

At university, I studied computer science and six months before I graduated, the team pulled me aside to see what I wanted to do. I was interested in an internal data role.

Starting in customer support most likely gave Nadja a lot more context on what types of issues Education Perfect’s customers care about as she moved to a more data-focused role. Sometimes it can be easy to “detach” from the front lines of the business when you’re deep in spreadsheets and databases all day.

Education Perfect didn’t always have a dedicated data person. In September 2019, a product manager laid out plans for a data team which now consists of 4 people who help the company make sense of all the data they have on how students and teachers are using the product.

Weekly reports that don’t suck

One of Nadja’s projects involves automating the reports for the customer support team. Efficiency is already top of mind for the CS team since they aim for a 10-minuted turnaround time on answering tickets.

The manager of the Inbound team used to take data coming from their email system to build end-of-week reports. These reports include tracking against the CS team’s KPIs, data coming from their email system, and more. The end result? Multiple spreadsheets that need to be copied and pasted in multiple places (who hasn’t been there before?).

So what is the new workflow that Nadja came up with? The head of the Inbound team just needs to export the data and and drop the files into a singleGoogle Drive. There is a Fivetran integration with Google Drive that automatically connects the data from that folder with the company’s main data warehouse. Nadja then set up dashboards in Mode that built off of the data warehouse that generate all the necessary reports for the CS team to show how they are tracking towards their KPIs. We were already using Fivetran so it was easy to start the integration with Google Drive.

Time saved? Typically this would take the head of customer support 2-4 hours every Friday morning to build these reports, and now it takes less then 30 minutes.

What’s interesting about Nadja’s process with designing the Mode dashboard is that she first saw what the head of customer support had created in Google Sheets. Nadja then drew some new charts in a paper notebook and met with the head of customer support to see if these were the charts that would show what the CS team needs to report every week. Instead of focusing on the numbers and trends, this strategy allows both the analyst and the stakeholder to focus on the charts and see if they would tell the right story.

Helping designers make better decisions about content

Another report that Nadja and her colleague are working on is for the company’s designers. This report shows how the educational content is actually being used by students. With over 1TB in the main questions table, there is a ton of data to analyze. Since students on the platform are submitting answers to questions, the data team can look at every attempt on every question in every lesson and pull trends and insights to give to the designers. These trends can then help determine what type of content needs to be tweaked to ensure students have a good experience on the platform.

Working effectively with internal stakeholders on data definitions

Given how new the data team is at Education Perfect, most of 2020 was spent identifying the gaps and potential confusion around existing reporting. Thus far, the data team has been meeting with their colleagues to find the best solutions to their data and reporting needs.

Nadja told an interesting story highlighting how even the definitions of certain metrics and data points can be a bit ambiguous. One of her colleagues asked Nadja a question about some metric, but Nadja happened to be on her lunch break. The colleague then sent the same question to another member on the data team. When Nadja returned to her desk and responded to her colleague’s question, she realized the answer she gave and the one her data teammate gave were completely different.

I imagine this problem is prevalent even in organizations with mature data teams. Ask yourself, what does an active user mean for your company? For Nadja and Education Perfect, this could be referring to teachers, students, or some other user type. What does “active” really mean?

This led to Nadja spearheading a “data definition doc” where the entire company’s metrics are listed in one doc for everyone to see. This doc creates transparency for the organization, and also ensures new teammates have one place to look to understand the common “data language” spoken across the company.

Inventing new metrics based on your company culture

One specific metric that Nadja and the data team had difficulty defining was that of a student accessing a subject in the curriculum. Given the current data definition doc, there were no metrics that her team could use to describe this activity. They researched a bunch of SaaS metrics but still couldn’t find something that resonated with internal stakeholders.

Turns out many of Nadja’s colleagues are based in the South Island (New Zealand) where skiing is popular in the winter. Since everyone likes to ski, the data team called this metric a “ski pass.” When they introduced this metrics to the team, it just clicked with all internal stakeholders. Nothing like some shared cultural context to inform how your data should be labeled. “Ski pass” is literally the name of a column in a table in their data warehouse.

Machine learning and Python

I asked Nadja what tools she’s interested in learning next, and she is currently exploring machine learning and getting data into a Python notebook. She took a 10-week course last year and is hoping to apply these skills to the growing data team at Education Perfect. Additionally, she is learning some of the custom features of dbt for transforming data in their data warehouse. It’s always great to see analysts going beyond spreadsheets and formulas and learning about all aspects of the data pipeline.