[ Interview ] Avo Co-Founder Stefanía Ólafsdóttir on Addressing Emerging Needs in Product Analytics

A few weeks back, I spoke to Stefanía Ólafsdóttir for The Lean B2B Podcast. We talked about entrepreneurship, data science, product analytics, customer research, and addressing emerging needs.

You can watch the full interview below, or access it on iTunes, Google, or Spotify.

Interview Transcript

Etienne Garbugli: My guest today is Stefanía Ólafsdóttir. Stefanía is the CEO and co-founder of Avo, the next-generation data governance platform, helping product managers, developers, and data teams plan, track, and govern their product analytics. Prior to co-founding Avo, Stefanía studied Abstract Maths and Statistics, and she was in charge of data science then revenue management for QuizUp, a trivia game that had over a hundred million players.

Stefanía, welcome to the podcast.

Stefanía Ólafsdóttir: Thank you, excited to be here. Thank you for having me.

Etienne Garbugli: Great. Maybe we can start at the beginning. So, you studied Abstract Maths and Statistics at the University of Iceland. What initially attracted you to this universe and what were your initial plans for your career?

Stefanía Ólafsdóttir: Starting with the easy questions.

Etienne Garbugli: Yeah, always.

Stefanía Ólafsdóttir: I guess, to make it fully accurate, I actually started Mathematics and Philosophy, and that’s also pretty abstract. My favorite subjects when I was a kid in school were mathematics and… I don’t know what to call it in English. It’s like everything around sentence structures and systems and grammar, the Icelandic grammar in the Icelandic language. Those were my two favorite subjects.

And I think that sort of speaks a little bit to my appreciation of systems and wanting to understand why things are the way they are and sort of liking when things have a system and a reason for why they are the way they are. And then when I was in high school, I guess you would call it, I ended up focusing on philosophy. And that was a little bit coincidental because I took a philosophy course and I really liked the teacher, basically. Or at least in my second philosophy course, I really liked the teacher, and the first one I guess I loved the thinking about everything part.

Then I decided to study philosophy, but when I was a kid, I thought I would probably study something like engineering. My dad is an engineer as well, an industrial engineer. Everything is so coincidental. The Icelandic University does this thing called the university day where, when you’re still in high school, you can go and check out some booths and learn a little bit more about what you might want to study.

And I was going to check out the philosophy booth, and dad was like, “Stef, you should check out the engineering section. Maybe you want to study engineering.” And I was like, “Okay, fine. Sure. I’ll do that.” And I walked over there and asked her, “What the hell is the difference between industrial engineering, electrical engineering, and all of these different engineering paths?” And I just asked the kids there. They were studying it. I was just like, “I wouldn’t even know where to start. What type of engineering should I choose?”

And they really couldn’t really answer the question. It was like, “Yeah, it’s pretty similar. I don’t really know. Do you know?” And they were just asking each other. So I sort of gave that up and I was like, “No, I won’t study that.” On my way out after that, I literally stumbled upon the mathematics booth and it was like, “Oh yeah, you can study mathematics at the university. That would be interesting.” And coincidentally, the person that was introducing math was in the same high school a couple of years ahead of me. So he told me a little bit about it and I ended up studying both. That’s a coincidence.

Etienne Garbugli: And what was the plan and then? What were you thinking that your adult life was going to be?

Stefanía Ólafsdóttir: Yeah, that’s a great question. I was literally talking about this with a few people last week on a panel about identity, what you identify as, and all those things. And there are these people that have these really strong opinions or beliefs about what they want to do and what they want to become when they grow up. I was not one of those people. That was not a part of my identity knowing really strongly what I wanted to focus on.

So I think from my perspective, I’ve always been a very curious person. And Mathematics and philosophy, just studying that seemed fun. And that was it at that point in time. When I was in university, there was a lot of pull from the banking and the financial industry for mathematicians. They seek out mathematicians and try to get them to work in banks and financial institutions and stuff like that. And I had no interest in that; just absolutely no interest in it.

I have always been very fond of participating in education and knowledge sharing. And particularly, the upbringing of children independent of their parents. I think that’s a really huge social system, government issue type of thing. So I was thinking about that type of thing.

I was also considering starting a residency/hostel type of thing in Iceland for bands that would come to Iceland and play music. So it was pretty diverse what I was considering doing. At least it was not clearly data science. That was not what I was targeting or anything like that. Life is full of coincidences and you end up doing what you’re doing.

Etienne Garbugli: Absolutely. So, I’m getting that you like the depth, you like curiosity, and you didn’t want to be in the banking industry, or that aspect didn’t attract you. So what initially attracted you to QuizUp? What did you know about startups or working for a startup at that point?

Stefanía Ólafsdóttir: Yeah, that’s a great question. When I finished my university studies of mathematics and philosophy, well, first I ended up going and traveling around Australia for half a year. That was really great. Then from there, when it was a couple of months until I was going home, I was like, “Okay. I should maybe start thinking about what I should do.”

And I was thinking, is there anything related to philosophy or mathematics that one does as a career? And that’s a really vague thing. It’s not like you study to become a dentist and then you just become a dentist. So what I looked towards at that point in time, just to stop there on the way to QuizUp, why I was attracted to QuizUp, I ended up looking at a little bit more of the more established “startups” in Iceland that were, at some point, startups.

They were creating something cool and new like a company for prosthetic limbs and the company for innovation in the food industry. And then there was this company that was innovating a lot in genetics research called deCODE Genetics.

I applied with these companies that were sort of creating something cool. They had then been around for anywhere between 10 and 15 or up to maybe 20, 30 years. So no longer a startup, but still creating something really cool. Innovation definitely—that attracted me a little bit.

And I got job offers from these companies and I ended up going into the genetics industry or joining deCODE Genetics as a research associate. So I joined the statistics department and I was working with huge datasets within deCODE, managing a lot of data engineering type of things that weren’t really called data engineering. It was like doing distributed computing, but I was also helping doctors and biologists make sense of the data to find correlations between physical DNA mutations and physical traits.

So I’d been working with huge data sets and learning a lot there. I learned how to program at deCODE although I would probably never qualify as a software engineer. But I learned what I needed to do to do my job there. I got introduced to Python and R and C++ and Bash managing all sorts of stuff like that.

And that’s when a mobile game called QuizUp blew up and reached a million users in their first five days, which was the fastest-growing app at that time. And that was a record that was held for a while until another mobile game called Flappy Bird, which I’m sure some folks that are listening remember that as well. They later beat that record.

I knew some of the people at QuizUp personally and they were a friendly bunch of people. I had been hanging out with them, grabbing beers and stuff like that just as friends. Then they were looking for someone to join as the first analyst to make sense of where these million people are coming from, why are they coming, and how this is growing so fast? Why are some of the people coming again and again, and again? And why do some of them not return? And what’s the difference between their experience or their traits?

It was so interesting because QuizUp created a lot of value for its users. It was a mobile trivia game where people would compete and we had thousands of topics or categories of quizzes, anywhere from general knowledge to the Pokemons or identify the flag, or something really specific like that. And there were growing communities within all of those; people really bonded over being obsessed over knowledge.

So when they asked me to join, I immediately identified that there were huge opportunities. The dataset behind this is really exciting. We can do a lot of cool data science-y stuff with all of this data and build recommendation algorithms and stuff like that. But also I already realized then how big of an opportunity this was in building something from scratch, building an analytics team or a data team from scratch. And I felt that was a really exciting challenge that I felt like it was a fun next move.

And a quick side story into that, before I actually joined QuizUp—this was late 2013 or something like that—mid-2013, I had actually started a PhD track. So I dropped out of that to join the QuizUp. I was just like, “That PhD can wait. I’m ready to build something really cool. This is a rocket ship and I want to join. I don’t want to miss this opportunity.”

So I think the people behind QuizUp and the opportunities for me to build a team and the opportunities to do something really cool with the data and work with really great people, all of those things sort of made it compelling for me.

Etienne Garbugli: Okay. So the rocket ship is taking off, there are already millions of users. What are some of the challenges that you faced or your team faced at QuizUp when it was blowing up internationally?

Stefanía Ólafsdóttir: Great question. Obviously, there were a lot of challenges and just building the right things. All of a sudden, you have something that just blows up like that. Then the next step is like, “Okay, can we help it continue to blow up?” And ensure the product-market fit, that’s, of course, an important one, and then it’s like, “How do we monetize this? How do we make this a sustainable company?”

And then I think the third pillar for these first two pillars to work is how do we build a team and scale the team to make sure we can find these things, find the product-market fit, and find the revenue model and make this a sustainable company?

So I think the challenges very early in a company like that that goes through this rapid scaling and user base, it typically means you have to also scale up the team behind it. Like I was talking about, we had thousands of categories and growing endlessly. When I joined, I think it was somewhere around 2000 topics. Behind that was a huge editorial team as well.

So there were a lot of challenges just maintaining fresh content as well. But to sum it up, some of the first challenges building something like this is making sure you hire the right people to do the right jobs and focus on the right things. And that’s a huge challenge, especially when things grow this fast to keep everyone aligned.

Etienne Garbugli: And from the data side, the data kept coming, the data set kept growing. You had to scale the way you learn from customers or scale the way you guys were able to collaborate on new features. What were some of the challenges that came up on that front?

Stefanía Ólafsdóttir: That’s a great question. The data team journey is really interesting and maps very much onto what I’m doing today. Today, I’m the CEO and co-founder of Avo, which you mentioned is an analytics governance platform, and it’s built for product organizations and for product managers and data scientists and product engineers to collaborate and make sure we’re measuring the product releases accurately and able to leverage that data to make decisions.

And that is very much based on the data team learnings over at QuizUp. We started off as a data team of one. That was me. And then the problem, basically, from there was we needed to answer more questions faster. Everyone was trying to make decisions on their day-to-day job, like what to build next, what topic to release next, how did this feature roll out? What’s the success of that? Should we focus anymore on that? Do we have any product innovation opportunities here? Is there anything hidden in the product that indicates that if we focus on this area, then XYZ?

So identifying all of those opportunities was a lot of work because we didn’t have all of the data we needed, first of all. But even for the data that we had, it was pretty disjointed. So, as a data team of one, I spent my time hacking together a bunch of data sets. Like we had a Mixpanel, we had the operational database data that literally runs the app. We had Flurry, we had Google analytics, we had the app store analytics and just all of those different data sets.

Yeah, it was a lot of work to pull the data together so that anyone could make any decision from it. And it could only be done by someone who could hack together datasets. So you had to have some engineering capabilities.

To combat that, we hired more data people first and scaled the data team a little bit. But it still kept growing as a problem. We still weren’t answering questions fast enough. So the next step in the journey, stage two from a centralized data team that was able to answer questions, is to unblock that bottleneck.

The bottleneck is decision-making is bottlenecked by the human throughput of data scientists being able to hack together data sets and answer questions. So we started to move further towards what is often called self-serve analytics. It’s a hot topic. A lot of people just don’t believe in it as a concept but I am a strong proponent of it. But it has its caveats. We still need analysts to obsess over the legitimacy of results and things like that.

But self-serve analytics at its core means that people can look up the data they need for their day-to-day job as they need it—some of the KPIs for their teams and things like that—and are able to evaluate the success of a release or the success of a campaign or something like that.

They might have to get an analyst or a data expert to jump in to validate their assumptions or look into whether the data is correct, just to confirm and all those things but the goal is to help people be able to be a little bit more self-sustainable in answering these questions, at least the early indicators of the results or the answers, and then go to an expert to look into things like, what are some of the follow-up questions we need to ask before we can actually make this assumption and things like that.

But the challenge there, even when we were at stage two, self-serve analytics, we still had a major challenge around data quality. Decision-making was bottlenecked by lack of data, quality, and lack of data literacy, I would say. And so we entered stage three where we created analytics governance.

So the data experts would support the product teams in making sure they were tracking the right metrics for the releases and structuring the data correctly and logging everything correctly that they needed to log to be able to look into the metrics. And then we basically created another bottleneck in stage three, which is product releases were bottlenecked by analytics governance.

So for every single product release, the product teams had to go through the data team and get confirmation that they would have the right metrics and all those things. And then the fourth stage was something closer to self-serve analytics governance to support self-serve analytics. Over these three, four years, we built all of these tools and processes to enable self-serve analytics and enable self-serve analytics governance.

And all of those tools took a lot of effort to build internally. But it was a dream state to be towards that end before QuizUp was then acquired. So, a long, strong journey of challenges.

Etienne Garbugli: Do you remember the first time where you had the idea of there should be a proud product there, there could be a product in the work that we’re doing, and there should be something in this that could become a startup?

Stefanía Ólafsdóttir: That’s a great question. From our perspective, what we did is we wanted to optimize this problem from the data quality perspective, but also from the product developer perspective because, ultimately, it’s the product developers that have to write code, to implement analytics events, to send them into the database so that the product team is able to answer the questions that they need.

We thought about it. I think this was super early product analytics. We were early adopters for product analytics. Web analytics has been around for a while, but all of this changed into the mobile analytics space and changed from page views and to analytics events. That was a huge shift that was going on. At that point, there weren’t even a lot of resources online about how to define retention? How do you define conversion funnels? How do you measure things successfully?

So I think we were that early in that journey that I think, from my perspective at least, my first thought was not Googling a product for how to solve it like what processes exist. Our perspective was definitely that we need to bring the stakeholders together that have this problem and figure out what is the solution for it.

I think immediately there when we started developing these products internally, data products basically, I think that was that trigger, even though, at that point, we didn’t necessarily see it as product development. But effectively, we were doing internal product development of internal tools.

Etienne Garbugli: Later, when you left QuizUp, what convinced you that this was an important thing worth pursuing?

Stefanía Ólafsdóttir: That’s a great question. Side note: when I left QuizUp, this was a huge, huge learning opportunity and I learned so much from this journey. But like so many journeys where you learn a lot, it was also a lot of challenges. So it was challenging as well, which is exactly what the best learning opportunities are.

And I was burned out from the data space after this. There was a company that acquired QuizUp and they offered me a job as a data scientist there. And a lot of folks were reaching out also to ask me to join us as a data scientist or a data leader somewhere. And there was a fun time to explore those opportunities, but ultimately, I just really felt deep inside, I was like, “No, I’m not doing anything more in the data space.” That was my feeling.

It’s just such a mess and I envisioned that every time I would have to join a company or every time I would join the team doing this, we would have to just solve all these challenges again and again and again. So I was originally like, “I’m not doing more data.”

Then I started a company with a couple of friends from QuizUp. That was not Avo; it was the predecessor of Avo, not related to Avo at all. It was gamified microlearning for employees. It was a B2B SaaS. The thing there was that it was in our fifth month that we shipped a product update to our customers that was based on incorrect data. And it was so painful to do it. And furthermore, it was so painful to realize that this would just continue to happen as long as I am a part of any product development team, anywhere in the world.

And I think that was the seed for like, “What are people doing to solve this? Not everyone is building all these tools internally. Could that be?” So that was what was going through our minds.

Etienne Garbugli: So the problem was following you?

Stefanía Ólafsdóttir: The problem was following me and it was just so painful. Fast forward a few more months of building that product and that company, and we went through a realization that gamified micro-learning for employees wasn’t necessarily our passion. We were doing it more as a business opportunity type of thing, which is a great reason to do things, but we also had the understanding that when you’re building a startup, you might be in it for 10 years. So you want it to be something that you’re passionate about.

So we learned a lot on that journey. And having gone through this again with all of the data quality issues and realizing that we don’t have the resources to build all these tools again internally, that triggered us to start talking to hundreds of our colleagues in the data space and understanding what are people doing to solve this?

And ultimately, our learning was that some of the biggest organizations like Spotify and Airbnb, and Twitch had built internal tools for this. But the others were really just scrambling. I think that learning with our passion for really solving this problem was enough to push us into, “Let’s build the solution to this problem.”

Etienne Garbugli: So you knew there was a problem, emerging needs, you had solved it or your team had solved it at QuizUp, but you also knew that other companies had found a way to solve it. So between all this, there should be a way to be able to apply the solution to different organizations.

Stefanía Ólafsdóttir: Yes, exactly.

Etienne Garbugli: What convinced the first company that decided to adopt Avo?

Stefanía Ólafsdóttir: I think it’s a classic tale of your network. As a B2B founder, or as any type of founder, your first job is to find people that will use or try your product and ideally pay for it at some point.

So the first thing is to keep hacking away at your network until you find people. Some of the first users of Avo were the people that worked with us at QuizUp. They were used to having a solution like that in place at the previous workplace. And now they had all gone on to other workplaces and it was a pain. So they were like, “You’re solving this!? Take my money.”

That was a good initial step going to the people that had seen a solution to this executed well before and seeing us that had already executed this solution before do that for them again.

Etienne Garbugli: There was trust.

Stefanía Ólafsdóttir: Yeah, there was trust. And then it’s about continuing with the customer discovery journey. We’d talked to a lot of people before we decided to go into this. So we just kept on talking to those. In some cases they had already had all of these investments internally, so they’re not going to rip and replace that. But we learned a lot from those conversations and they could point us in the next direction and the next direction. It’s just crunching away?

Etienne Garbugli: How wide was the gap between the initial vision and what your team was selling initially? How did your team go about bridging that gap?

Stefanía Ólafsdóttir: When we started Avo, we were very clear on wanting to build a very customer-first organization. We were also really clear that we wanted to build with our customers and not fall into the trap of that so often as preached, particularly with the lean concept—don’t just build somewhere in a hole and then release it and hope they come or something. But actually learn rapidly from whoever you’re building for and building with.

So we took that very seriously. In the previous question, I was talking about how we got our first customers and we got our first customers to try out Avo when it was just a hack job.

Etienne Garbugli: Okay. There was a product but it was not up to par up to anywhere where you guys were up?

Stefanía Ólafsdóttir: Yeah, absolutely not. We had people using the product. What do they say? If you are not ashamed of your product when you’re releasing it for the first time, you’re releasing too late. And we definitely follow that religiously.

So we had people using Avo very early on and we had a large vision for what we wanted to solve. We really wanted to help companies build reliable data. And reliable doesn’t only mean just data quality; it also means that it has to be relevant data. So a lot of that vision doesn’t only include data reliability and confirming that the data structures actually work as expected. It also includes a lot of helping people design good data and helping people decide what they should actually measure and track.

So our step-by-step was, “We want to help people design good data. We want to help people measure the right things and be thoughtful in how they are strategizing their product and are able to use data to drive that vision.” And for that, we have to help people design good data.

What we also knew was that even if people designed good data that they want to measure, it won’t mean anything if it’s not reliable, if the data isn’t correct, or if the data is broken all the time. So we started off really raw. The first thing we released was about data design. Then we realized that the first thing we have to solve is the foundation of the data quality.

So we started off with data quality and it was only for data quality purposes and making sure the implementation of the tracking code was correct. And we’re still working towards a large vision of where we want to be. There’s so much still to be done to reach that goal, even though we’re already solving so many problems for our customers already today.

So our ultimate drive and our strategy for this is okay, we have a vision for where we want to get our customers. But everything that we release to them, in an intuitive way, has to also solve a specific problem along the way. That’s how we gradually built that up.

Etienne Garbugli: Maybe that ties into the next question. How does a company that works in the analytics space learn from its customers and maybe how does that differ? You worked in crazy B2C, super-fast scaling, and now you’re in a bit of a different context where I’m assuming there’s not a million people using the platform. How does that also change the way you approach learning from your customer?

Stefanía Ólafsdóttir: That’s a great question. I think when you are drowning in data, it’s easier to use quantitative data to discover when you want to dive into qualitative. I think from my perspective, it’s really important to mix those two—qualitative and quantitative. Quantitative, meaning you gather a bunch of data points and then you look at charts and you measure conversion funnels and retention, things like that.

And you segment it and create cohorts of people like a group of people that joined on the same week, a group of people that joined on the same week and tried a specific topic that week, or a group of people that joined on a specific week and won their first game or something like that. So you create all of these cohorts when you’re doing quantitative analysis.

Then qualitative, you have to talk to people. You both use things like user interviews and send out surveys and things like that. And we actively do both of those things at Avo and have, for the entirety of the time. Obviously, the first version is a very qualitative thing.

So we would just use every opportunity that we had to sit with people as they were using the product. We would also try to have really clear communication, open communication channels with our customers. So they could be reporting on Slack or somewhere anytime they were hitting questions or blockers or things like that, and that really stayed with the team.

Most of our customers talk about how we care about them and we’re a responsive team. If you give us feedback, we take that feedback and we try to act on it. I think qualitative is really important, but we also measure our quantitative success results and use them to identify opportunities, to ask questions, for example, and then vice versa. We talk to our customers a lot and then use that opportunity to confirm whether that’s a pattern using quantitative data.

Etienne Garbugli: You have experienced the pains and have done a lot of the work to solve the challenges that Avo is addressing today when you were at QuizUp. Do you feel that an entrepreneur without that expertise, that experience could have identified a similar opportunity? What were the advantages of having those experiences yourself, having experienced the pain and having been in those situations?

Stefanía Ólafsdóttir: This is a great question. So I think maybe it’s two-layered. Obviously, having been a data scientist and running a data team, it gives a perspective on how you can use data for product strategy. That was a part of my experience that I could bring into the toolbox of building Avo.

But I think what you’re ultimately asking here is, is it important to have walked in your customer’s shoes when you’re building a solution for them? And how valuable is it to have experienced the pain point that you’re actually solving?

I think that is very helpful. If you have not already done that when you’re building a product, you need to make sure it’s a really strong part of your culture that you’re constantly walking in your customer’s shoes.

For example, one of our customers, they’re very early in building tools for Dungeons & Dragons game masters. So it’s really specific. And obviously, you don’t necessarily have a huge selection of product managers and software developers that have experience with that. So what they do is every Friday, they play down to some dragons and they experience the pain points that they’re trying to solve.

I think this is a classic thing. I just stumbled upon a post from Justin Kan, the founder of Twitch TV. He talked about the first product that he and Michael Seibel and the crew behind Twitch built really early. It was like a calendar app. They did that before they did Justin TV that later turned into Twitch TV and all that stuff. What he talked about and identified, and I relate heavily to is they were college students and calendar management was not a huge pain point for them. All they had to know was like when’s my next class or something.

While being a CEO or a part of a big team or something like that, there are multiple uses for calendar management, but they didn’t have those problems. And they’ve talked about how that held them back.

To stitch it back to your question, I think yes, ultimately, that helped a lot and allowed us as a team to have deep insights into the actual pains that our customers have. But I think it’s also really important and I think it can be a trap that people fall into– Yes, it’s great to be able to solve your own problem, but you still need to listen to your customers to find emerging needs.

Etienne Garbugli: Yes. I think interestingly enough, with Justin Kan is that they stumbled through the same challenge afterward at their legal company a little later.

Stefanía Ólafsdóttir: Yeah exactly.

Etienne Garbugli: Maybe as a last question, how would you advise a new entrepreneur to think about business opportunities? What should he/she be looking for? It’s a very broad question.

Stefanía Ólafsdóttir: Yeah. So, we’re thinking before people become founders?

Etienne Garbugli: Yes. If you’re thinking back, say when you had left the company that had acquired QuizUp, for example.

Stefanía Ólafsdóttir: I think it is very helpful to be close to the problem that you are attempting to solve. So identifying problems to solve for people, it helps a lot if you have some experience with that problem or have at least been pretty close to someone who has had that experience. And ultimately, I like the perspective that if you can’t get 30 meetings from people that have the problem that you want to solve, where you want to talk about the problem and get them excited about solving it and things like that, then you don’t have a problem to solve.

Etienne Garbugli: So you’re not talking to them about the right thing. Okay, that’s great advice.

Stefanía Ólafsdóttir: Or if you can’t find at least 30 people to talk about the problem or the emerging needs with, then you probably won’t have a market to sell it to.

Etienne Garbugli: Yeah. Well, it’s about stats. Thanks for taking the time, Stefanía. Where can people go to learn more about Avo and your work?

Stefanía Ólafsdóttir: Yeah, great question. You can go to avo.app to learn more about Avo. I also host a podcast called The Right Track, where I interview leaders in data engineering and product about building data cultures. That’s really fun. That’s ultimately a lot of where my passion lies is just building tools to solve this. But it’s a mix of culture and tooling as well like most things are, and building a team.

So that’s something that I’m really interested in. So you can go to therighttrack.avo.app to learn more about that as well.

Etienne Garbugli: Lots of great guests. I will link that with the interview. Thank you very much for doing this.

Stefanía Ólafsdóttir: My pleasure. Thank you for having me.

More on Finding Emerging Needs

• How to Map the Needs of the Decision-Making Unit (DMU) in B2B Sales
• [ Interview ] Superpowered Co-Founder Patrick Vlaskovits on the Importance of Focus and Anti-Personas in B2B
• How Loopio Grew from 0 to 300 Customers by Focusing on Relationships