Last summer, the Zaius team took stock of the state of our data science discipline, its future, and along what principles we wanted to grow. Ultimately, we intended to develop a data science operating charter, answering these key questions:
- What makes us believe in our data science discipline?
- Why should our clients trust our data science?
- How do we prioritize client value and outcomes?
- What is the right approach to measuring accuracy and tracking outcomes?
These aren’t easy questions, and they need to harmonize with how we build product, provide value to our customers, and protect the relationship between brands and consumers. We strongly believe that the best solutions are those that marketers can understand, trust, and participate in improving.
Transparency in data science has been a much discussed topic in the market with explainable AI and “glass-box” approaches. At Zaius, we’ve always valued transparency, but data science is often seen as a “black box” and “magic” that mere mortals can’t understand. We want to stay true to our core values in a space where many have struggled to deliver straightforward and clear solutions. How do we espouse these beliefs in a market where every vendor says “yes” and “we do that” to every possible capability?
This isn’t a one-and-done topic, and we’ll continue to share insight about the questions we’ve broached above. Today, we want to talk about some practical ways to determine if a marketing technology provider is delivering on their data science promises. Here are three questions to ask when deciding if the capability is worth your time, attention, and marketing dollars.
Is the data credible and complete?
The most important factor in data science is also one of the hardest to convey to clients: the importance of having the right type of data inputs, and, if possible, a lot of them. Also referred to as “the unreasonable effectiveness of data”, having the right data is often more important than the most complex of algorithms.
While smart approaches are always appreciated, every marketer should be asking about the quality and completeness of the data inputs, or, in other words, what data is required for the insight to be accurate for my business?
For example, to predict email opens, it makes sense to start with email send and engagement data. But it gets more complicated—if the model is predicting orders, or revenue, or churn, does it understand enough about your business and its touchpoints to see the complete picture? Imagine creating a churn model with only order data, and not having your channel marketing, support, site visit, or loyalty data integrated.
Marketers don’t have to be data scientists to understand what inputs generate meaningful and actionable outcomes; a marketer is likely better-equipped than any technology provider to identify what data is relevant to their business. And if that data isn’t included, marketers should be asking why. Next time you’re presented with a new model, start with this question and see where it takes you: What data is considered for this model?
Is the output actionable?
A lot of us at Zaius love interesting data just for the sake of the conversations it produces, and we’ve built great relationships with our teammates on that foundation. However, when it comes to delivering new insights, we don’t hold with building features just to check a box. We think that all of our data insights should deliver real value and actionable next steps.
To that end, when considering if a feature is actionable, we like to consider context and coverage.
First, let’s talk about context. The key to context is knowing enough to put the insight into practice. For years, a top requested feature was predicted lifetime value (PLTV). When we started Zaius, we even considered making that a central concept on which to drive all marketing experiences. Over time, though, we learned that there is almost never enough context for PLTV to be actionable, because it has no timetable associated with it.
For example, if customer 1 has a PLTV of $400 and customer 2 has a PLTV of $1,000, is that enough to change the action you would take right now? What if one has currently spent $50 and the other $900 to-date? Given the inherent lack of context with PLTV, we’ve built a different approach to customer value, addressing context and actionability through a Likelihood to Order model with a clearly defined timetable.
The other key aspect to being actionable is coverage. To be actionable, the insight needs to be executable in marketing experiences at scale. If a model is available for purchase likelihood, but can only be accurate for those who have purchased 4 times in the last 6 months, is that enough of your customer base to matter? Is it addressing the part of your customer base where you most need support?
Most of our new clients at Zaius only see 5% of their known customer base on their site in a given month. To ensure that they make the most of all site visitors, we build models that address anonymous customers all the way through loyal customers, not just the customers that purchase like clockwork. One of our clients recently told us they have a threshold of 20%: Will this new capability impact 20% of my customer base?
Is the data science approach appropriate?
We may do an entire post someday about the poor approaches we see to data science and the impact it has on marketers. That said, let’s give this question a quick once-over, in the interest of being complete.
Much of whether the approach is appropriate has to do with whether it is appropriate for your business. Data science is not a one-size-fits-all discipline, and what’s right for a luxury fashion brand may not be right for a speciality foods retailer or an in-car audio specialist. Consider whether the model takes into account data that is custom to your business. What happens when you add new data to ecosystem? How does the model consider the unique dynamics of your customer journey?
Ultimately, if an approach does not adjust to the context of your business, the output never performs as well for your brand as it could. That’s why, at Zaius, we specifically build solutions that can understand and act against the unique aspects of each of clients’ brand, and are flexible enough to regularly update when new data is available in the ecosystem.
We know marketers can feel intimidated when questioning a data science approach, but this is your data and your business, and not every approach is right for every brand. Next time you are introduced to a new model, ask yourself, “Does the approach being used conflict with or complement what I know about my business?” and trust yourself to make the right decision based on the answers you learn.
We’ll continue our discussion of data science in marketing, and data science for marketers in the coming months. We don’t take lightly the trust that our clients place in us to provide transparent, actionable insights, and we are committed to helping brands reach their potential through that data. Drop us a line if there is something you’d like to see in this space.