Lumar’s SEO & digital marketing interview series is back! Our technical SEO extraordinaire, Jamie Indigo, recorded a whole new season of SEO interviews at last month’s BrightonSEO conference — and we’ll be posting new interviews every week.
To kick off this new season of interviews, we have a discussion about SEO testing and SEO data analytics, how to get involved with search industry events, and a whole lot more. Jamie is joined by first-time BrightonSEO speaker and independent SEO consultant (not to mention figure skater and “former neuroscientist turned SEO”!), Giulia Panozzo.
Listen to the full conversation below, or read on for an abridged transcript.
Jamie: Tell me about your BrightonSEO panel, what was the title?
Giulia: My title was “Winner or loser? Analyzing your test results with causal impact on R studio.” It was a long title.
Jamie: So, A/B testing?
Giulia: Time series testing.
Jamie: Time series testing, excellent — how to compensate for seasonality and additional factors…
Giulia: Well that’s one of the things that I mention in my talk. It’s the fact that sometimes when we do that kind of testing, you need to take into account those [additional] factors …
And that was one of the questions, actually: how do you make sure that you isolate your treatment from all of the seasonality, events, and all the outliers that might come through? And actually, at present, we haven’t really found our perfect way to do that – so we’ll need to explore that better.
Jamie: I love that. I find that public speaking makes me take something that I think I know really well and dive much deeper into it and understand it in an interesting, new way after.
So for those who attended your panel, what’s one thing you hope they walked away with?
Giulia: I hope they walk away with inspiration to try [testing] on their own. I’m aware that there are tools to do time series analysis, but sometimes we come across budget constraints. So with the tool that I show, it’s free to use and anyone can use it. Also, it doesn’t require any coding skills or anything at all, really, it does pretty much everything for you once you apply the script. So hopefully I inspire people to try that.
Jamie: That’s fantastic. How did you get into your specialty?
Giulia: By chance, and by necessity.
So actually I used R before, in university, for statistics. I hated it! And I remember saying, ‘this will never be useful in my life after I’ve done this exam [which was the very last one in my degree] I’m never going to touch R again.” And then fast-forward a few years later, I found myself needing to justify some strategy changes that I wanted to make.
I was working in an agency at the time. And sometimes, when you propose changes [to a client], it’s really hard for them to see the win. And even when you work in-house, you always have different teams to collaborate with and different things that you have dependencies on. So having that kind of statistical significance brought by tests on proposed changes can make a case for more resources and can help you have the buy-in from the business in general for those changes.
Jamie: So it was a way of explaining ROI to your stakeholders and getting buy-in. How did you make allies across these teams? Is it all stats? Like, “Here, I’ll win your heart with some stats!”
Giulia: Now that’s a work in progress, really, because although statistics, you know, is the go-to to show stakeholders to clear any doubts, sometimes it’s hard anyway to make space in their agendas or in their backlogs.
Jamie: Backlogs especially in legacy sites — they are big, very packed beasts.
Giulia: Yeah. So sometimes I find that it’s actually more helpful, once you get their sign-off, to show results on their own ideas. … Sometimes stakeholders spend so much time or resources on stuff that they’ve seen other competitors doing. And then they can’t really quantify what the return is, other than like, “do you rank ahead of the competition now?”… And sometimes, when they spend a lot of time or resources doing that, it’s useful to show them that actually, the change is not meaningful, or it cannot be justified by the single change that they made. So it can help direct their strategy better in the future. Sometimes it’s hard to get across the layer between agency and client, I think.
Jamie: It’s a big divide!
Giulia: In the past, I was very lucky that they were actually using causal impact in-house as well. So that’s basically where I learned to use it from, so they would be trusting me with the results. But sometimes when, you know, you work somewhere else and you bring that [testing and data] in… other teams can be a bit different because they have their own way of testing. Sometimes they trust tools more, but then they don’t realize that [the model you are proposing] is actually the same model that is used in the tools to measure the same things.
Jamie: I think that’s a fundamental myth that a lot of marketers face, where they just think they put in a token, and then the tool just tells them: here’s the answer. And it’s similar to search engines themselves. You have a question, you ask your phone, but why do you trust it?
So someone like yourself, who knows the step-by-step mechanics, the machinations to go ahead and take erratic behaviors of humans (because my goodness, they are erratic!), and pull out meaningful insight into being able to say something like, “Every change we make costs money, costs time, costs labor. The ROI is here, but it’s not here,” and pivoting the strategy… I mean, we can have a lot of money-pit sites. It sounds like if there can be a viable solution in this causal reporting that you’re working on and inspiring others to move deeper into, then that can help isolate the variables in a meaningful way.
Giulia: I think at the same time, there’s a caveat to be made, which is, you know, you’re dealing with statistics. But as I mentioned in my talk, statistics can be made to show anything — including the truth.
Jamie: It’s the old joke, right? Three types of lies: lies, dirty lies, and stats.
Giulia: Yeah, exactly. So, I think you need to put some critical thinking into it. You have your data, and it’s great when you can have that kind of confidence to report on statistically significant results. But at the same time, sometimes you have like, big fat outliers. So for example, sometimes we have indexation issues, or tracking tools that have bugs. I think even Google Search Console at some point had some reporting issues. And that reflects into tracking. Sometimes you have seasonal events: Black Friday, Cyber Monday, and so on. When you run a test in those timeframes, there’s always something [or some product] that comes out as like, higher up in sales. And if that product is part of your test group, obviously it can skew your results. So there are a number of factors in the digital landscape that have to be acknowledged. Sometimes it’s a bit of a challenge because anyway, it’s not like you can just mindlessly input your data and expect to trust the result as it is.
Jamie: So data integrity is a big deal. How do we keep the bots out of there?
Giulia: Yeah, I think it’s something that we need to take into account. And so I hope that people take the time to do that as well, because it’s really easy to report on positive results. And I would like to do it all the time as well, especially when I’m the one proposing a change. But sometimes you need to take a step back, look at the data set and be like, “okay, but these doesn’t make much sense.”
And although it’s a great result to be reported to your stakeholders, they also need to trust you with the the kind of data that you provide. And I think in the long term it’s actually better to just trust the integrity more than the result itself, because it can help you strategize better, and you can learn yourself how to distinguish what works and doesn’t work for your audience.
Jamie: Absolutely. And I think that’s the trick there; we have to make decisions based on our audiences. But if your client, like many companies out there, are making their decisions based on competitive analysis, which is often gleamed by using bots on their site, it would absolutely skew how you perceive users to be using your site. Because those aren’t real humans. Those are competitive analysis bots. Wouldn’t that be funny, if we’re all chasing each other’s tails by a competitive analysis and by bots.
What has been your favorite part about coming into SEO as a career path?
Giulia: I think that I’ve never stopped learning. And I’m never, you know, learning remotely enough. Even when I come to these events — and BrightonSEO is unique because you get so many valuable, valuable talks — but even when I go to smaller events, there’s always something to be taken from those talks. And I love it.
I’m an advocate for lifelong learning. I think that’s the most valuable part. My favorite part of being an SEO.
Jamie: What do you see, in the upcoming year, being the challenges that SEOs are going to face and should perhaps put some of their lifelong learning efforts into exploring?
Giulia: Oh, that’s an interesting question. Because I think normally, we can just take whatever the latest algorithm update has been, and take that as the next challenge. But there have been so many just in the past couple of months. You can never sit still.
So I think that might be one of the challenges, having to adapt to what the new algorithm seemed to focus on. And also trying to keep the focus while doing that. Because we have strategy plans that, depending on the client, can be one year long, or it can be a five-year plan. And you can’t throw everything out every time you read something about a Google update.
Jamie: It’s reactive. Everyone’s gotten those emails, that are like, “How did the latest algorithm affect me?!” And it’s like: it just started rolling out. It’s not even complete yet. We don’t know if it did.
And so many of these algorithm updates now are highly specific to a niche to a vertical industry. But what do you say to those reactive clients who want to throw out road mapping, because they just heard a new update has come?
Giulia: I think I try to make them see the bigger plan. Sometimes, when they don’t really have an understanding, or a deeper understanding, of rankings and how the holistic search landscape works, their only metric of success is ranking first on Google for everything, right? So the second they drop to second for a keyword, they might panic. But then you need to remind them look, this is the kind of data that we have. And I like using data a lot because that’s the way that you can make your case.
Jamie: Good data makes good decisions.
Giulia: Exactly. So once they see the graphs, and they see that, yep, maybe you dropped for these keywords, but they are longtail keywords, and it’s fine. Like, you also gained positions for something that’s way more valuable. I just try to show the data. And yeah, I think normally it works.
Jamie: Thank you so much for joining me. For folks listening who might want to get in touch or follow you, how can they get in touch?
Giulia: I’m on LinkedIn. And I’m on Twitter as sequinsNsearch, because I’m also a figure skater!
Jamie Indigo can also be found on Twitter — and on Lumar’s professional services team.
Our professional services team works directly with Lumar clients to provide bespoke technical SEO guidance, SEO audits and website reviews, organic search strategy advice, and much, much more. (Want to work with Lumar’s expert tech SEOs? Get started with Lumar today!)
