There’s no shortage of analytics data available. In fact, that tends to be the problem. So much data exists that it can be difficult to figure out what’s important and actionable. Particularly from a search perspective, the key is remembering those goals you defined when building searcher personas. What metrics give you insight into how well you are reaching those goals? And just as importantly, what can you do with the data you’re tracking?
Businesses are finally starting to break out conversion rates from search. But too many of them start and end there. At a recent tech event in Seattle, the CEO from Redfin noted that their conversion rate from search is low.1 But ‘‘from search’’ is like calculating a conversion rate from TV ads with no interest in whether the commercials aired during a Friday night monster truck rally or a Saturday morning cartoon, or like only tracking the conversion rate for ‘‘advertising,’’ when that might include TV, radio, print, and online.
Conversion rate is absolutely important to understanding how well a site is performing, but it’s much more actionable when viewed by acquisition segment. Even better is if you can tie visitors to future behavior. Do visitors who come in through message boards talking about eco-friendly materials read an average of three articles on the site, share those links on two social media sites, which in turn bring in 12 visitors who convert?
Avinash Kaushik, Google’s analytics evangelist and author of Web Analytics: An Hour a Day and Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity, suggests rolling all conversion data into one metric: task completion rate by primary purpose.
He suggests finding out the core reasons why visitors come to your site. One way is to display an optional survey that asks, ‘‘why are you visiting the site today,’’ and then determining if each visitor segment was able to complete the task. Another way is to simply ask.
He suggests that this metric is more valuable than conversion rate to your business goal (selling something, etc.) because not all visitors are there for that purpose and, therefore, will never convert in the way you are measuring. 3 Some percentage of visitors is looking for company information, product support, or other information. You want to find out all the reasons that bring visitors to your site and if you are meeting their needs. In some cases, understanding this can help increase satisfaction of existing customers and improve customer lifetime value; in other cases, this can help increase engagement of potential customers and improve the likelihood that they will return to purchase later. But, just as important, by segmenting those visitors who are not there to purchase today (or do whatever task is mapped to your conversion goal), you have better insight into how well you are converting those visitors who do have purchase intent.
And you have substantially better insight into how your Web site is failing your visitor needs.
Define your goals before setting up metrics for your search acquisition strategy. When I ask people what their goals are for a particular site or page, I hear answers like ‘‘to rank well’’ and ‘‘to get more traffic.’’
But those aren’t business goals. You only want to rank well so you can attract visitors to your site and you only want to attract visitors to your site if you have a reasonable chance of converting them.
What’s Not Important
Look at most Web analytics and SEO reports and you’ll see things like the following:
- Rankings reports that list hundreds of keywords and how the site ranks for them.
- Indexing reports that show the change in the number of indexed pages over time.
- Absolute traffic numbers.
- Overall percentage of traffic from organic search and overall conversion rate from organic search.
- Page views.
- Daily unique visitors.
- Top entry and exit pages.
- Visitor screen resolutions and browser versions.
But these numbers tell us little if anything about the performance of the site. Who are these visitors? What are they looking for? Are they the visitors we are trying to attract? Did we engage them?
Better metrics would focus on goals and actionable insights:
- What’s the organic search traffic breakdown for query categories we are targeting and how are those visitors behaving on the site? (Are they converting? Engaging? Abandoning right away?)
- What query categories aren’t performing as well as we’d like? Is the performance breakdown at the search result (resulting in lack of traffic for those queries to the site) or at the site level (resulting in early abandonment or low conversion rates)?
- What are the primary reasons visitors are coming to the site and what are their task completion rates?
- What referring links are bringing the most traffic and how well are those visitors performing on the site? If they’re performing well, this may be an audience you should spent time cultivating.
The specific metrics that help you measure your business are unique to your site, but always evaluate them against the following:
- Do they give you insight into how well you are meeting your goals?
- Are they actionable? What will you do with this data?
Competitive Intelligence as a Benchmark to How Well You’re Really Doing
Benchmarking data can be a valuable data point about how well you’re really doing. It can help you know if an increase in search traffic is due to an overall trend of higher search volumes for your industry or is at the expense of your competition.
Having Actionable Analytics Data Is a Competitive Advantage
By some estimates, only 23 percent of sites have an analytics package installed, and only 1 percent are doing A/B or multivariate testing.4 By simply having this data at hand, you’re well ahead of most of your competition.
Attribution issues make metrics even more complicated. As we’ve seen, searchers don’t always follow a straight path from search to purchase. They not only may search for several different things (with subsequent queries triggered in part by results they saw in earlier queries), but may also search over more than one session. One study found that 56 percent of purchases occur in a search session later than the first one.5 Another found that only 43 percent of purchasers make that purchase within an
hour of the site visit. For some categories, the lag times can be substantial, such as shown below with a graph from a DoubleClick study (see Figure 8.1).
eBay, which we’ll learn more about shortly, handles the attribution issue in its metrics by using cookie data to store visitor information and attributing a conversion to search if it happens within 24 hours. The numbers tell us that this method isn’t perfect, but it’s better than only attributing conversions at the point of searching.
In addition to the time lag element of attribution, there’s the triggering element. We can see this in a hypothetical example for Volvo. In this example, we’ll consider entering a zip code to find the nearest dealer a conversion (see Figure 8.2).
Looking at these hypothetical numbers separately, branded search converts significantly better than non-branded search or display ads. With display ads, 20,000 people saw the ad, but only 1,000 of those clicked on it; and of that 1,000, only 50 converted. But with branded search, 10,000 people clicked on the Volvo result after searching for it and 6,000 of them converted. In isolation, it seems clear that we should stop our display advertising spend, stop optimizing for non-branded searches, and put all of our energy toward branded search. Or should we?
Let’s take a closer look. In this example, 50,000 people did nonbranded searches (such as fuel-efficient cars and safe cars). Of those 50,000, 30,000 visited the site and 30,000 x .15 converted, but all 50,000 saw the Volvo brand in the results. Of the 20,000 people who saw the display ad, only 1,000 clicked through to the site, but all 20,000 saw the Volvo brand.
One thousand of the people who saw the Volvo ad later searched for [Volvo] and clicked through to the site and 5,000 of the people who did non-branded searches and saw the Volvo result later searched for [Volvo] and clicked through to the site.
So without the display ads and non-branded search results, we’d be left with only 4,000 visitors. Study6 after study7 has found that most prepurchase activity involves generic terms and that brand searches tend to happen only close to purchase (see Figures 8.3 and 8.4)
Clearly the generic searches are providing insight into brands or the searchers would simply begin with brand searches. Many searches don’t search for brands at all before purchasing.
But how do you track searcher behavior across multiple sessions? Several companies are trying to solve this problem. Some of them store information in a user’s cookie when a display ad is served, for instance, and later can determine what users saw the display ad then later did searches that led those users to the site (and what those searches were). These technologies aren’t able to tell when a searcher has seen an organic result and not clicked on it. (The only way to currently get organic impression information is via Google Webmaster Tools, and that data is only in aggregate, not per visitor).
You can track visitors by IP address as well, although this method isn’t 100 percent reliable either, since many people search at work and
buy at home or search in one location and then e-mail their spouse the link for later purchase.
For more in-depth discussion of attribution in web analytics and possible approaches, See Web Analytics 2.0: The Art of Online Accountability and Science of Customer Centricity by Avinash Kaushik.
How eBay Uses Analytics to Inform What They Should Do, Not Just Show What They’ve Done
eBay has one of the largest Web sites in the world and, judging by the paid search ads served up with nearly every query, is one of the largest bidders on keywords. Between the paid search data about what people are searching for and what they click on and information about how visitors navigate the site, eBay has one of the largest data sets on user behavior of any site on the Web. A few short years ago, those at eBay threw most of that information away after only six months. In 2006, they realized they had a goldmine of data that could help them better understand their customers, build better buying and selling experiences, and improve eBay. Now, they protect that data vigilantly and have realigned business processes and goals to prioritize analytics throughout the organization.
According to Greg Fant, VP of Customer Insights and Marketing, analytics data doesn’t just tell eBay how it has done; they tell what it should do. Decisions that used to take weeks of one person’s opinion versus another are now data driven. The site’s testing platform helps confirm the decisions. Using data to drive strategy was an organizational shift in solving problems that didn’t come easily. But the benefits were huge. Fant says those at eBay used to be blind to problems but now they can see issues as they pop up. Issues that used to take three to four months to track down can now be found in a week. And shifting from financial metrics to site metrics has helped them better connect with customers and maximize happiness for both the buyer and seller. They found that data was so vital that they changed their site infrastructure to get the data they needed.
Those at eBay are quick to stress that search and Web analytics data helps them with all decisions, not just marketing-related ones.
Both organic and paid search data helps them discern searcher intent, find patterns in queries, and learn broader classifications and interests than what they can get from just monitoring on-site behavior. They pay close attention to what queries searchers are pairing with the word ‘‘eBay.’’ What problems are they having? What are they looking for eBay to provide? By identifying ‘‘silent sufferers,’’ they can learn about issues and solve them before they become widespread pain points.
Segmenting by query type has helped those at eBay identify different loyalty points, learn about ways different segments are motivated, and begin to better understand the analytics holy grail that is customer lifetime value.
How has eBay been able to transition to a data-driven organization, build search data into product strategy, and break down data silos? It has been a long process. Those at eBay credit support from the top: boardlevel agreement led to an organizational shift and analytical platform from which to base decisions. And the rest of the company came on board through a common set of metrics and vision across the entire company. When everyone is working towards the same goal and is measuring the same things, sharing data becomes significantly easier. For eBay, that common metric is purchases per week. Analytics used to be spread out under multiple divisions. They’ve now centralized. They focus on data that’s practical and actionable.
They go after low-hanging fruit for quick wins along the way while they work on longer-term initiatives such as new infrastructure and an A/ B testing platform. They build some things on a separate infrastructure for faster implementation and less impact to the primary site. They prioritize (based, of course, on data). And they’ve built an in-house organic search team that is focused on understanding how Google crawls, indexes, and ranks. Dennis Goedegebuure, Senior Manager of SEO, has made it his mission to build a centralized organic search team and develop search-friendly best practices throughout the organization.
Fant talks about a new breed of data-driven marketers and leadership teams who are ushering eBay into a new era of audience analysis, market research, and product strategy. At eBay, everyone understands that the user experience often begins on Google, and they spend a lot of time working to understand what Google searchers are really looking for.
Both intent and attribution are issues that eBay spends a lot of time on. A sample customer interaction on eBay is shown in Figure 8.5.
In this example, the searcher initially looks for [Google Android] and clicks on eBay’s paid search ad. This brings the searcher to an eBay page that lists items related to Google Android, including phones, phone covers, and chargers. The searcher later goes back to Google and this time remembers both that eBay had relevant information and that the previous search hadn’t been specific enough to bring back only phones. So, the second search is for [eBay Android G1]. This time, the searcher clicks eBay’s organic result and lands on another search results page that still isn’t quite specific enough. The searcher then searches within eBay to better refine the results and ultimately clicks an item listing.
In this scenario, if eBay tracked only the search that led to the click on the item listing, they would know only that the visitor had searched for [G1 HTC Android phone]. Had they only followed the path back one level, they would know that being visible in organic results is important, but they wouldn’t know that the paid search ad was important for branding and caused the searcher to include the word ‘‘eBay’’ in that search. By following the visitor all the way to the initial search, eBay knows that the word ‘‘Google’’ was initially an important component, dropped later only to make room for more specific keywords. The complete picture is much more helpful in formulating customer understanding than just the final query.
But this very common navigation path raises hard attribution questions. Who gets credit for the sale? Paid search? Organic search? Internal site search? In truth, they all contributed to the customer’s path to conversion. Without any one of them, the sale may never have happened.
When eBay started revising their product pages based on user data such as this, traffic from search went down. eBay couldn’t be more pleased. Why? Because eBay has higher quality, more descriptive pages in the search results, the quality of the traffic now is substantially better. Searchers who are genuinely interested in buying and selling are the ones clicking through the search results and visiting the site. And the pages they are visiting are the pages most likely to convert. eBay no longer has as much unqualified traffic to pages that have low conversion value. eBay has learned that traffic alone is meaningless. More traffic isn’t necessarily better.
By incorporating search data throughout their organization, eBay makes better business decisions, builds more effective product strategy, connects more closely with customers, and knows about problems before they escalate. It has created a more efficient working relationship between teams, and ultimately, has increased purchases per week.
Can Software Accurately Calculate Attribution?
Enquisite is a software company trying to address the issues of attribution and return on investment (ROI) metrics for organic search. It is building predictive modeling and analysis tools for all aspects of search analytics, including organic, paid, and referral links. It keeps track of traffic differences between search engines, generates long-tail traffic reports, and provides insights into business-to-business traffic. It is using search and analytics data to help companies understand user behavior, what topics and products perform best, and to build rollout dashboards that provide concrete measurements at a glance.
Can Enquisite help eBay with its attribution problem? Maybe. Richard Zwicky, Enquisite founder, says those at Enquisite use persistent and session-based cookies to understand the full visitor path, not simply the last click. They know what the visitor searched for before the last search, and they can weigh the value of the clicks themselves. They are also attempting to relate value across channels. Just how valuable was that paid search ad in influencing the searcher’s decision to later do a brand search and ultimately purchase from eBay?
ClearSaleing is also tackling the attribution issue. Their Attribution Management Buyer’s Guide outlines their approach to calculating attribution across online channels, including both paid and organic search.10 As they note,
. . . many conversions are the result of multiple forms of advertising. For example, a banner impression leads someone [to] click on a paid search ad, then an organic search, and then they convert. If the solution you’re using is only able to capture paid search, it would be oblivious to the fact that the banner impression is what introduced that person to your brand, and that the organic listing is what eventually closed the deal.
What About Offline Attribution?
Of course, ideally, attribution tracking would include offline channels as well (both for bringing in customers and for the conversion). How can you track a billboard that leads to a search that leads to a sale, such as the case of the movie 2012? What about a search that leads to a phone call that leads to a sale? We know that television advertising triggers search,12 and we know that searches are often the first step to offline purchases, 13 but can we track that?
A number of at least partial solutions exist for offline attribution, but the key is understanding that a relationship exists, sharing data between departments, and devoting resources to learning how these relationships work for your customer base.
For instance, dynamic number insertion14 can help track phone calls based on different search ads. You can use a different phone number online than in printed materials so that you can track online and offline channels. You can provide online coupons to be used for offline purchases. You can even display different coupons based on what search led the visitor to the page. At a more basic level, you can track the change in offline purchases as your site becomes more visible in organic search (noting other factors, such as advertising and seasonal fluctuations that may impact this).
The Trouble with Data
The trouble with data on the Web is that it’s impossible to measure with absolute accuracy.15 This is true of data everywhere, of course; it’s just more measurable on the Web. Cities hire car counting firms to track how busy intersections are to make decisions about new traffic signals, and, although it’s likely the counters miss a few cars here and there, the city can still be fairly confident in the conclusions. We track how many people are watching particular television shows by relying on a sampling of viewers to accurately log their viewing habits. They don’t, of course, but it’s still the best signal we’ve got. With Web analytics data, it’s much easier to tell just how far off the numbers are. If you run two Web analytics programs on your site in parallel (and many sites do), you will come up with two sets of data that can never be reconciled. But that’s OK. Rather than spend your time trying to reconcile them, just know that they are about as accurate as Nielson data or car counting and can be used for overall trends and measurements, if not exact counts.
The Value of an Experienced In-House Web Analytics Expert
Accurate, actionable data is critical in today’s online environment. And with so much data available—much of it confusing and not actionable— the best way to cut through to what really matters is to ensure you have a skilled Web analytics expert on staff who can cull through the hundreds of data points and give you the top five metrics that show you how the business is going and help you make business decisions.
Avinash Kaushik, wrote on his blog about the top things to look for in a Web analytics manager. They include:
- Has passion: ‘‘You have to love the Web, you have to love its life changing power, you have to believe that at some level you, yes you, have the power to change people’s lives by killing reports that show, in 8 font, conversion and visitors stats and replacing them with analysis that will fundamentally improve the experiences of your customers and ease their stress.’’
- Loves change: ‘‘The Web changes and grows and morphs like an alien being. You just got used to prefetching and here is Google trying to guess searcher intent (and now ‘‘personalized search’’ where even your awesome organic listing might not result in your being number one) and then come RIAs and Ajax and then will come something else and, like a tidal wave, it keeps coming. Ask for one thing they know of that will mess up the analytics world in the next 180 days. If they have an answer, you have a winner.’’
- Questions data: ‘‘The core problem with all the tools today is that they rely on data that usually tells you little. [Good analysts] question the data and analysis and bring a new and different perspective and can overcome cool recommendations from the latest consultant.’’
- Believes in customer-driven innovation: ‘‘True innovation and sustainable competitive advantage will [often] come from actions that are Customer Driven. What this translates into is a Manager who has had exposure to Testing and Usability and Surveys and Site Visits and Field Studies or Market Research . . . someone who can bring to bear their experience to find the right ways for you to apply the power of analytics and research to find the right answers for your customers.’’
Data is tricky. You need to make sure the underlying data is solid so you know you can trust your conclusions. You need to measure the right things. And you need to focus your investments on analyzing data that’s actionable. A great Web analytics manager, particularly one who understands that all acquisition channels bring the same customers, can ensure that you are looking at the right data and taking full advantage of it.