Google Analytics & "Spam Data" - Removing Automated & Ghost Referrals From Your Analytics Reports

Seeing a sudden, unexpected and potentially dramatic rise in your traffic or other unexplained changes in patterns of traffic or engagement within Google Analytics? Even if nothing appears untoward we'd advise you to check your referrals and hostname reports to see whether you've been affected by either spam or ghost referrals.

Referrers such as semalt.com, ranksonic.info, social-buttons.com or buttons-for-website (and many others) can generate large numbers of referral sessions within your Google Analytics reports and seriously interfere with your traffic figures and overall engagement statistics. You may well find traffic from these and similar sites within the All Referrals report (which appears under Acquisition / All Traffic) - regardless of whether these are services you ever signed up for.

Sometimes you may see fairly dramatic spikes in your traffic - alerting you to a problem. In many cases however the volume of sessions generated may stay low enough to be below the radar and only identified after some investigation. Whilst low though the aggregate affect of multiple spam referrals can add up. Small businesses / sites need to beware - whilst an increase of a few hundred sessions may be a tiny blip for a large site with tens of thousands of visits a month if your site has a lower volume of traffic spam referrals can seriously distort your figures.

Within the All Referrals report look out for unusual patterns of traffic from specific referring sites. You may sometimes see a large spike in traffic from a site occurring on a single day with nothing generated thereafter for some time. Look out also for referrals generating a very high percentage of new sessions (often 100%) often with a bounce rate of or approaching 100% (although this is not always the case). Any referrals matching these criteria warrant further investigation (although be careful and don't automatically visit the suspect site - do some research via a search engine first).

You should also check your event tracking reports (available within Behaviour / Top Events) as we've recently encountered spam data appearing here (via ghost referrals). Event tracking has to be specifically setup for your website and is not available by default so if there's something you don't recognise then check with your web team.

Automated Spam & Bot Filtering

Google rolled out a bot and spider filtering tool in 2014. You configure this at a view settings level (via a tick box) and need to enable this for each view you wish it to apply to.

Spam traffic is currently a growing problem within Analytics. Selecting "remove bots and filters" within your view settings however still allows some automated referrals to make their way through to your reports and so you'll need to take action e.g. via setting up exclusion filters to remove them or taking one of the steps outlined below.

Ghost Referrals

It's recommended that you also check your "Audience / Technology / Network" report and then change the "primary dimension" (link below the graph) to "Hostname" to check for anything suspicious - the hostname is the domain from which your content was viewed or tracking was triggered.

If you see unexpected results here (e.g. darodar.com) these may be from ghost referrals - this traffic never actually visited your site but tracking was triggered via the Google Analytics "Measurement Protocol". An include filter (telling Google to only include your domain and other relevant domains) can help limit such ghost referrals but for a more comprehensive (but more technical) solution LunaMetrics have described a technique using a cookie and Custom dimension.

If adding a filter as always ensure you have a master view with no filters applied to ensure that if you make a mistake no data is lost. You might also wish to create a test view and apply the filter to it before rolling it out to your main views.

As the number of sites generating automated referrals and ghost referrals appears to be growing (we've seen a big increase over the last few months) and site sources change you'll also need to be proactive about this. Having performed an initial audit to identify and exclude any suspicious referrals from your reports you may then want to set up some alerts within the Intelligence Events section of your primary analytics view.

What if we have already been affected?

If you are preparing reports based upon historic data that includes referral spam you'll need to use advanced segments to filter this data out. Identify the hostnames and referrals you wish to exclude and then create an advanced filter to exclude traffic by source / hostname. (Note if there are a lot of suspect hostnames you may instead want to use an include statement).

Ultimately this is a problem that Google will need to (and we are confident will) address or using Analytics is going to become a lot more problematic - there is a danger though that the problem could get worse (and potentially more complex) in the meantime. For now we strongly advise that GA users are proactive about identifying whether they are affected and take steps to ensure the integrity of their data and their reporting.

If you have concerns about automated referrals compromising your data get in touch and we'll be happy to explain how we can help - drop us an email at info@andersanalytics.com. It's likely that there will be further developments here - we'll provide additional updates in due course.

An Introduction to Google Analytics Enhanced eCommerce Reporting

With the remarkable rise in online retail competition is now tougher than ever. On top of this, consumers are becoming increasingly savvy and demanding so retailers have to work extra hard to adapt their services to ever changing online shopping patterns.

Google’s Enhanced eCommerce tracking can help by providing insight into customer behaviour and the effectiveness of merchandising efforts. The emphasis now lies in understanding shopping habits, sales regeneration activities and effective customer segmentation.

With Enhanced Ecommerce you can see:

• View and click data for promotional campaigns

• View and click data for individual products

• View data for product detail pages

• Added and removed products from your shopping basket

• Data on started, completed and abandoned checkouts

• Refund requests

Once Enhanced eCommerce is enabled and feeding data through to Google Analytics you gain access to a number of reports:

• Shopping behaviour. You can look at the whole purchase funnel from a product page view through to a purchase. This can be customised to accommodate for specific elements unique to your shopping funnel

• Checkout behaviour. This report specifically looks at identifying the paths taken through your checkout process. This can be a powerful tool in understanding checkout funnel exits and can help in providing ideas for improvement.

• Product performance. Allows for the evaluation of individual products. You will be able to view the number of times a product has been viewed, added or removed from a cart and purchased.

• Product list performance. Helps with understanding and managing intra-site marketing. This means you can now compare products that are in a ‘Bestselling’ category to ‘Other people have purchased’ products.

• Coupon and affiliate marketing. You track revenue, transactions, and average order value as they are associated with affiliate sites that drive customers to your site. Additionally you can look at the effectiveness of internal promotions and product-level coupons.

Google Analytics Enhanced eCommerce is a complete revamp of the traditional eCommerce tracking as it provides many more ways to collect and analyse data. As a result it may feel a little daunting to implement but, if done properly, it may open up a world of opportunities.

Enhancements to Google Tag Manager

Google have recently rolled out a series of improvements to Google Tag Manager.

These include:

• New APIs that tailor the power of Google Tag Manager to your unique needs

• A new intuitive interface to help you launch and edit tags even faster

• More 3rd-party templates to make tagging easier

For further information please visit the Google Analytics blog.

Google Adwords - Callout Extensions

Callouts are a new ad extension from Google AdWords that allows for additional text information, complementing the product or service, to be shown with an ad.

Callouts can be created using the Ad extensions tab in Adwords and can be added at either the account, campaign or adgroup level. For example, if you own a hairdresser’s and you are running a discounted service, at the campaign level you may wish to display a callout text that says “discount available” and then you can have more specific callout text information detailing your discount at the adgroup level, such as “25% off hair wash”.

Callouts can be up to 25 characters long and Google recommends that they should be kept “short”, “specific” and “unique” to the ad in question, so tailoring your callout text in this way can be an effective way to target potential customers and give them a greater insight into your services. Callouts, like site links, also now feed through to AdRank – used to determine where your ads appear in search results.

Why website URL structure matters to Google Analytics

Many website content management systems by default show page URLs using "query string data" - i.e. the name of the your website domain followed by a question mark and some data indicating a particular page. For example www.mysite.com/?page_id=57.

Beyond analytics there are a number of advantages of having a "friendlier" URL structure. Some of the key advantages include:

1. A logical hierarchy using words rather than data in your URLs can convey information and help people understand where they are in your site content.

2. Page URLs that use query string data can be harder for people to remember.

3. There are advantages for SEO.

4. URLs appear in search results - if a relevant word matching a search term appears in your URL it will shown in bold making your entry stand out a little more. This could potentially increase the chances of someone clicking through to your site.

An often forgotten advantage of a well thought out URL hierarchy is the impact on working with Google Analytics.

If you use query string data for all page URLs understanding how people interact with your content can become very difficult.

For example if "page_id=44" performed better than "page_id=66" and there are another 50 popular pages how do you easily compare the performance of different content?

Assuming you've not got a photographic memory and can immediately recall the id for each page you'll either need to cross reference these numbers against a list or actually click through to each page. This makes analysis cumbersome and if you have a lot of content it becomes near impossible to identify trends or draw any deeper insight.

An improvement would be to amend your CMS (content management system) to replace this data with meaningful words. If "page_id=44" becomes "about-us" and "page_id-66" changes to "our-services" it's obviously a lot easier to understand the content reports within your analytics account.

For simple sites - for example a small company website - the above may well be sufficient. For larger sites or sites with different types of content however this still poses a challenge.

Lets consider a small business website that promotes a range of services and has an active blog. Blog content often performs very differently to other types of content on a website.

People often reach a blog as a result of a search, blog articles are frequently built around a particular topic and the article often provides sufficient information to answer the "question" that led people to perform the search. As a result blog articles frequently have a much higher bounce rate than other content on many sites.

Going back to analytics if you have a lot of blog content on your site but are also promoting services you'll need to be able to distinguish your blogs performance from the rest of your site. With a simple "flat" URL structure this can be difficult.

Luckily a simple solution is available. Adding a hierarchy to your URLs makes aggregating different types of content much easier within Analytics. For example preface blog pages with "/blog/", service pages with "/services/" and use product categories for e-commerce sites e.g. "/books/". This hierarchy will enable you to segment content via views, using the filter field, advanced segments and the content drill down reports.

Bear in mind that for complex sites the hierarchy can extend beyond one category. For example for an e-commerce site you may offer a range of books for sale and so may want to extend "/books/" to "/books/sci-fi/", "/books/biography/", "/books/fiction/" etc.

A quick tip where you do implement a URL hierarchy is to consider the opening page for each section. If you have an opening page for "/blog" and then actual blog content (e.g. /blog/articlename) you'll see two entries in the content drilldown reports for "/blog" and "/blog/". To avoid having to aggregate this data change the opening page URL to "/blog/" or alternatively use an Analytics advanced filter to rewrite it.

Inevitably for some complex websites query string data will be required. Where this query string data conveys meaning about the content being viewed you can use Advanced Filters to adjust your URLs in Analytics to make better sense of them. Think carefully about how you use query string data in advance as these filters will need to use logical rules about what page URIs to rewrite.

Google Tag Manager Concepts - Tags, Containers, Macros and More...

In a previous post we talked about some of the benefits of working with Google Tag Manager.

In case you were left wondering what a container or a tag is or how you set up a tag for Google Analytics this post looks at some of the key concepts you need to be familiar with when working with Google Tag Manager.

To start off with let’s look at the structure of Google Tag Manager. This contains three main components - the Account, the Container and Tags.

The Account represents the highest level of organisation. Usually only one account is needed per company with tags for all your company’s websites managed within this account.

A Container contains all the tags for a specific site. To create a new container you first select the relevant account and click the ‘New Container’ button. You will need to add a container name, time zone and domain.

Once you have created a container you can start creating Tags. These are essentially snippets of JavaScript that send information to Google and that can fire when you want them to (according to rules - see below).

Google provides ‘templates’ for its own tags but also allows custom tags to be used. To create a tag select the relevant container (i.e. website) and click the ‘New tag’ button. This will take you to a page where you will have to choose the tag type. Google Analytics, Adwords and Remarketing tags can all be found here.

Rules are also introduced at this level. Rules attached to a tag control when the tag is fired. Each tag must have at least one rule in order to fire. Rules that prevent tags from firing will override a firing rule.

Macros are "name-value pairs for which the ‘name’ is populated during runtime". For example you could use a macro to run some custom Javascript to set the category of an event as part of your event tracking within Google Analytics. Google Tag Manager provides a set of predefined macros for each container you create – for example {{url}} relates to the URL of the current page.

Google Analytics - Bot Filtering Option

Google recently added a new bot filtering tool that can be enabled by adjusting your "view settings" within your Analytics configuration.

Bots are essentially automated software programs that access your website. Many don't run Javascript on your site and so won't be captured but those that do can often lead to unexpected spikes in traffic or inflated traffic figures that can also skew engagement metrics (e.g. a large volume of traffic from a particular source with a 100% bounce rate).

Bear in mind that bots may access your site as you actually signed up to a service - for example to check site availability or to run an SEO audit of your site. With some detailed analysis it's usually possible to identify these but it can make analysis harder and if you miss something it's possible that your figures could be skewed artificially.

To enable this setting you need to go to your Admin settings and tick a box to exclude known bots.

According to Google:

Selecting this option will exclude all hits that come from bots and spiders on the IAB known bots and spiders list. The backend will exclude hits matching the User Agents named in the list as though they were subject to a profile filter. This will allow you to identify the real number of visitors that are coming to your site.

You'll need to check the tick box in each of the views you want to apply it to. Bear in mind that best practice is to always maintain one master profile with no filtering.

An Introduction to Google Tag Manager

Google Analytics is a fantastic tool for measuring interactions with your website. It enables you to assess your marketing channels, analyse visitor journeys and capture what really matters to your organisation via goals.

The setup of Google Analytics is also pretty easy – you simply add a snippet of code that you obtain from your Google Analytics count to your site – ensuring that it appears on all pages.

Out of the box Google Analytics however has some limitations in terms of what you can capture on your site by default. For example if you want to capture visitors downloading files, clicking onto email links or outbound links or watching a video you need to add some additional code to your site to capture these via event tracking or virtual page views. Historically this has been achieved using Javascript added to your site – for example to onClick events (to capture when people click on a link or other web page element) or JQuery code – a Javascript library.

Google Tag Manager is a free tool from Google that makes setting up event tracking and adding tags to your site a lot easier. Rather than rely on your web development team to make complex changes to your site you can now use Google Tag Manager to add the Google Analytics tracking snippet, set up events (for example to track downloads or form completions) and even add other non-Google related tags to your site (e.g. Floodlight).

Google Tag Manager has been designed to be relatively easy to use and with a little learning you are free to take full control over tracking on your site. You start by creating a Google Tag Manager account at – www.google.com/tagmanager and create a “container” for your site. It’s then a matter of adding your Analytics account, configuring any advanced options you want to use and adding event tracking and other tags as required. Once complete you copy the tag manager code to your site so that it appears just after the opening <body> tag.

Improvements to Search Query Data in Google Webmaster

An improvement to the search queries report within Google Webmaster has just been rolled out and announced on the Google Webmaster blog.

Rather than rounding up or bucketing search impressions and clicks data you'll now see far more accurate figures for both and data will be available for the last 90 days

With the rise of "not provided" search data within Analytics this enhancement will no doubt be very welcome by the SEO community and companies wishing to better understand their performance within organic search results. We'll revisit the impact this has upon measuring SEO in a later blog post.

Analytics - The Rise of 'Not Provided' Organic Search Data and Issues for SEO

A major change to how Google passes organic search keyword data to Analytics is currently underway.

Previously people logged into Google performed searches over a secure connection. This meant that keyword data was not provided to Google Analytics and appeared as such within organic search reports.

Google have now decided to divert other (and eventually all) users to also perform searches over a secure connection. The effect of this will be a rapidly dwindling volume of organic search keyword data within Analytics and an inexorable rise in the volume of "not provided" search.

Historically SEO companies have relied heavily upon Analytics data to demonstrate the impact of their work. Measurement has focused on the volume of traffic from particular keywords, long tail search and non-branded terms as a whole. It's also been important to demonstrate that keyword search traffic actually led to desirable onsite behaviours (e.g. visitors making a purchase or signing up for a newsletter) rather than inflating traffic figures simply for the sake of it.

In future it will be much harder to demonstrate that increases in organic search are purely down to the efforts of your SEO provider. For example are increases in search traffic due to an improved ranking or simply down to increased brand awareness that has nothing to do with your SEO company. This poses a major challenge to SEO providers with SEOMoz even questioning whether this is the first existential threat to SEO.

So what to do about the change? SEO is not about to die but it did just get harder. Companies will need to place heavier reliance upon Google Webmaster data - it's rumoured here that reports may extend from the last 90 days to 1 year which should help if true.

Paid search data is likely to become increasingly important in testing keyword performance on your site - for example to decide what terms are more likely to drive conversion. Assuming Bing and Yahoo continue to provide keyword data, analysis will remain possible here. However the much smaller share of search (particularly in certain markets) may make extrapolation difficult.

Analysing what landing pages people are directed to via organic search in Analytics is likely to also prove important. The use of additional profiles and advanced filters should help here.

In summary "Not Provided" data is likely to pose a challenge to many organisations. If you are having problems and would like to discuss options to improve your monitoring of organic search please get in touch.