Archive for the 'Google Analytics' Category

Sep, 14th, 2015

Google Partners Masterclass 2015 insights part 5: get rid of referral spam from Google Analytics

Get rid of spam with Google

Previously we talked about Remarketing Lists for Search Ads, powered by Google Analytics, making it possible for you to target your search ads based on your customer’s past activity on your site, create deeper segmented audiences, customise your messages according to your user segments and save you the marketing budget for users you don’t want to re-attract. Today, we’ll talk about what everybody hates – referral spam and how to get rid of it!


Are your Google Analytics reports getting jammed up with bogus data from nasty referral spammers? Don’t worry. You’re not alone!

Throughout this post, I will be referring to great sources of information from the following authors and sites:

Let’s begin!

What’s the Problem – Annoying referral spammers!

During the event, David discussed the types of Google Analytics (GA) referral spam. In GA, there are two types:

  • Nasty bots and spammy web crawlers

These are robots that actually visit your website and show up in your analytics reports as sessions, often (but not always) with a 100% bounce rate and 0 second duration. They may or may not identify themselves as bots and can be filtered out.

  • Ghost referrers

They are tougher to identify and deal with as they never actually visit your website but exploit the Google Analytics Measurement Protocol.

How is that possible?

David cited a blog post by Ben Travis of Viget that provides you with a solution on how to deal with referral spam. According to the post, with ghost referral traffic, spammers can “spoof” a session very easily by generating a simple program that sends fake HTTP requests aimed at different Google Analytics properties, so this ghost referral traffic doesn’t even hit your website.

What’s the Solution – Crush them!

David presented 6 possible solutions on how best to get rid of these annoying referral spams:

Solution #1 – Enable Bot Filtering

This can be done within Google Analytics Reporting View Settings. By checking this box, you can automatically exclude known bots and spiders from your GA data. Read more about understanding bot and spider filtering from Google Analytics in this post by Sayf Sharif.

Solution #2 – Use Valid Hostname Filters 

Hostname filters

Source: Megalytic

David advises to include ONLY your site hostname because ghost referrals are coming from other hosts. The article from Mark Hansen of Megalytic shows how to create a list of valid hostnames that shouldn’t be part of your reports. Additionally, according to the article of Mike Sullivan of Analytics Edge, “since spam referrers don’t know whose website the tracking ID belongs to (randomly picking numbers); they send the ‘referral’ using a hostname that is not one of yours. You can then create an INCLUDE filter that keeps only what was recorded from one of your valid web hosts”. It cautions users to make sure to identify ALL valid hostnames or else end up excluding valid traffic.

Solution #3 – Exclude Bots With Campaign Source Filters 

Source filter

Source: Viget

For those non-ghost referrals that actually visit your site, you need to create another filter that will exclude a list of known referral spam sources or domains. Read Mark’s post to know how you can implement this on your site and Mike’s post from Analytics Edge for more information on this solution.

Solution #4 – Use Advance Segments To Get Historical Data  

Advance segments

Source: Viget

You can use advance segments to view historical data from before filters were implemented. Creating segments can help you select sessions that match your set criteria. Scroll down Mike’s post which shows how a single segment can include valid hostnames to remove ghost referral traffic and to exclude spam referrals created by crawlers.

Solution #5 – Use the Spam Insertion Tool

spam filter installer

Source: Simo Ahava

With over 400+ spam referral domains, Simo Ahava has written a post on this awesome spam filter tool which uses GA Management API. You can use this to remove referral spam by creating and applying filters. David added that this tool automates the process of adding bot filters across all accounts and then down to profiles you have EDIT access to. He notes that this tool can’t help with polluted Measurement Protocol hits, or with spam that doesn’t come in as referral traffic, or with spam that comes in as referral traffic but isn’t in the filters yet. See Simo’s post and follow the steps to get you started with this tool.

Solution #6 – Cookies and Custom Dimensions via Google Tag Manager

GTM cookie monster

Source: LunaMetrics

In Sayf’s article on Eliminating Dumb Ghost Referral Traffic in GA, he walks you through 5 steps to block out traffic from outside of your website with a combination of tracking changes and filters within Google Analytics or with Google Tag Manager (GTM). Below are the steps.

  1. Set a cookie on your website for anyone who reaches your website
  2. Create a Custom Dimension
  3. Grab the cookie value via GTM
  4. Pass in the cookie value
  5. Filter out the bad traffic by including only traffic where your Custom Dimension is set to your specific value

Get more details here.

You’re now only collecting data for those you cookied on a visit to your website.

Time to celebrate?! Not just yet…

Unfortunately, David and Sayf points out that it’s possible for ghost traffic to find and mimic that custom dimension too.

It should be clear by now that there is no long-term solution for getting rid of those nasty referral spams. However, by implementing the solutions above you can get cleaner and more accurate data from GA. Want to know more? Check out Carlos Escalera’s article complete with awesome visuals on how to keep away referrals spam with 2 filters.


Up Next…

For the sixth part of the Google Partners Masterclass 2015 series, our article tomorrow will discuss about another Google update which is on the enhancement to Dynamic Search Ads (DSA). With the new DSA, Google recommends categories based on the content of your websites and make suggestions on text ads. Find out more on tomorrow’s post.

No responses yet

Sep, 9th, 2015

Google Partners Masterclass 2015 insights part 4: Remarketing Lists for Search Ads (RLSA) within Google Analytics

Speak to right audience
Our previous post was about the two data feed enhancements to the Google Merchant Center, which improves efficiency for large retailers when it comes to data feed updates and uploading. It helps medium size and smaller retailers to quickly get started with shopping ads. For the fourth post of the Google Partners Masterclass 2015 series, we’ll talk about how you can use audiences in Google Analytics (GA) to reach your customers in Google Search, with no tagging changes needed.

Google Analytics Remarketing Lists for Search Ads (RLSA) – at last!

Remarketing is a great way to align digital marketing activities based on your user’s experience. Recently, Google introduced the RLSA functionality as part of the Google Analytics (GA) Remarketing option.


What does this mean for you?

With RLSA, you can effectively target your search ads based on your customers’ past activity on your website, anyone that matches your target criteria and anything you can track in GA. With over hundreds of GA dimensions and metrics you can leverage for remarketing, you can then use audiences to reach and re-engage your customers with a consistent brand message across both Google Search and Display.

Typically, you could create the “All visitors” audience in GA and add it to your existing Search campaigns, with a Bid only targeting setting. It’s a good idea to outbid on this audience as usually you get a better conversion ratefrom past visitors – so you want to increase your share of voice towards them.

Below a +100% bid adjustment has been set up, meaning that for instance a $0.50 max. CPC will become $0.50 x (+100%) = $1 for your past visitors.

Bid adjustment set up
Similarly, you could create deeper segmented audiences depending on their past engagement, such as:

  • cart abandoners
  • newsletter subscribers, etc.

and outbid on these audiences.


RLSA is also useful to customise your messages according to user segments, especially for those who entered your conversion funnel but didn’t complete it. For instance, it’s a good idea to have different messages for your leads (past visitors who signed up but haven’t converted yet).

To achieve this, you will need to duplicate your existing Search campaigns and add your RLSA with the “Target and bid” targeting setting, and of course adapt your creative accordingly.

Finally, RLSA is powerful functionality that prevents you from wasting money towards users you don’t want to re-attract. For instance, if you run a lead capturing campaign, you don’t want to pay for clicks from someone who is already one of your members. Or you could also wish to exclude users who didn’t meet qualitative requirements (age, income, location, etc.) you asked during their previous visit on your site.

To do this, you just need to add these audiences to your campaigns as negative audiences at the campaign level.


Up next…

For the fifth part of the Google Partners Masterclass 2015 series, we will be posting on getting rid of those nasty referral spammers from Google Analytics (GA). At the event, David Booth walked through the types of GA spam and possible solutions from various brilliant digital professionals.


No responses yet

Sep, 4th, 2015

Google Partners Masterclass 2015 insights part 3: Google Merchant Center Feed updates for large and small retailers

Save time with Google
To give you a recap of last week’s posts on the Google Partners Masterclass 2015 series, the first article covered Google’s concept of Micro-Moments and how we can leverage the series of small contextual moments of our customers to better understand and engage them. The second article presented key points raised by David Booth, the main speaker, on using data in a way that provides the most value for customers. For the third and succeeding articles of the series, we will be discussing the Google Analytics and AdWords updates presented at the event.

Let’s begin!

Google announced two new data feed enhancements to Merchant Center to help digital advertisers easily update ads and quickly get products in front of new customers.

Here are the enhancements and their benefits:

Online product inventory update feed


Online Product Inventory Update Feed

Source: Merkle | RKG

This works best for large retailers who require regular inventory updating.

Applications might include where inventory regularly changes throughout the day. You can simply update key attributes without submitting again the whole product feed. Only a few attributes are available to be updated in this way, including price, availability, sales price and sales price effective date.

What could this do for you? 

There are 3 main benefits of this update…

  1. Uber-fast processing – Shopping feed updates instantaneously after processing so Google Shopping will be able to display your updated information much quicker.
  2. Give your customers what they’re looking for – If there are product pricing or availability changes, you can submit on the fly, several times throughout the day. Google Shopping then displays only the latest and freshest information to your customers.
  3. It’s flexible – You can update your price, availability and sales price information for all or just a subset of product items separately from updating the main product feed.

Click here for a helpful article from Google on this update. There are other blog posts that might help you flesh out the added value of this feature. Check this post from Merkle | RKG.

Google Sheets add-on


Google sheets addon

This option is suitable for small- and medium-sized retailers with smaller inventory feeds or to give customers easy access to update the feed in real time. The Google Sheets extension directly connects your spreadsheet to Merchant Center for faster and easier upload. This gets your shopping ads up and running quickly.

What could this do for you? 

There are 2 main benefits of this update:

  1. Product validation directly from Google Sheets – the sidebar in the add-on lets you validate your data by individual row or for the entire sheet. It also shows you errors and warnings before you upload your data feed.
  2. Product upload directly from Google Sheets – from the sidebar, you can upload your entire spreadsheet into Merchant Center without leaving Sheets. You can easily see your upload results on the sidebar providing you with immediate feedback on feed processing. You can then make the changes, if needed.

Google has an article and video on submitting data feeds via Google Sheets here. Check it out to find out more about the update and some advantages and disadvantages of using it.


Up Next…

For the fourth part of the Google Partners Masterclass 2015 series, we will talk about another Google update which is on remarketing lists for search ads with Google Analytics. Google has made it possible to create remarketing audiences using the Google Analytics tag providing you with advanced list building capabilities.

No responses yet

Aug, 24th, 2015

How to retrieve traffic sources data without Google Analytics cookies

Customised analytics
At FIRST, we have a deep level of understanding of analytics and its application to our clients’ requirements.  As an example, a client has a custom CRM and wanted to retrieve the first and last click data into their CRM system. They wanted to truly measure new customer acquisition by understanding their very first originating channel. However, with the upgrade to Universal Analytics, there is no Google Analytics cookie present anymore where values can be extracted.

To overcome this, FIRST designed and built a solution where two custom cookies were created per user and where their originating traffic source was not overwritten.  What this means is that the client can understand how they attracted the user to visit their site in the first place – thus, allowing for smarter business decision in terms of where to spend valuable marketing budget.

In this post, I’m going to show you a way to retrieve traffic sources without Google Analytics cookies. As this is a working project, feel free to contribute. Your ideas are welcome.

Now, let’s get down to specifics.

The problem with Universal Analytics in tracking traffic sources

With the Google Analytics traditional tracking method (ga.js), it was easy to get this data since they were stored in the __utmz cookie. However, in Universal Analytics (analytics.js), the data is stored on Google’s server, making it impossible to retrieve. You can only see the value of traffic sources in your Google Analytics reports.

You are probably wondering why would you need to use these traffic sources data outside Google Analytics anyway.

Well, retrieving the data can be useful for mainly two reasons:

  • To push traffic sources data to your CRM system, which doesn’t have a built-in solution to integrate Google Analytics. As explained earlier, we designed this solution for a client who needed to access the first and last click data to push it in their CRM system;
  • To fire tags based on the traffic source data, e.g. enable a heatmap tracking tool only for users coming from your Google Adwords campaigns.

The solution – stimulating the Google Analytics processing flow

In order to retrieve the data, you would need to stimulate the Google Analytics processing flow to determine the values of traffic sources parameters:

  • source
  • medium
  • campaign
  • content
  • keyword

However, this solution sometimes gives different dimensions values than Google Analytics for the following reasons:

  • There is no source precedence meaning traffic sources are overridden by any source – unlike Google Analytics where traffic sources are not overridden when a Direct session follows a paid campaign
  • There is no session timeout defined – unlike Google analytics where a session ends after 30 minutes of inactivity by default, and at midnight
  • There is no ignored referrers – unlike Google Analytics, where you can specify referral domains to be ignored (See Referral exclusion feature). You can still customise the code provided below to ignore a specific referral
  • Only the main search engines are defined for organic source: Google, Bing, Yahoo, Baidu, Ask and Yandex – but again you can customise the code to add your own search engines if needed

The Code:

See the code to retrieve traffic sources data without GA cookie.

What it does

The first section of the code retrieves the utm_ parameters by following a similar processing logic as Google Analytics. You’ll need to change the “” to your domain.
The self-invoking function creates 2 cookies:

  • the FirstSession cookie retrieving the traffic sources data of the first time session;
  • the ReturningSession cookie retrieving the traffic sources data when the FirstSession cookie already exists.

These cookies expire after 182 days, i.e. approximately six months (same as Google Analytics campaign timeout period).

How to implement this with Google Tag Manager

Implementing this code in GTM is relatively straightforward. All you need is to create:

  • One Custom HTML tag including the code to fire on the landing page only
  • Two 1st Party Cookie variables for the FirstSession and ReturningSession cookies

The “Traffic Sources Data” custom HTML tag (click image to enlarge):

The “Traffic Sources Data” Custom HTML tag

The 1st Party Cookies variables for the FirstSession and ReturningSession cookies (click image to enlarge):

The 1st Party Cookies variables for the FirstSession and ReturningSession cookies:

Below are some illustrations of FirstSession and ReturningSession variable values as shown in the GTM Preview (click images to enlarge):

FirstSession Variable illustration

ReturningSession Variable illustration

ReturningSession Variable 2 illustration

Then, you can use these 1st Party Cookie variables to trigger other tags depending on their values. For instance, fire a heatmap tool for returning visitors from your email campaigns only or trigger a pop up for first time users only.


Building a customised solution helped our client identify and understand where customers first interacted with their website. This in turn helped them come up with improved new customer acquisition strategies that increased the number of conversions and satisfied customers on their website.

Now it’s your turn. Have you tried any customised tracking methods? How did you use them? Let us know!

No responses yet

Sep, 19th, 2014

Tracking fragment URLs in Universal Analytics with Google Tag Manager

Ah, fragment URLs – you know these URLs with a hashtag (such as, that can’t be tracked with Google Analytics

Indeed, Google Analytics strips out everything after the hashtag. Yet seeing what’s after the hashtag can be useful to know what are the most popular filtering options, for example, or to check if your visitors go beyond the first page of your product listing.

With Google Tag Manager, tracking these URLs as virtual pageviews becomes relatively easy. No need to be a developer or adding any code on your web pages.

Here is how to do it :

1. Create a Universal Analytics tag for Virtual Pageviews with a hashtag

This tag is different from your existing Universal Analytics Page View tag.

In the field for Document Path, click on the brick to insert the new {{hashtag URL}} macro that you create in step 2.


UA virtual pageview tag for hashtag URLs

UA virtual pageview tag for hashtag URLs
(Click image to enlarge)

2. Create the {{hashtag URL}} Custom JavaSscript macro with the function below:

function() {
var newURL = window.location.pathname + + window.location.hash;
return newURL;

3. Create the firing rules for the UA Virtual Pageview tag

You want to fire the virtual pageview tag when:

  • The URL changes in the browser without the page loading again (history change event rule)
  • Or when the user lands directly on a hashtag URL (fragment URL is present on the page load).

The first “History change event” firing rule is as follows:


Firing rule for the UA Virtual Pageviews tag

Firing rule for the UA Virtual Pageviews tag
(Click image to enlarge)


The rule calls the History Listener tag: {{event}} equals gtm.historyChange (see

The second “Fragment URL loaded” firing rule is like below:


Fragment URL loaded firing rule

“Fragment URL loaded” firing rule
(Click image to enlarge)


For this rule, you’ll need to create the {{fragment url}} macro as below:


Fragment URL macro

Fragment URL macro
(Click image to enlarge)


4. Save your UA Virtual Pageview tag

5. Create the History Listener tag


History listener tag to fire on all pages

History listener tag to fire on all pages
(Click image to enlarge)

6. Add the “Fragment URL loaded” rule as a blocking rule in your “standard” UA pageview tag

Because you don’t want to double track the hashtag URL on the page load.

7. Preview and debug your container version

Check that your Virtual Pageview tag for hashtag URLs fires when a fragment URL is loaded (tag fired on GTM Page Load event) AND also when you navigate to another hashtag URL (tag fired on gtm.hisotryChange event)

8. Visit your site and check your Real-Time Content report in Google Analytics


Google Analytics Real-Time Content report

Google Analytics Real-Time Content report
(Click image to enlarge)

Hooray, it works!

9. Create your new Google Tag Manager container version, name it, publish it, and get yourself a well-deserved cup of coffee, tea or whatever beverage you most prefer :-)


 Story based on:

2 responses so far

« Prev - Next »

© 2016 FIRST, part of the BeyondD Digital Marketing Group a wholly owned subsidiary of Beyond International.