To make it easier to follow I split this guide into 4 parts. The first 2 are practical, the last 2 are informative:

Author

Analytics & GTM Expert

UX-SEO Advocate

Follow me on
Category | Google Analytics
Difficulty |

Filtering Google Analytics spam, bots, and other junk traffic: A comprehensive solution!

Google Analytics is probably one of the most important elements of the decision-making process of your website. The success or failure of your efforts (SEO, ad campaigns, social media, content marketing, etc.) can be easily determined by the accuracy and value of your GA reports.

If you don't take the appropriate measures, unwanted data such as bots, internal traffic, and spam will decrease the accuracy of your reports and lead to poor decisions. The good news is that GA has a powerful filter functionality, that if used well, will help prevent all that junk traffic. The bad news is that in my experience most sites don't use it properly.

99% of the sites I audit either don't use filters at all or they do but not correctly, which makes the problem even worse.

So to help you get data you can trust I will show you:

  1. The most efficient ways of filtering bots, spam, and other junk traffic in your Google Analytics,
  2. And also important, how to do it safely so you don't risk your real user data.

A quick FAQ about this guide

To save you some time looking through the comments, here are the answers to some of the most common questions I get:

  • Which type of spam and bots does this guide cover?
    • Most common ones. The filters will help you understand and stop any type of Google Analytics spam: Ghosts (referral, keyword, language, page, etc.), crawlers and many known bots,
  • Does this work in WordPress, Joomla, Shopify, Wix, Weebly, Squarespace...?
    • Yes. The solutions below are purely based in GA filters, so it will work independently of the platform (CMS) you use.
  • How often do you check for new threats and update the expressions?
    • I'm constantly monitoring for bots and new spam (3-5 times a week) and the expressions are updated as soon as new significant threats are detected, so you can check them frequently or even better you can get notified when new expressions are out. (See the historical spam blacklist)
    Want to get notified of important updates, new threats, and new ways of keeping your Analytics data accurate?

Dos and don'ts when filtering in Google Analytics

Let's go quickly through a list of things you should consider when filtering in GA.

How does Ghost Spam attacks Google Analtyics?
  • Good: A comprehensive solution. A hostname filter that will take care of most of the spam, and a set of filters with optimized expressions to block sneaky referrer spam and bots.

First things first. Protect your data from misconfigurations

Before creating any filter in GA you have to make sure you have at least 2 views, one were you will apply the filters and a second one that you will leave unfiltered, this will work as a backup and to check the progress of your filters.

If you need help creating these views, here you can find the best practices for views in Google Analytics.

5 types of filters to stop spam and bots in Google Analytics

Once your views are correctly configured, is time to stop all of that dirty traffic that skews your reports and don't let you see the real performance of your site.

There is no one all mighty solution that will stop all junk traffic at once, so if you want to have accurate Analytics you will have to work for it.

The filters I'm going to show you have been proven to work for over 4 years regardless of the methods used by spammers and the types of bots.

Need some proof? Here are some examples of users that followed this guide and shared their results with me. Show Examples

The screenshots are from 2016 but the results for your analytics will be the same to this date.

The Google Analytics filters you will need are:

  1. Valid hostname filter for ghost spam (referral, page, keyword spam, language, etc.),
  2. Campaign source filters for crawler referral spam,
  3. Language filter for sneaky spam (and some bots),
  4. ISP organization filters for bot traffic,
  5. ISP domain / Network filter to stop bot traffic,
  6. Extra: Enable the built-in feature "Bot Filtering" (to exclude a few known bots)

Don't have the time? Want me to fix this for you? I can review your analytics and apply all the necessary measures to ensure you are receiving the most accurate data possible.

General notes about filters.

  • While most of the time filters start working within minutes, officially it may take up to 24 hours before the filter effects become visible in your data, so be patient.
  • You will apply the filters either in the master view, the view(s) to be used for analysis, or the test view if you want to try them first.
  • Filters only work forward, for historical data you will use the segment.

a. Valid hostname filter to stop ghost spam

The valid hostname filter is the single most effective solution against the spam. This filter will permanently stop all ghost spam with fake hostnames no matter how it comes or what name it uses.

Google Analytics spam filter
What is a hostname vs a source? expand

People often mistake hostnames with sources. To avoid confusions while preparing the filters, I'll briefly explain the difference:

  • The Source is where your visit comes from and there can be any number of them, for example, Facebook, Google, Twitter, Youtube, links from other sites to your site, etc.
  • The hostname, on the other hand, is the site where the visitor arrives. Your main hostname will be your domain and, and depending on the configuration of your site, there may be others.
Source vs Hostname Google Analytics
To build this filter you will need:
  1. Make a list of your hostnames:
    • To see a list of all the active hostnames you need to go to the Network report in your Analytics:
      • Audience > Technology > Network
    • Change the primary dimension to Hostnames  (blue text at the top of the report)
    • Make a list of all the valid ones you find. You should see at list one valid, which is your main domain, the rest will depend on the configuration of your site.
  2. Build your hostname expression: Once you have the list of all your hostnames, you should put all of them together separating them with a pipe "|" character like this:
    If you need extra help finding your valid hostnames and building your expression check this guide or let me know and I can personally help you.
  3. Create the filter: Once you are sure the expression is correct, create the filter as follows.

How to create a filter to block ghost spam in Google Analytics

To block all ghost spam in Google Analytics, you need to create an include hostname filter:

  1. Go to the Admin tab, and select the view where you want to apply the filter. If you follow the naming above, this will be the Master view or Test view.
  2. Select Filters under the View column, and select + Add Filter
    Add filter button Google Anlaytics
  3. Enter as a name for the filter Include Valid Hostnames.
  4. Configure the filter as follows:
    • Filter Type Custom > Include
    • Filter Field Hostname
  5. In the Filter Pattern box copy the hostname expression that you built before.
    How to filter spam in google analtyics?
  6. You can click on Verify this filter, and you will get a quick glance at how the filter will work. But take into account this feature is limited to a small amount of data. So you might get the following message:

    "This filter would not have changed your data. Either the filter configuration is incorrect, or the set of sampled data is too small"

    But don't worry, if you followed the instructions in the previous step (Build your expression) you already tested your filter, if for some reason you missed it, here it's again how to verify a filter with an in-table filter. ;)

  7. Once you make sure your filter is ok, Save the filter.

IMPORTANT: This filter doesn't require updates for new ghost spam, but it's essential to update the expression whenever you add the tracking ID (UA-00000-1) to a new service or domain.

b. Source filter to stop Crawler referral spam

Crawler spam uses a valid hostname so it is a bit harder to detect. To block it you'll need a filter with an expression that matches the source of all known crawler spam.

To save you some time, I've created a set of optimized regular expressions (REGEX) with all the relevant crawler spam detected over the last years, you'll find them below in the instructions.

How to create a filter to block crawler referrer spam in Google Analytics

To block referrer spam in Google Analytics you will need to create an exclude filter using the campaign source:

  1. Again go to the admin section of your GA.
  2. On the last column "VIEW", select Filters  and then click + Add Filter
    Add filter button Google Anlaytics
  3. Enter as a name for the filter "Exclude Crawler Spam"
  4. Configure the filter as follows:
    • Filter Type select Custom > Exclude
    • Filter Field select Campaign Source (don't use referral field or it won't work)
      How to block referrer spam in Google Analytics?
  5. Filter Pattern > Paste the following crawler referrer spam expression.
    These expressions were re-built to optimize the number of filters. If you created your filter before November 23, 2018, replace all the old expressions and remove any extra filter.

    Create 1 filter for each expression

    Crawler Expression 1

    TOTAL CHARACTERS: 251
    (videos|buttons)\-for\-your|share\-?button|buttons\-for(\-your)?\-website|semalt|ranksonic|timer4web|anticrawler|dailyrank|sitevaluation|forum69|profit\.xyz|checkpagerank|keywords\-monitoring|kings\-analytics|responsive\-test|fix\-website\-|top10\-way

    Crawler Expression 2

    TOTAL CHARACTERS: 231
    seo\-?(watch|services2018|2\-0|analysis|solution|offer)|(10best|auto|express|audit|dollars|success|top1|amazon|commerce|resell|99)\-?seo|(\-liar|pr\-cy)\.ru|uptime(robot|bot|check|\-|\.com)|free\-video\-|:8888|hacĸer|ɢoogl|foxweber

    Crawler Expression 3

    TOTAL CHARACTERS: 213
    (ḷ|\-backlinks)\.com|3\-letter\-|free\-fbook|your\-rankings|be\-escorts|vidyoze|blog1989|incomekey|amazon\-ads\.ovh|krumble\.net|blog100|free\-website\-traffic|mycheaptraffic|bestbaby\.life|(blogping|blogseo)\.xyz

    Crawler Expression 4

    TOTAL CHARACTERS: 63
    (artblog|howblog|seobook|merryblog|axcus|dotmass|artstart|dorothea|artpress|matpre)\.top

    Get free notifications with the updated expressions whenever I detect new crawlers.

  6. After everything is set Save.

Note: You may find other referrals that may not be spam, but neither relevant for you. For example, mobile test sites or cache sites. You can create a similar filter with the same configuration and add all the irrelevant referrals there to keep your data pristine and reliable.

Now that you are familiar with the filter window I won't repeat the full instructions on the following filters. To create them you will need to follow the exact same steps of the previous 2 filters and change the following fields:

  • Filter name
  • Filter field
  • Filter expression

c. Language filter for sneaky crawlers and bots

From time to time you may see weird languages showing in your analytics. I prepared an expression that will prevent any language that doesn't have a proper format like es-ESen-US, fr-FR, etc.

I also added to the expression the "Language c" which seems to be left by bots.

  • Create a new filter with the following settings:
    • Filter name: Exclude invalid languages
    • Filter configuration:
      • Filter typeCustom > Exclude
      • Filter field: Language Settings
      • Filter pattern: enter the following expression as it is:
        \s[^\s]*\s|.{15,}|\.|,|^c$
        spam and language c google analytics

d. ISP organization / Service provider filter to stop bot traffic

Not all irrelevant traffic comes from spammers, some companies use bots to crawl sites for different purposes (indexing, analytics, etc.). Those bots may not have bad purposes but they still inflate your reports.

For example, Google bots coming from their corporate ISP "Google LLC":

google llc and the referral 127.0.0.1.8888 in Google Analyitcs
Or if you use Facebook or Instagram Ads you may receive bot traffic from their corporate site "Facebook Ireland ltd" with the following characteristics:
  • Screen resolution: 2000x2000
  • Country: Peru, Philipines, United States
  • City: (not set), Quezon City, Manila
  • Service provider: Facebook Ireland Ltd

List of corporate sites and ISPs with high usage of bots:

Common ISP providers using bots:
facebook ireland ltd google llc google inc.
alibaba.com llc ovh hosting inc. microsoft corp
microsoft corporation hubspot evercompliant ltd.
kazooisyee google switzerland gmbh google corporate
     

Note: These are just a few examples of ISP with high bot activity, the expression below contains more and it is constantly updated.

  • Create a new filter with the following settings:
    • Filter NameExclude ISP provider bots
    • Filter configuration:
      • Filter type: Custom > Exclude
      • Filter fieldISP organization
      • Filter pattern: enter the following expressions as they are:

        IMPORTANT

        I extensively test the expressions below across dozens of GA properties to avoid interference with real user data. However, in very few cases the expressions could match some real user data.

        For example, the expression below blocks the ISP "Google llc" and "Microsoft corp", which are ISP used in these companies offices.

        In most cases the visits with those ISPs come from bots, however, they can also be used by employees, so if your site sells a product to those companies, just remove them from the expression. If you are not sure you can test the expressions before applying them to your GA.

        ISP Expression 1

        TOTAL CHARACTERS: 255
        hubspot|^google\sllc$|^google\sinc\.$|alibaba\.com\sllc|ovh\shosting\sinc\.|microsoft\scorp|facebook\sireland\sltd|online\ssas|evercompliant|early\sregistration\saddresses|inktomi\scorporation|google\scorporate|google\sswitzerland\sgmbh|kazooisyee|cloud69

        ISP Expression 2

        TOTAL CHARACTERS: 57
        amazon\sdata\sservices|vultr\sholdings|hos\-329450

        ISP Expression TEMP

        TOTAL CHARACTERS: 237
        internet\ssecurity\s\-|secure\sinternet\sllc|versia\sltd|altushost\ssweden\snetwork|web4africa\s\-ng|altushost\sluxembourg\snetwork|gz\ssystems\slimited\s\-|gz\ssystems\slimited\s\-|north\sstar\sinformation\shi\.tech|putian\scity\sfujian
        Filter google llc and 127.0.0.1.8888 in Google Analytics

e. ISP domain/network domain filter to stop bot traffic

This filter is similar to the previous one but this time we will target the ISP domain or network.

  • Create a new filter with the following settings:
    • Filter Name: Exclude ISP domain bots
    • Filter configuration:
      • Filter type: Custom > Exclude
      • Filter fieldISP domain
      • Filter pattern: enter the following expression as it is

        paloaltonetworks|scaleway|kcura

This filter covers the weird amazon keywords from Bing organic traffic with network domain paloaltonetworks.com

f. Extra: Enabling "Exclude all hits from known bots and spiders"

This is a pre-built filter that will take care of known bots from the IAB bots and spiders list, it is not perfect but it helps.

In this case, is a bit easier than custom filters because you just need to check a box.

How to enable bot filtering

  1. Again in the Admin section of your Analytics, select your Master view under the VIEW column. (Also for any other filtered view)
  2. Click View Settings
    how to block known bots in Google Analytics
  3. Near the bottom check the box Exclude all hits from known bots and spiders (Bot Filtering)
    Exclude all hits from known bots and spiders
  4. Save and repeat the process with all your Views

What's next? Get even more value from Analytics data

  1. Clean spam from past data: The filters above will prevent future hits, here you can find instructions to clean spam from your historical data.
  2. Exclude internal traffic: This type of junk traffic is often overlooked. If you don't apply filters for the traffic generated by you or other people of your team, this data will get mixed up with your real visits data, and a difference with the spam, this is much harder to identify later.

Additional resources

Wrapping it up

Whether you are a blogger, a small local website, or a multinational company, filtering your data is crucial for the accuracy of your reports.

"Even on high volume websites were data spamming would be marginal, you still have to explain why there's such a discrepancy. As an analyst you can't dismiss it simply by saying "nah... we're not too sure what it is..."

-Stéphane Hamel

However, you have to do it right. Handling each spammer individually is time-consuming and inefficient. The Google Analytics spam filters explained in this guide may take a bit longer to configure but they will save you a lot of time in the long run.

I will be updating this guide as new threats appear so you can keep it as a reference.

Do you have any questions or feedback?

I've tried to cover every important detail in this guide, however, if there is any part of the guide where you got stuck, let me know in the comments section below.

If this article helped you, consider leaving a comment below with your experience, it may help others! :)need help implementing, configuring, and/or protecting your Google Analytics? I can help

Need help setting up a robust and reliable Google Analytics reporting for your website/business?

  • Filters for data quality
  • User interaction tracking (events, goals)
  • E-commerce tracking
  • GDPR compliance
  • Google Tag Manager implementation
  • Integrations (Google Ads, Search Console, etc)
  • Custom reports (Dashboards, Data studio)
  • Monthly reporting and more...
Be the first to comment :)