Identifying and Managing Bot Traffic in GA4
To exclude known bots and spiders from Universal Analytics data you needed to edit each view within your UA property and select “Exclude all hits from known bots and spiders”.
This went some way to removing bot traffic but there were occasionally issues with spam and ghost referrals (the latter where spammers used the UA measurement protocol to send data to accounts) that required the addition of other exclusion filters.
Within GA4, known bots and spiders are excluded from your data automatically. Known bot traffic is identified using both Google research and lists managed by the Interactive Advertising Bureau. GA4 also tightens up the measurement protocol with a ‘secret key’ now required to send data to your account.
At the moment though it’s not possible to add other filters to exclude bot traffic or to see how much data has been excluded from your account.
How can you identify bot traffic in GA4?
The first signs of a bot hitting your site may well be picked up by anomalies in your data. You may see sudden spikes in traffic or a drop off in engagement rates from certain channels.
Sudden spikes in direct or unassigned traffic can often be a warning sign – particularly if they are accompanied by a low number of engaged sessions or average engagement time. Check the traffic source / medium for suspicious referrals that are suddenly sending large volumes of traffic – these sites may end up in a blocked list but can cause real problems in the meantime.
If you suspect a bot is hitting your site, use the Explore tools in GA4 to investigate traffic patterns further. Things to check out include:
Was there a sudden one-off peak in traffic from a particular source?
Does the traffic anomaly repeat at a regular interval? (e.g. every 6 hours)
Where geographically is the traffic coming from – is it the same countries / cities and are the user locations unexpected?
What pages is the traffic landing on – is it all the home page and are there unusual query strings associated with the page views?
Are the browser, device type and screen size the same?
What can you do about bot traffic?
So having identified a potential bot traffic problem what can you do about it?
Unfortunately in GA4 itself options are limited in terms of filters where, at the time of writing, you can only add IP related filters. Some potential options to consider include:
Ask your developers to check the server logs to see if they can identify any suspicious activity from particular IP addresses. If any are identified explore whether these can be blocked at a server level or should be added to an IP exclusion filter in GA4.
Explore third party bot blocking tools
Create a robots.txt file in your website's root directory to instruct legitimate search engine bots which pages they can and cannot crawl. While this won't block malicious bots, it can help guide legitimate ones
If you can identify suspicious traffic coming from a particular region or combination of technical dimensions (screen size, device type etc) you may be able to use segments in GA4 Explore reports or add filters to Looker Studio to exclude this data. Caution is advised in case you end up excluding legitimate traffic too.
If you manage Google Analytics via Google Tag Manager then there may potentially be options to exclude this traffic at a tracking level before it hits your account – although this may potentially get complex.
Keep an active eye on your data and consider setting up automated anomaly detection email alerts. Alerts for ‘Anomaly in daily users” and “Anomaly in daily views” can be helpful in identifying issues.
Looking for a Google Analytics agency to help with a bot issue? Email us at info@andersanlaytics.com to find out more.
