The Cookiebot scanner crawls your website – typically over a time of up to 24 hours – to look for all the cookies and tracking technology in use.
In order to have the least possible impact on your website's performance, the scanner only sends one request every 2 seconds.
The scanner simulates a number of regular website users visiting your website. You can think of it as a simulation of say e.g. 7-8 users simultaneously visiting the website and going through all the subpages, clicking all the links, menu points and buttons, playing embedded videos and taking all the actions possible – in other words doing everything that it is possible for a website visitor to do on the website being scanned while at the same time picking up on cookies and trackers in use. The only thing the scanner does not do is fill in forms (like actually subscribing to your newsletter) and pay for goods in a shopping cart (if you have a webshop, the scanner will place items in the shopping cart but will not actually proceed and pay for those items).
The scans are done that way to ensure that all cookies and trackers that a regular user could possibly encounter on your website are found.
What can I do to prevent this spike in traffic?
There is no other way to do the scans, so you cannot prevent the traffic from happening. You can, however, exclude the traffic from your regular website statistics by either excluding the provider 'microsoft corporation' or by excluding the IP addresses we scan from: Whitelist: what IP addresses do you scan from?. More information about how to exclude certain IP addresses from Google Analytics can be found in Google's own Analytics support area.
Last updated: 22 June 2018
Comments
6 comments
Are these IPs up to date? I blocked them on Google Analytics via a filter but I still see a spike in traffic from your scan.
Seems like they changed the IP adresses? The listed IP's above worked up untill october.
Please update!
I haven't noticed another spike in Google Analytics traffic. The following filter is what I use to block Cookiebot's scanners - formatted to accommodate IP Anonymization in Google Analytics, which sets the last octet of user IP addresses to zero.
^13\.74\.44\.|^40\.91\.211\.|^52\.232\.29\.|^23\.100\.63\.
Until last month IP filters worked well in Google Analytics and the Cookiebot scanner crawling was excluded from the traffic of the website.
Yesterday, during the monthly scan, the activity was tracked and there was a spike on page visualization.
Are Cookiebot IP adresses changed?
I just checked my Google Analytics - I didn't see any spikes from Cookiebot during my monthly scan that happens on the 21st of the month. I'll report back if I see any spikes.
The "ISP Provider" and "ISP Domain" fields have been deprecated by Google Analytics since February - https://www.seerinteractive.com/blog/deprecating-network-domain-service-provider/
Please update your documentation because "excluding the provider 'microsoft corporation'" will no longer do anything.
Please sign in to leave a comment.