- My website has been scanning for more than 24 hours
- The scanner says I have more than 50 pages
- Message "Under construction. The website is currently being scanned for the first time" appears
- Message "New scan initiated - waiting for available scanner" appears
- Scanner finds cookies that I can not see on my website
- Difference between number of cookies listed and how many cookies appear in the description
My website has been scanning for more than 24 hours
If you have just signed up or just added a new domain to your Cookiebot CMP account, you may see a message saying that the system is waiting for the next available scanner. A scan should start within 30 minutes.
Please allow for up to 24 hours for the full scan to complete. We need to scan your website in-depth and we also scan slowly so that it does not have a negative impact on your website's performance.
If the status says "scanning" and it has been for more than 24 hours, this could be caused by any of the following reasons:
- The website was down for maintenance during the time of the scan
- The website responded with an error during the scan
- Issues with the SSL certificate
- Security settings blocking the scanner from accessing the website
- Recaptcha blocking the scanner from accessing the website
If one of the first 3 reasons could have been the cause of the scan not resulting in a new report, you can fix this by starting a new scan. The scan status should change to:
Sometimes we come across websites we are not able to scan. Typically this is due to the website using one or more automated scanning prevention measures or sensitive website security systems. Most common are the use of Recaptcha or firewalls blocking traffic.
Solution: You will need to whitelist the IP addresses used by the Cookiebot Scanners in order for us to be able to scan your website.
More about Recaptcha
Your domain is using a Bot Traffic Prevention System designed to keep robots from scanning and indexing your website. This is affecting our ability to complete a scan and identify the cookies and tracking mechanisms used on your website. We encourage your to whitelist our servers IP addresses in order for us to scan your website uninterrupted.
Please visit the help pages for your captcha software to find out how. If your software supports a token based solution to circumvent this issue, please contact our support department for assistance.
If that is not possible, a removal of the BTPS would be needed to ensure Cookiebot's service is working as intended.
The scanner says I have more than 50 pages
If you have received a price quote from Usercentrics A/S: attached to the email you can find a URL list of up to 5,000 of the pages found by our scanner.
If you have received an email saying your account has been upgraded from a free subscription to a 14-day free trial: attached to the email you can find a URL list of up to 200 of the subpages found by the Cookiebot scanner.
If you have signed up for a Cookiebot CMP subscription you can also find details and an URL list with up to 10,000 of the subpages that our scanner has found on the 'Cookies' page:
- Log into your Cookiebot CMP account
- Navigate to the ‘Cookies’ tab (and select the desired domain)
- Click on the link containing the number of pages, the URL list of pages will start downloading automatically.
Message "Under construction. The website is currently being scanned for the first time" appears
If you have recently created a Cookiebot CMP account and added a domain or if you have added an additional domain to an existing account, the domain will be scanned for cookies and trackers for the first time. Please allow for up to 24 hours for the scan to complete.
During this scan, the message 'Under construction. The website is currently being scanned for the first time' will be displayed on both the cookie consent banner/dialog and on the Cookie Declaration. As soon as the scan is complete, the message will disappear and the cookie information will appear automatically.
If you see this message on your consent banner or declaration, you can log in your Cookiebot manager and navigate to the "Cookies" page to check the status of the scan:
If the status says "scanning" and it has been for more than 24 hours, this could be caused by any of the following reasons:
- The website was down for maintenance during the time of the scan
- The website responded with an error during the scan
- Issues with the SSL certificate
- Security settings blocking the scanner from accessing the website
- Recaptcha blocking the scanner from accessing the website
If one of the first 3 reasons could have been the cause of the scan not resulting in a new report, you can fix this by starting a new scan. The scan status should change to:
This new scan should result in a new scan report and updated banner and declaration information within 24 hours. If that is not the case, we might need to investigate the issue and you can reach out to us via the support form.
If you suspect the scanner is not allowed to access the website, due to security settings your CMS may have installed, please see our guide on whitelisting the scanner.
Message "New scan initiated - waiting for available scanner" appears
If you have just signed up for a Cookiebot CMP account or if you have just added a new domain, you will initially see the message saying 'New scan initiated - Waiting for available scanner...'. This usually means that the scan will start within app. 30 minutes and then please allow up to 24 hours for the scan to complete.
If you experience that this message stays the same for much longer than the 30 minutes, the most common reason is that the domain is not responsive. In this case, the scanner will keep trying up to 5 times in 12-hour intervals. Eventually, after the failed attempts, the below message will be displayed. The scanner will not automatically try again until the next monthly, planned scan.
Things you can check in this case:
- Please check the spelling of your domain name
- Check whether there are any issues with your domain. The scanner impersonates a number of website users visiting your website so if the domain is unresponsive to the scanner, it is likely also unresponsive to a regular website user.
- Check whether there are issues with the SSL certificate on the website
- Check whether your firewall settings might be blocking our scanner from crawling the website
- Check whether you have Recaptcha enabled
If you suspect the scanner is not allowed to access the website, due to security settings your CMS may have installed, please see our guide on whitelisting the scanner.
If you are unable to find any reasons the scanner can not access your domain, feel free to reach out to us and we will be happy to have a look.
Scanner finds cookies that I can not see on my website
The Cookiebot scanner scans for a number of different types of cookies and online trackers from the ‘classical’ HTTP/JavaScript cookies to more advanced tracking technology such as pixel tags and ultrasound beacons. This includes both 1st party and 3rd party cookies and trackers.
See also: What kinds of cookies and tracking technologies does the Cookiebot scanner find?
The Cookiebot scanner has been developed and refined over the past 6 years and that is why it is able to pick up on all these types of cookies and trackers.
1. How does the scanner find the cookies and trackers?
The Cookiebot scanner crawls your website – typically over 24 hours – to look for all the cookies and tracking technology in use. It simulates a number of regular website users visiting your website. You can think of it as a simulation of say e.g. 7-8 users simultaneously visiting the website and going through all the subpages, clicking all the links, menu points and buttons, playing embedded videos and taking all the actions possible – in other words doing everything that it is possible for a website visitor to do on the website being scanned while at the same time picking up on cookies and trackers in use. The only thing the scanner does not do is fill in forms (like actually subscribing to your newsletter) and paying for goods in a shopping cart (if you have a webshop, the scanner will put items in the shopping cart but will not actually proceed and pay for those items). The results are presented in the scan report.
2. If the scanner is simulating a regular website user then why can’t I see the same cookies in my browser?
The browsers of today are simply not advanced enough to see all the cookies and trackers being set.
Also, there is a difference between what cookies for example Safari and Chrome are able to see. So, if you look for cookies listed in the developer console (or similar) on your browser, chances are that you will see different results in different browsers. Also, browsers cannot register the cookies for the *entire* website but only show (some of the) cookies used on the particular page you are currently visiting. At best, if you are checking the cookies via your browser console, you are only seeing a partial picture of the cookies and trackers in use on your website.
It is also important to understand that many cookies are so-called dynamic cookies. These – unlike ‘classical’ HTTP/JavaScript cookies – are not being set when the website is loaded. They are being set during the visit depending on the user’s behavior on your website.
Firefox is sometimes able to display some of these cookies if you click around on the webpage, hit ‘refresh’ etc. However, since they are triggered based on (unknown) patterns of behavior it can be very difficult to recreate the behavior so that they are shown in the browser console.
3. If I want to delete or inspect some of the cookies and trackers listed in your scan report, how can I find them?
As a help for you as a website owner, we have included additional information in the scan report. For each cookie or tracker identified, you will be able to see where on your website it was first found by our scanner (First found URL), what type it is and – where applicable – in what line of the source code the script setting the cookie is located. The source is also indicated (in the example below it is coming from Google Tag Manager (GTM) because implementation has been done via GTM).
Example of a cookie listed in the scan report.
The scan report is being sent to you by email after each monthly scan is completed and is also available from the menu point ‘Reports’ on your Cookiebot CMP account.
Cookies and trackers are set either client- or server side:
For client side cookies:
Cookies that are set client side are generally listed in the scan report indicating the line in the source code where it is found. You can then inspect that line of your source code and either remove or mark up the cookie script.
Cookies that are dynamic – as described above – will be identified by the Cookiebot scanner but will not be listed with a corresponding line in the source code. However, the dynamic cookies are being set by one of the other scripts setting regular/static cookies. Therefore, if you make sure to mark up those cookies that have an indication of the line in the source code, then that marking up will also automatically control the dynamic scripts that are generated based on a static cookie.
If you cannot locate a cookie based on the information in the scan report, as a last resort you can right click on your website and choose ‘inspect’ – then search for the name of the cookie or the provider to see if you can locate it. This will not work for dynamic cookies, however.
For server side cookies:
Most of the server side cookies being set are ‘strictly necessary’. In the context of Cookiebot, they can therefore be ignored and do not need to be handled further.
If you have a server side cookie which the scan report lists as ‘statistics’ ‘preferences’ or ‘marketing’, then you need to take action.
On the scan report, the cookie will likely be listed as ‘Initiator: Webserver’.
A cookie of this type cannot be marked up following the usual procedure (step 3 of our 3-step installation guide). Instead you need to check the user’s current consent state to determine if you can set the cookie or not. This must be handled as described in our developer section – see under ‘Server side usage’ at the bottom. Please note that there are 3 separate tabs for C#, PHP and VB.
4. The scanner finds cookies that are not set on my website
Sometimes, websites redirect users to other external websites. The scanner normally will not follow links that point to 3rd party domains. However, if you make use of masked URLs* so that it appears to the user that the user is still on your website, then the Cookiebot scanner will include these cookies in the scan report in order to provide transparency of the cookies and trackers in use.
* Masked URLs appear as links to pages on the same domain, which in fact lead to a third party domain.
For example: www.first-domain.com/12345 actually points to www.second-domain.com.
A visitor would not be able to tell this from looking at the link, and thus the scanner follows the link, scans the page it landed on, but immediately returns to the original domain without scanning the entire second domain.
An example: On your website are some icons for social media. If the user clicks one of the icons, the user will be taken to the website of the provider of that social media. However, you have masked this in a way so that your own URL is still visible to the user and so the user does not understand that he has in fact navigated to a page outside of your website. The cookies being set – in this case social media related cookies – are included in the scan report.
If you would like for these cookies to not be included in the scan report, then please unmask the URL and make it transparent to the user that the user has navigated to an external page, e.g. a social media website, that is not part of your website.
You can also read our blog post How do websites track users? | Technologies and methods | GDPR Compliance
Difference between number of cookies listed and how many cookies appear in the description
A single cookie can have multiple entries (i.e. be used multiple times on your website and by different providers). It will, however, only be shown once in the description and with the count in brackets. See the example below:
Example: Number of preference cookies listed as 3. Two different preference cookies listed and described - one of them with the count in brackets [x2].
Comments
0 comments
Please sign in to leave a comment.