british-casino-bonuses.com
robots.txt

Robots Exclusion Standard data for british-casino-bonuses.com

Resource Scan

Scan Details

Site Domain british-casino-bonuses.com
Base Domain british-casino-bonuses.com
Scan Status Ok
Last Scan2025-04-01T17:19:31+00:00
Next Scan 2025-05-01T17:19:31+00:00

Last Scan

Scanned2025-04-01T17:19:31+00:00
URL https://british-casino-bonuses.com/robots.txt
Domain IPs 104.21.64.171, 172.67.187.48, 2606:4700:3031::ac43:bb30, 2606:4700:3036::6815:40ab
Response IP 104.21.64.171
Found Yes
Hash be2eb9e3c627433d7febb6df826dc952a1a6c52c5b7fe17bde6a3b22d203cc3a
SimHash d71c5c428ec9

Groups

*

Rule Path
Disallow /xmlrpc.php
Disallow /wp-login.php
Disallow /trackback/
Disallow /offers/*
Disallow /go-store/*
Disallow /go/*
Disallow /feed/
Disallow /go/*
Disallow /get/page/*
Disallow /bonus-category/online-casinos/page/*
Disallow /play-now/*
Disallow /claim/*
Disallow /registration/*
Disallow /bwin.php*
Disallow /playnow.php*
Disallow /goo/*
Disallow /go-store/*
Disallow /r/*
Disallow /c/*
Disallow /fly/*
Disallow /content/go/*

meta-externalagent

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

awariobot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://british-casino-bonuses.com/sitemap.xml