stageandstylebyleigh.ca
robots.txt

Robots Exclusion Standard data for stageandstylebyleigh.ca

Resource Scan

Scan Details

Site Domain stageandstylebyleigh.ca
Base Domain stageandstylebyleigh.ca
Scan Status Ok
Last Scan2024-10-04T03:13:43+00:00
Next Scan 2024-10-05T03:13:43+00:00

Last Scan

Scanned2024-10-04T03:13:43+00:00
URL https://stageandstylebyleigh.ca/robots.txt
Domain IPs 192.0.78.155, 192.0.78.248
Response IP 192.0.78.155
Found Yes
Hash a17ca55fde3aef792ccd4ad8ba4d51cc6b3bbfc125048fc9f27ae335b99a415d
SimHash 97bcd1600fa2

Groups

*

Rule Path
Allow
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin/
Disallow /readme.html$

Other Records

Field Value
crawl-delay 0

mediapartners-google

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

adsbot-google

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

proximic

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

criteobot/0.1

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

grapeshot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

semrushbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

blexbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

serpstatbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

dataforseobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

omgilibot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

barkrowler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

webreaper

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

msiecrawler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

webcopier

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

httrack

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

velenpublicwebcrawler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

awariobot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

awariorssbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

awariosmartbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

amazonbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 0

Comments

  • BEGIN Magic robots.txt
  • ---------------------------
  • General
  • Ad networks
  • Link analyzers
  • Downloaders
  • ---------------------------
  • END Magic robots.txt