worldbookday.com
robots.txt

Robots Exclusion Standard data for worldbookday.com

Resource Scan

Scan Details

Site Domain worldbookday.com
Base Domain worldbookday.com
Scan Status Ok
Last Scan2024-09-24T21:26:41+00:00
Next Scan 2024-10-24T21:26:41+00:00

Last Scan

Scanned2024-09-24T21:26:41+00:00
URL https://worldbookday.com/robots.txt
Redirect https://www.worldbookday.com/robots.txt
Redirect Domain www.worldbookday.com
Redirect Base worldbookday.com
Domain IPs 104.21.30.106, 172.67.172.193, 2606:4700:3030::ac43:acc1, 2606:4700:3037::6815:1e6a
Redirect IPs 104.21.30.106, 172.67.172.193, 2606:4700:3030::ac43:acc1, 2606:4700:3037::6815:1e6a
Response IP 172.67.172.193
Found Yes
Hash ddcade2072b284c5213b2bc5596b98b7806fd33f94408c02232bb21a43f5aa3a
SimHash 9b2bc420bc99

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /?s=
Disallow /search/
Disallow /no-access/
Disallow /wp-admin/
Disallow /wp-json/
Disallow /wp-admin/admin-ajax.php

nuclei
wikido
riddler
petalbot
zoominfobot
go-http-client
node/simplecrawler
cazoodlebot
dotbot/1.0
gigabot
barkrowler
blexbot
magpie-crawler
semrushbot
mj12bot
ahrefsbot
yandexbot
petalbot
dotbot
blexbot
dataforseobot
zoominfobot

Rule Path
Disallow /%60

Other Records

Field Value
sitemap https://www.worldbookday.com/sitemap_index.xml

Comments

  • Ban bots that don’t benefit us.
  • ——————————–