Log In Sign Up

calligraphys.com
robots.txt

Robots Exclusion Standard data for calligraphys.com

Resource Scan

Scan Details

Site Domain calligraphys.com
Base Domain calligraphys.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-27T16:54:47+00:00
Next Scan 2025-11-26T16:54:47+00:00

Last Successful Scan

Scanned2025-10-03T13:35:43+00:00
URL https://calligraphys.com/robots.txt
Domain IPs 104.21.78.33, 172.67.215.98, 2606:4700:3033::ac43:d762, 2606:4700:3037::6815:4e21
Response IP 104.21.78.33
Found Yes
Hash 765c41f4660837a42423a883a4ace743c14533b7249f248102e823887958d8b7
SimHash 2010fa504591

Groups

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/

googlebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

googlebot-image

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

mediapartners-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

adsbot-google

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

ia_archiver

Rule Path
Disallow /
Back to top

Comments

  • Global rules
  • Google bots (Google does NOT support Crawl-delay, manage via Search Console)
  • Bing / Yahoo / MSN (support Crawl-delay)
  • Yandex (supports Crawl-delay)
Back to top