Log In Sign Up

schaeffer.com
robots.txt

Robots Exclusion Standard data for schaeffer.com

Resource Scan

Scan Details

Site Domain schaeffer.com
Base Domain schaeffer.com
Scan Status Ok
Last Scan2024-09-20T06:07:51+00:00
Next Scan 2024-09-27T06:07:51+00:00

Last Scan

Scanned2024-09-20T06:07:51+00:00
URL http://schaeffer.com/robots.txt
Redirect https://www.schaeffersresearch.com/robots.txt
Redirect Domain www.schaeffersresearch.com
Redirect Base schaeffersresearch.com
Domain IPs 52.204.54.73
Redirect IPs 52.204.54.73
Response IP 52.204.54.73
Found Yes
Hash 9bb06ccb6a8b765751e46dfc4756227cc56e99f499e5e7c3a13702a41d2a8429
SimHash 38d519c3d552

Groups

*

Rule Path
Disallow /*.pdf$
Disallow /streetools/
Disallow /streetools/*
Disallow /Forms/*
Disallow /commentary/content/marketobs/*
Disallow /search/
Disallow /search/*

adsbot-google

Rule Path
Disallow /*.pdf$
Disallow /streetools/
Disallow /Forms/*
Disallow /commentary/content/marketobs/*
Back to top

Other Records

Field Value
sitemap https://www.schaeffersresearch.com/sitemap/sitemap.gz
Back to top

Comments

  • Start of robots.txt file
  • This is for everybody else
  • End of robots.txt file
Back to top

Warnings

  • `noindex` is not a known field.
Back to top