amtamassage.org
robots.txt

Robots Exclusion Standard data for amtamassage.org

Resource Scan

Scan Details

Site Domain amtamassage.org
Base Domain amtamassage.org
Scan Status Ok
Last Scan2024-11-15T19:08:28+00:00
Next Scan 2024-12-15T19:08:28+00:00

Last Scan

Scanned2024-11-15T19:08:28+00:00
URL https://amtamassage.org/robots.txt
Redirect https://www.amtamassage.org/robots.txt
Redirect Domain www.amtamassage.org
Redirect Base amtamassage.org
Domain IPs 65.52.56.100
Redirect IPs 65.52.56.100
Response IP 65.52.56.100
Found Yes
Hash 39299807a04cf01b38d2f0654fbb659e0446d6ef5513a54ee42e3aeea9ad33f1
SimHash c83ddb50e7b3

Groups

baiduspider

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gsa-crawler
*

Rule Path
Allow /
Allow /globalassets/images/
Allow /globalassets/logo.png
Disallow /globalassets/
Disallow /search/
Disallow /account/creation/
Disallow /join/supporting-member-soothe
Disallow /join/amta-benefits-and-resources
Disallow /join/supporting-membership-massage-heights
Disallow /join/join-amta-supporting-member-employee
Disallow /famt/contact/
Disallow /fams/contact/
Disallow /news_publications/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.amtamassage.org/sitemap.xml
sitemap https://www.amtamassage.org/sitemap.xml

Comments

  • directed to all other spiders
  • Sitemap files