the-tartan.org
robots.txt

Robots Exclusion Standard data for the-tartan.org

Resource Scan

Scan Details

Site Domain the-tartan.org
Base Domain the-tartan.org
Scan Status Ok
Last Scan2024-11-15T18:37:21+00:00
Next Scan 2024-12-15T18:37:21+00:00

Last Scan

Scanned2024-11-15T18:37:21+00:00
URL https://www.the-tartan.org/robots.txt
Redirect https://the-tartan.org/robots.txt
Redirect Domain the-tartan.org
Redirect Base the-tartan.org
Domain IPs 192.0.78.142, 192.0.78.250
Redirect IPs 192.0.78.142, 192.0.78.250
Response IP 192.0.78.142
Found Yes
Hash 04cc5b574c97c352c4b97996b8b163a3b489a7ad71db90cffb64dbeee73e7c7b
SimHash 6a1048428036

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

amazonbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://the-tartan.org/sitemap.xml
sitemap https://the-tartan.org/news-sitemap.xml