ftwtoday.6amcity.com
robots.txt

Robots Exclusion Standard data for ftwtoday.6amcity.com

Resource Scan

Scan Details

Site Domain ftwtoday.6amcity.com
Base Domain 6amcity.com
Scan Status Ok
Last Scan2025-02-20T15:08:39+00:00
Next Scan 2025-03-22T15:08:39+00:00

Last Scan

Scanned2025-02-20T15:08:39+00:00
URL https://ftwtoday.6amcity.com/robots.txt
Domain IPs 3.165.75.125, 3.165.75.2, 3.165.75.30, 3.165.75.64
Response IP 3.165.75.64
Found Yes
Hash 79ab046109b5d392d4b6bd86e23db82d1b620ae53c812e2a04f63132b65987ec
SimHash 2a08d840a097

Groups

*

Rule Path
Disallow /cms/
Disallow /_debug/
Disallow /_preview
Disallow /cms/preview

Other Records

Field Value
crawl-delay 10

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://ftwtoday.6amcity.com/sitemap.xml