tagtaly.com
robots.txt

Robots Exclusion Standard data for tagtaly.com

Resource Scan

Scan Details

Site Domain tagtaly.com
Base Domain tagtaly.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2026-01-06T20:01:41+00:00
Next Scan 2026-01-13T20:01:41+00:00

Last Successful Scan

Scanned2025-12-29T20:01:14+00:00
URL https://tagtaly.com/robots.txt
Domain IPs 2a02:4780:84:59d6:f73e:a9df:e361:4b6, 2a02:4780:84:6461:7b5e:6502:76e9:c332, 84.32.84.18, 84.32.84.193
Response IP 93.127.201.245
Found Yes
Hash 913e975f7134ac3ff665649473bc50da9fa9892720697337a39e80b9426f2d2d
SimHash 698412d089a4

Groups

*

Rule Path
Allow /
Disallow /debug_share.html
Disallow /fresh.html
Disallow /test_*.html
Allow /blog/
Allow /guides/
Allow /methodology/
Allow /resources/
Allow /faq/
Allow /glossary/
Allow /case-studies/

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap https://tagtaly.com/sitemap.xml
sitemap https://tagtaly.com/blog/feed.xml
sitemap https://tagtaly.com/blog/daily.xml
sitemap https://tagtaly.com/blog/weekly.xml

Comments

  • Tagtaly - News Analysis Platform
  • Crawl delay
  • Sitemaps
  • Disallow test and debug pages
  • Allow specific content