the-tls.com
robots.txt

Robots Exclusion Standard data for the-tls.com

Resource Scan

Scan Details

Site Domain the-tls.com
Base Domain the-tls.com
Scan Status Ok
Last Scan2025-09-23T18:57:24+00:00
Next Scan 2025-09-30T18:57:24+00:00

Last Scan

Scanned2025-09-23T18:57:24+00:00
URL https://the-tls.com/robots.txt
Redirect https://www.the-tls.com/robots.txt
Redirect Domain www.the-tls.com
Redirect Base the-tls.com
Domain IPs 34.240.28.43, 52.208.17.106, 54.76.240.177
Redirect IPs 13.35.37.100, 13.35.37.116, 13.35.37.59, 13.35.37.87
Response IP 13.35.37.100
Found Yes
Hash dd9b35bd7620ebf2b74cbb3d0cb2a8898bc1636b4dcb72cda5be9bfb5ab2dd37
SimHash 402cc8c2a99f

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /search/
Disallow /?s=

newsnow

Rule Path
Disallow /

omgili

Rule Path
Disallow /

webvac

Rule Path
Disallow /

webzip

Rule Path
Disallow /

psbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

*

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.the-tls.com/sitemap_index.xml
sitemap https://www.the-tls.com/sitemap_index.xml

Comments

  • Agent Specific Disallowed Sections
  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK