thetab.com
robots.txt

Robots Exclusion Standard data for thetab.com

Resource Scan

Scan Details

Site Domain thetab.com
Base Domain thetab.com
Scan Status Ok
Last Scan2024-11-15T09:01:15+00:00
Next Scan 2024-11-22T09:01:15+00:00

Last Scan

Scanned2024-11-15T09:01:15+00:00
URL https://thetab.com/robots.txt
Domain IPs 104.22.14.190, 104.22.15.190, 172.67.27.3, 2606:4700:10::6816:ebe, 2606:4700:10::6816:fbe, 2606:4700:10::ac43:1b03
Response IP 172.67.27.3
Found Yes
Hash 73ca86aea3e04472823dbd49e36f4de4f219fd3239d8528e12503baa058c3964
SimHash eb3c88d0f43b

Groups

*
ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 6

Other Records

Field Value
sitemap https://thetab.com/sitemap_index.xml

Comments

  • START YOAST BLOCK
  • ---------------------------
  • ---------------------------
  • END YOAST BLOCK