tubesexcity.com
robots.txt

Robots Exclusion Standard data for tubesexcity.com

Resource Scan

Scan Details

Site Domain tubesexcity.com
Base Domain tubesexcity.com
Scan Status Ok
Last Scan2024-06-03T09:20:37+00:00
Next Scan 2024-07-03T09:20:37+00:00

Last Scan

Scanned2024-06-03T09:20:37+00:00
URL https://tubesexcity.com/robots.txt
Redirect http://www.tubesexcity.com/robots.txt
Redirect Domain www.tubesexcity.com
Redirect Base tubesexcity.com
Domain IPs 104.21.234.252, 104.21.234.253, 2606:4700:3038::6815:eafc, 2606:4700:3038::6815:eafd
Redirect IPs 104.21.234.252, 104.21.234.253, 2606:4700:3038::6815:eafc, 2606:4700:3038::6815:eafd
Response IP 104.21.234.253
Found Yes
Hash e703c17287d5be4cd3bf46bb921f8485b271a300207d440acee3004f90c64eb2
SimHash a21cdc006289

Groups

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin
Disallow /new/
Disallow /long/
Disallow /search/

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10