secretsearchenginelabs.com
robots.txt

Robots Exclusion Standard data for secretsearchenginelabs.com

Resource Scan

Scan Details

Site Domain secretsearchenginelabs.com
Base Domain secretsearchenginelabs.com
Scan Status Ok
Last Scan2024-06-14T03:51:15+00:00
Next Scan 2024-06-21T03:51:15+00:00

Last Scan

Scanned2024-06-14T03:51:15+00:00
URL http://secretsearchenginelabs.com/robots.txt
Redirect http://www.secretsearchenginelabs.com/robots.txt
Redirect Domain www.secretsearchenginelabs.com
Redirect Base secretsearchenginelabs.com
Domain IPs 2600:3c00::f03c:94ff:fe64:607d, 69.164.205.222
Redirect IPs 2600:3c00::f03c:94ff:fe64:607d, 69.164.205.222
Response IP 69.164.205.222
Found Yes
Hash 0d44ddb44da79f6af2c31573b652f92b1dcf7c146520ec7cc3ab322555557023
SimHash 2619e100af1a

Groups

*

Rule Path
Disallow /keywords-and-links.php
Disallow /performance-tools.php
Disallow /similar/
Disallow /findmore/
Disallow /members/

happy monster

Rule Path
Disallow

dotbot

Rule Path
Disallow /find

sistrix

Rule Path
Disallow /find

sbsearch

Rule Path
Disallow /find/
Disallow /findmore/
Disallow /information/
Disallow /similar/

Comments

  • Don't spider our own search results