Log In Sign Up

itsth.com
robots.txt

Robots Exclusion Standard data for itsth.com

Resource Scan

Scan Details

Site Domain itsth.com
Base Domain itsth.com
Scan Status Ok
Last Scan2025-09-04T18:27:06+00:00
Next Scan 2025-10-04T18:27:06+00:00

Last Scan

Scanned2025-09-04T18:27:06+00:00
URL http://itsth.com/robots.txt
Domain IPs 217.160.0.121
Response IP 217.160.0.121
Found Yes
Hash 2a67f6265daf198550ba77ac66694c08e4c45c024947fa53afc28e9fd2a58a32
SimHash b05758c8c693

Groups

wget

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

seekmo

Rule Path
Disallow /

linguee bot

Rule Path
Disallow /

purebot

Rule Path
Disallow /

purebot*

Rule Path
Disallow /

purebot/1.1

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

cityreview robot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

comodospider

Rule Path
Disallow /

ahrefs.com

Rule Path
Disallow /

xenu link sleuth

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

*

Rule Path
Disallow /udm-resources
Back to top

Other Records

Field Value
sitemap http://www.easy2sync.com/sitemap.php
Back to top