mechalog.com
robots.txt

Robots Exclusion Standard data for mechalog.com

Resource Scan

Scan Details

Site Domain mechalog.com
Base Domain mechalog.com
Scan Status Ok
Last Scan5/23/2025, 10:14:39 PM
Next Scan 5/30/2025, 10:14:39 PM

Last Scan

Scanned5/23/2025, 10:14:39 PM
URL https://mechalog.com/robots.txt
Redirect https://thisworldofdew.com/robots.txt
Redirect Domain thisworldofdew.com
Redirect Base thisworldofdew.com
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.32.1
Found Yes
Hash e980e5050a8370e1ed3ad0bf038cc4ee2912916ea6d0b53e0c4089ca578d8099
SimHash ff4f7c867d11

Groups

teleport
teleportpro
emailcollector
emailsiphon
webbandit
webzip
webreaper
webstripper
web downloader
ahrefsbot
semrushbot
mj12bot
webcopier
offline explorer pro
offline explorer
httrack website copier
offline commander
leech
websnake
blackwidow
http weazel

Rule Path
Disallow /

*

Rule Path
Disallow /video/*
Disallow /admin/
Disallow /dieu-khoan.html
Disallow /lien-he.html
Disallow /api/*

Other Records

Field Value
sitemap https://thisworldofdew.com/abcccc-sitemap.xml