websleuths.com
robots.txt
Robots Exclusion Standard data for websleuths.com
Resource Scan
Scan Details
Site Domain | websleuths.com |
Base Domain | websleuths.com |
Scan Status | Ok |
Last Scan | 2024-09-21T08:31:30+00:00 |
Next Scan | 2024-09-28T08:31:30+00:00 |
Last Scan
Scanned | 2024-09-21T08:31:30+00:00 |
URL | https://websleuths.com/robots.txt |
Domain IPs | 104.243.41.90 |
Response IP | 104.243.41.90 |
Found | Yes |
Hash | a56f36ac36ac040b8e339ddf4f515e4f5facdd5b7581f51d106f65ce1adcda3a |
SimHash | 63db12e04b89 |
Groups
*
Rule | Path |
---|---|
Disallow | /cgi-bin/ |
*
Rule | Path |
---|---|
Disallow | /cache.php |
Disallow | /forums/archive/ |
Disallow | /forums/attachment.php |
Disallow | /forums/attachments/ |
Disallow | /forums/editpost.php |
Disallow | /forums/newreply.php |
Disallow | /forums/newthread.php |
Disallow | /forums/private.php |
Disallow | /forums/report.php |
Disallow | /forums/sendmessage.php |
Disallow | /forums/threadrate.php |
Disallow | /forums/usercp.php |
Disallow | /forums/subscriptions/ |
Disallow | /forums/tmp/ |
Disallow | /forums/joinrequests.php |
Disallow | /forums/newattachment.php |
Disallow | /forums/printthread.php |
Disallow | /forums/showgroups.php |
Disallow | /forums/subscription.php |
Disallow | /forums/subscriptions.php |
Disallow | /forums/usernote.php |
Disallow | /forums/customavatars/ |
Disallow | /forums/customprofilepics/ |
Disallow | /forums/global.php |
Disallow | /forums/inlinemod.php |
Disallow | /forums/reputation.php |
Disallow | /forums/member.php |
Disallow | /forums/tags.php |
Disallow | /forums/album.php |
Disallow | /forums/search.php |
Disallow | /testvb/ |
Disallow | /tags.php |
Disallow | /showthread.php |
Disallow | /search.php |
Disallow | /member.php |
Disallow | /forumdisplay.php |
Disallow | /blog.php |
Disallow | /attachment.php |
Disallow | /calendar.php |
Disallow | /external.php |
Other Records
Field | Value |
---|---|
sitemap | http://www.websleuths.com/forums/store_sitemap/vbulletin_sitemap_index.xml.gz |