smwdb.me
robots.txt

Robots Exclusion Standard data for smwdb.me

Resource Scan

Scan Details

Site Domain smwdb.me
Base Domain smwdb.me
Scan Status Failed
Failure ReasonScan timed out.
Last Scan2025-05-23T18:14:42+00:00
Next Scan 2025-08-21T18:14:42+00:00

Last Successful Scan

Scanned2024-01-30T18:08:26+00:00
URL https://smwdb.me/robots.txt
Domain IPs 71.197.118.159
Response IP 71.197.118.159
Found Yes
Hash 371c68bcbb93a2b09fb883561779f941373a02b79fd28d8dfa05e0cd1f4f93b0
SimHash 50965441feb2

Groups

*

Rule Path
Disallow /log/
Disallow /c/*
Disallow /db/*/patch.*
Disallow /db/*/lv*.png
Disallow /db/*/x_*.htm
Disallow /db/*/asm_*
Disallow /db/*/doc_*
Disallow /db/*/mus_*
Disallow /db/*/sav_*
Disallow /db/*/etc_*
Disallow /db/*/worlds.htm
Disallow /db/*/credits.png

claudebot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

adsbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

bingpreview

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

coccocbot

Rule Path
Disallow /

curl

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

indeedbot

Rule Path
Disallow /

keybot translation-search-machine

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

netestate

Rule Path
Disallow /

node-fetch

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

pinterestbot

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

wget

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 1

Other Records

Field Value
sitemap http://smwdb.me/sitemap.xml