hmcrave.net
robots.txt
Robots Exclusion Standard data for hmcrave.net
Resource Scan
Scan Details
| Site Domain | hmcrave.net |
| Base Domain | hmcrave.net |
| Scan Status | Failed |
| Failure Stage | Fetching resource. |
| Failure Reason | Server returned a client error. |
| Last Scan | 2025-02-25T04:57:34+00:00 |
| Next Scan | 2025-05-26T04:57:34+00:00 |
Last Successful Scan
| Scanned | 2023-02-06T11:12:33+00:00 |
| URL | https://hmcrave.net/robots.txt |
| Domain IPs | 104.21.55.117, 172.67.147.253, 2606:4700:3031::6815:3775, 2606:4700:3035::ac43:93fd |
| Response IP | 172.67.147.253 |
| Found | Yes |
| Hash | 7e087e7967f21ac0a4d8a4a990c9996bc5c8faf8202488ad19fb831748352f82 |
| SimHash | 12087946ac32 |
Groups
netestate ne crawler
netestate foaf crawler
netestate rss crawler
searchmetericsbot
siteliner
spbot
turnitinbot
pixray-seeker
seokicks
seokicks-robot
wbsearchbot
aranhabot
backlinkcrawler
blexbot
ezooms
mail.ru
mail.ru_bot
piplbot
ccbot
scoutjet
netestate ne crawler
netestate foaf crawler
netestate rss crawler
searchmetericsbot
siteliner
spbot
turnitinbot
pixray-seeker
seokicks
seokicks-robot
wbsearchbot
aranhabot
backlinkcrawler
ezooms
gsitecrawler
mail.ru
mail.ru_bot
jorgee
piplbot
ccbot
scoutjet
semrushbot
semrushbot-sa
siteauditbot
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-coub
infotigerbot
dotbot
dotbot
velenpublicwebcrawler
megaindex.ru
dotbotzoominfobot
moget
ichiro
naverbot
yeti
youdaobot
mtrobot
blexbot
seznambot
yandex
petalbot
obot
outbrain
magpie-crawler
baiduspider
baiduspider-video
baiduspider-image
sogou spider
barkrowler
| Rule | Path |
|---|---|
| Disallow | / |
adsbot-google
*
| Rule | Path |
|---|---|
| Disallow | /share |
| Disallow | /widget |
| Disallow | /widgets |
| Disallow | /adcentric |
| Disallow | /adinterax |
| Disallow | /amobee |
| Disallow | /atlas |
| Disallow | /doubleclick |
| Disallow | /eyereturn |
| Disallow | /eyewonder |
| Disallow | /liquidus |
| Disallow | /klipmart |
| Disallow | /pointroll |
| Disallow | /smartadserver |
| Disallow | /unicast |
| Disallow | /viewpoint |
| Disallow | /addineyeV2.html |
| Disallow | /canvas.html |
| Disallow | /DARTIframe.html |
| Disallow | /interim.html |
| Disallow | /oggiPlayerLoader.htm |
| Disallow | /videoeggbackup.html |
| Disallow | /jivox |
| Disallow | /revjet |
| Disallow | /exponential |
| Disallow | /ipinyou |
| Disallow | /spongecell |
| Disallow | /bonzai |
| Disallow | /facebook_xd_receiver.html |
| Disallow | /readme.html |
| Disallow | /rpc_relay.html |
| Disallow | /twitterlists |
| Disallow | /cdn-cgi |
| Disallow | /admin |
| Disallow | /errors |
| Disallow | /1254144 |
| Disallow | /null |
| Disallow | /admin |
| Disallow | /components |
| Disallow | /wp |
| Disallow | /login |
| Disallow | /logout |
| Disallow | /view.php* |
| Disallow | /phpinfo |
| Disallow | /gmc_ |
| Disallow | /upfilees |
| Disallow | /google_matched_content_* |
| Disallow | /uploads |
| Allow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://hmcrave.net/sitemap.xml |
| sitemap | https://hmcrave.net/sitemap.xml.gz |
Comments