rifme.net
robots.txt

Robots Exclusion Standard data for rifme.net

Resource Scan

Scan Details

Site Domain rifme.net
Base Domain rifme.net
Scan Status Ok
Last Scan2024-11-16T12:26:00+00:00
Next Scan 2024-11-23T12:26:00+00:00

Last Scan

Scanned2024-11-16T12:26:00+00:00
URL https://rifme.net/robots.txt
Domain IPs 46.36.219.212
Response IP 46.36.219.212
Found Yes
Hash 9e9e483cbecc664e95d88222443b4bebb98fe8c5f97c1ac3110f032146406451
SimHash 6b4c4467711c

Groups

*

Rule Path
Disallow /u/
Disallow /contact
Disallow /switcher.php?st
Disallow /assonans-switcher.php
Disallow /switcher.php?slogov=*
Disallow /switcher.php?chast=*
Disallow /switcher.php?hide
Disallow /switcher.php?netochnye

mediapartners-google

Rule Path
Disallow

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

spbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

siteexplorer

Rule Path
Disallow /r/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

spbot

Rule Path
Disallow /r/

ia_archiver

Rule Path
Disallow /r/

archive.org_bot

Rule Path
Disallow /r/

ia_archiver-web.archive.org

Rule Path
Disallow /r/

ccbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 36000

semrushbot

Rule Path
Disallow /r/

spiderling

Rule Path
Disallow /r/

hubspot webcrawler

Rule Path
Disallow /r/

cliqzbot

Rule Path
Disallow /r/

seokicks

Rule Path
Disallow /r/

petalbot

Rule Path
Disallow /r/

seekport crawler

Rule Path
Disallow /r/

dataforseobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /r/

bytespider

Rule Path
Disallow /r/

barkrowler

Rule Path
Disallow /

Warnings

  • `host` is not a known field.