rcsearch.info
robots.txt

Robots Exclusion Standard data for rcsearch.info

Resource Scan

Scan Details

Site Domain rcsearch.info
Base Domain rcsearch.info
Scan Status Ok
Last Scan2024-05-18T05:58:28+00:00
Next Scan 2024-05-25T05:58:28+00:00

Last Scan

Scanned2024-05-18T05:58:28+00:00
URL https://rcsearch.info/robots.txt
Domain IPs 104.21.40.146, 172.67.153.11, 2606:4700:3031::ac43:990b, 2606:4700:3036::6815:2892
Response IP 172.67.153.11
Found Yes
Hash b9e4cd58d6a36165e9f353522253f632dddd76031257cc441af563248c174815
SimHash b23e42632380

Groups

*

Rule Path
Disallow /js
Disallow /c/
Disallow /404.php
Disallow /w/index.php?title=
Disallow /w/index.php?title=
Disallow /?s=
Disallow *?store=
Disallow /new/

Other Records

Field Value
crawl-delay 5

acoon

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

discoverybot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

searchmetricsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

seekport

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

zoominfo

Rule Path
Disallow /

proximic

Rule Path
Disallow /

flipboard

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

serpstat

Rule Path
Disallow /

seopult

Rule Path
Disallow /

cloudfind

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

niraiya

Rule Path
Disallow /

yak

Rule Path
Disallow /

mbcrawler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

zumbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /