Log In Sign Up

rainadmin.com
robots.txt

Robots Exclusion Standard data for rainadmin.com

Resource Scan

Scan Details

Site Domain rainadmin.com
Base Domain rainadmin.com
Scan Status Ok
Last Scan2024-05-12T05:33:22+00:00
Next Scan 2024-06-11T05:33:22+00:00

Last Scan

Scanned2024-05-12T05:33:22+00:00
URL https://www.rainadmin.com/robots.txt
Domain IPs 204.246.191.101, 204.246.191.2, 204.246.191.29, 204.246.191.3, 2600:9000:25ef:5400:8:8f39:1c00:93a1, 2600:9000:25ef:7200:8:8f39:1c00:93a1, 2600:9000:25ef:c000:8:8f39:1c00:93a1, 2600:9000:25ef:d600:8:8f39:1c00:93a1, 2600:9000:25ef:da00:8:8f39:1c00:93a1, 2600:9000:25ef:e400:8:8f39:1c00:93a1, 2600:9000:25ef:e600:8:8f39:1c00:93a1, 2600:9000:25ef:ec00:8:8f39:1c00:93a1
Response IP 18.165.171.127
Found Yes
Hash 34eb3958ec33e893904347b7f60a6b0855a3ef375823bcda513555051a96ad05
SimHash d31c55da3f89

Groups

*

Rule Path
Disallow

Other Records

Field Value
crawl-delay 10

blexbot
mj12bot
goodzer
ahrefsbot
spbot
dotbot
dotbot
turnitinbot
seznambot
the knowledge ai
checkmarknetwork
ever accountable
seekportbot
claudebot
mauibot
houzzbot
baiduspider
baiduspider-image
serpstatbot
sogou blog
sogou inst spider
sogou news spider
sogou orion spider
sogou spider2
sogou web spider
uptimebot
yandex
yandexmobilebot
zoominfobot
megaindex.ru
alphaseobot-sa
proximic
amazonbot
petalbot
re-re studio
barkrowler
siteauditbot
bytespider
dataforseobot
ioncrawl
nutch

Rule Path
Disallow /

semrushbot-ba
semrushbot

Rule Path
Disallow /
Back to top

Comments

  • See if Semrush can behave - March 2023.
  • Update Nov 2023, no, they can't behave.
Back to top

Warnings

  • 1 invalid line.
Back to top