rivaji.com
robots.txt

Robots Exclusion Standard data for rivaji.com

Resource Scan

Scan Details

Site Domain rivaji.com
Base Domain rivaji.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-09-08T10:09:22+00:00
Next Scan 2024-12-07T10:09:22+00:00

Last Successful Scan

Scanned2024-01-19T20:26:12+00:00
URL https://rivaji.com/robots.txt
Domain IPs 104.21.36.136, 172.67.194.213, 2606:4700:3033::6815:2488, 2606:4700:3037::ac43:c2d5
Response IP 172.67.194.213
Found Yes
Hash 894aee9ad76754025ffdb24214c53557fb6d1e877e4a7cb775bc57c25823f9d6
SimHash d05dd340e790

Groups

*

Rule Path
Disallow
Disallow /index.php?m=api*
Disallow /template/*

semrushbot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /