operaonvideo.com
robots.txt

Robots Exclusion Standard data for operaonvideo.com

Resource Scan

Scan Details

Site Domain operaonvideo.com
Base Domain operaonvideo.com
Scan Status Ok
Last Scan2024-09-20T05:31:08+00:00
Next Scan 2024-09-27T05:31:08+00:00

Last Scan

Scanned2024-09-20T05:31:08+00:00
URL https://operaonvideo.com/robots.txt
Domain IPs 69.48.143.211
Response IP 69.48.143.211
Found Yes
Hash 48cafe229319f3ab7a4cdda0543986f22e1305f98bb09147271c8823e47c2fdb
SimHash 4530d052af12

Groups

scrapy

Rule Path
Allow /

ahrefsbot

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mozilla/5.0 applewebkit/537.36 (khtml, like gecko; compatible; claudebot/1.0; +claudebot@anthropic.com

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

proximic

Rule Path
Disallow /

facebookbot

No rules defined. All paths allowed.

Other Records

Field Value Comment
crawl-delay 2 1 page per 2 seconds

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2