forum.blu-ray.com
robots.txt

Robots Exclusion Standard data for forum.blu-ray.com

Resource Scan

Scan Details

Site Domain forum.blu-ray.com
Base Domain blu-ray.com
Scan Status Ok
Last Scan2024-05-26T14:57:02+00:00
Next Scan 2024-06-25T14:57:02+00:00

Last Scan

Scanned2024-05-26T14:57:02+00:00
URL https://forum.blu-ray.com/robots.txt
Domain IPs 64.21.0.131
Response IP 64.21.0.131
Found Yes
Hash 75f9e76636d9c2a7fe5234d80ce2df55d54ddc3850c5a7971b810dda64119dcb
SimHash 0304c250c601

Groups

*

Rule Path
Disallow /offline/
Disallow /discussion/
Disallow /hardware/
Disallow /printthread.php
Disallow /admincp/
Disallow /images/
Disallow /modcp/
Disallow /attachment.php
Disallow /search.php
Disallow /newreply.php
Disallow /newthread.php
Disallow /editpost.php
Disallow /profile.php
Disallow /register.php
Disallow /login.php
Disallow /subscription.php
Disallow /private.php
Disallow /report.php
Disallow /sendmessage.php
Disallow /memberlist.php
Disallow /misc.php
Disallow /moderator.php
Disallow /postings.php
Disallow /sendtofriend.php
Disallow /threadrate.php
Disallow /usercp.php
Disallow /showgroups.php

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mediapartners-google

Rule Path
Allow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

yandex

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ahrefsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

proximic

Rule Path
Disallow /

riddler

Rule Path
Disallow /

maxpointcrawler

Rule Path
Disallow /

maxpointcrawler/nutch-1.10

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

seekport

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

ntentbot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

sirdatabot

Rule Path
Disallow /

peer39_crawler

Rule Path
Disallow /

peer39_crawler/1.0

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /