bjdcollectors.com
robots.txt

Robots Exclusion Standard data for bjdcollectors.com

Resource Scan

Scan Details

Site Domain bjdcollectors.com
Base Domain bjdcollectors.com
Scan Status Ok
Last Scan2024-11-09T11:42:07+00:00
Next Scan 2024-11-16T11:42:07+00:00

Last Scan

Scanned2024-11-09T11:42:07+00:00
URL https://bjdcollectors.com/robots.txt
Redirect https://www.bjdcollectors.com/robots.txt
Redirect Domain www.bjdcollectors.com
Redirect Base bjdcollectors.com
Domain IPs 104.21.6.59, 172.67.154.249, 2606:4700:3035::6815:63b, 2606:4700:3036::ac43:9af9
Redirect IPs 104.21.6.59, 172.67.154.249, 2606:4700:3035::6815:63b, 2606:4700:3036::ac43:9af9
Response IP 104.21.6.59
Found Yes
Hash e4678f4341e60924e736bef5d1199f3543cefe64b5104ba6aecc123b4974eef3
SimHash 4440d470e335

Groups

*

Rule Path
Disallow /login/

baiduspider

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

yandex

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1

pinterestbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 1