maxtrader.pl
robots.txt

Robots Exclusion Standard data for maxtrader.pl

Resource Scan

Scan Details

Site Domain maxtrader.pl
Base Domain maxtrader.pl
Scan Status Ok
Last Scan2025-08-10T06:49:51+00:00
Next Scan 2025-09-09T06:49:51+00:00

Last Scan

Scanned2025-08-10T06:49:51+00:00
URL http://maxtrader.pl/robots.txt
Domain IPs 51.75.61.209
Response IP 51.75.61.209
Found Yes
Hash 82d2c10882236840a4dab3bd58802bd67b4b07e1046b1821e4d9bf4bda3f26d3
SimHash 2114445366b3

Groups

*

Rule Path
Disallow /accessDenied

baiduspider

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

iccrawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

francis

Rule Path
Disallow /

stackrambler

Rule Path
Disallow /

seekbot

Rule Path
Disallow /

goku

Rule Path
Disallow /

turtlescanner/1.4

Rule Path
Disallow /

turtlescanner

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

e-societyrobot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

teoma

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

lwnutch/nutch-1.4

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10