autodoc.lu
robots.txt

Robots Exclusion Standard data for autodoc.lu

Resource Scan

Scan Details

Site Domain autodoc.lu
Base Domain autodoc.lu
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-06-27T03:11:04+00:00
Next Scan 2025-07-11T03:11:04+00:00

Last Successful Scan

Scanned2025-04-24T03:08:27+00:00
URL https://autodoc.lu/robots.txt
Redirect https://www.autodoc.lu/robots.txt
Redirect Domain www.autodoc.lu
Redirect Base autodoc.lu
Domain IPs 34.141.86.85
Redirect IPs 34.141.86.85
Response IP 34.141.86.85
Found Yes
Hash 34593a3d52fcad028e458a571b99ac027c799d776373feef08752e386555c830
SimHash 6d8df4227332

Groups

blexbot

Rule Path
Disallow /

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /cdn-cgi/*
Disallow /ajax/*
Disallow /register$
Disallow /profile
Disallow /basket
Disallow /pdf/*
Disallow /search*
Disallow /*?parameter
Disallow /*?sort_supplier
Disallow /*?list=
Disallow /*?price_range
Disallow /*?gadw
Disallow /*?maker
Disallow /td_api_receiver
Disallow /p/*
Disallow *?criteria%
Disallow /profile/unsubscribe/*

yandex

Rule Path
Disallow /

chatgpt-user

Rule Path
Allow /info/*
Allow /ersatzteile*
Disallow /

gptbot

Rule Path
Allow /info/*
Allow /ersatzteile*
Disallow /