autopadre.com
robots.txt

Robots Exclusion Standard data for autopadre.com

Resource Scan

Scan Details

Site Domain autopadre.com
Base Domain autopadre.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-26T21:51:17+00:00
Next Scan 2025-12-26T21:51:17+00:00

Last Successful Scan

Scanned2025-10-25T06:13:28+00:00
URL https://autopadre.com/robots.txt
Redirect https://www.autopadre.com/robots.txt
Redirect Domain www.autopadre.com
Redirect Base autopadre.com
Domain IPs 104.26.8.170, 104.26.9.170, 172.67.70.18, 2606:4700:20::681a:8aa, 2606:4700:20::681a:9aa, 2606:4700:20::ac43:4612
Redirect IPs 104.26.8.170, 104.26.9.170, 172.67.70.18, 2606:4700:20::681a:8aa, 2606:4700:20::681a:9aa, 2606:4700:20::ac43:4612
Response IP 104.26.9.170
Found Yes
Hash 6187301fdea366debecc3e6f85e1dbb516acffc0091122509c57a8e6cb34401c
SimHash 471cdcf76601

Groups

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10