carinfos.net
robots.txt

Robots Exclusion Standard data for carinfos.net

Resource Scan

Scan Details

Site Domain carinfos.net
Base Domain carinfos.net
Scan Status Ok
Last Scan2026-01-18T00:57:58+00:00
Next Scan 2026-02-17T00:57:58+00:00

Last Scan

Scanned2026-01-18T00:57:58+00:00
URL https://carinfos.net/robots.txt
Domain IPs 104.21.57.159, 172.67.164.168, 2606:4700:3031::ac43:a4a8, 2606:4700:3037::6815:399f
Response IP 172.67.164.168
Found Yes
Hash 93870eab2f690e937d8ad8de09d4959087a839c166f67783450cee9104f16ee5
SimHash 501cc482e790

Groups

*

Rule Path
Disallow /reports/view/
Disallow /reports/status/
Disallow /reports/full-report/
Disallow /*/reports/view/
Disallow /*/reports/status/
Disallow /*/reports/full-report/
Disallow /accounts/
Disallow /*/accounts/
Disallow /profile
Disallow /*/profile
Disallow /private/
Disallow /tmp/
Disallow /shared/
Disallow /*/shared/

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

applebot

Rule Path
Disallow

slurp

Product Comment
slurp Yahoo's bot
Rule Path
Disallow

sogou

Rule Path
Disallow

exabot

Rule Path
Disallow

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

waybackmachine

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

Comments

  • === Rules for all bots ===
  • === Allow major search engines ===
  • === Block archivers and aggressive crawlers ===