Log In Sign Up

autoexus.be
robots.txt

Robots Exclusion Standard data for autoexus.be

Resource Scan

Scan Details

Site Domain autoexus.be
Base Domain autoexus.be
Scan Status Ok
Last Scan2025-10-24T05:47:03+00:00
Next Scan 2025-11-07T05:47:03+00:00

Last Scan

Scanned2025-10-24T05:47:03+00:00
URL https://autoexus.be/robots.txt
Domain IPs 195.201.104.248
Response IP 195.201.104.248
Found Yes
Hash 20ab57bded228538a539585ce6199c7a7f5ab46da47baee1ca9c3b8f7d8238c5
SimHash 723c8cd0e79b

Groups

*

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

yandexbot

Rule Path
Allow /

semrushbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

seekport

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sputnikbot

Rule Path
Disallow /

statdom.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

webdatastats

Rule Path
Disallow /

jooblebot

Rule Path
Disallow /

backupland

Rule Path
Disallow /

netcraftsurveyagent

Rule Path
Disallow /

openstat.ru

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://autoexus.com/sitemap.xml
Back to top

Comments

  • Global rule (default): allow everything
  • --- Block unwanted bots ---
  • --- AI / Research bots ---
Back to top