informatique.nl
robots.txt

Robots Exclusion Standard data for informatique.nl

Resource Scan

Scan Details

Site Domain informatique.nl
Base Domain informatique.nl
Scan Status Ok
Last Scan2025-07-14T04:56:40+00:00
Next Scan 2025-08-13T04:56:40+00:00

Last Scan

Scanned2025-07-14T04:56:40+00:00
URL https://informatique.nl/robots.txt
Redirect https://www.informatique.nl/robots.txt
Redirect Domain www.informatique.nl
Redirect Base informatique.nl
Domain IPs 172.66.169.51, 172.67.42.181, 2606:4700:10::ac42:a933, 2606:4700:10::ac43:2ab5
Redirect IPs 172.66.169.51, 172.67.42.181, 2606:4700:10::ac42:a933, 2606:4700:10::ac43:2ab5
Response IP 172.67.42.181
Found Yes
Hash de918b429c7eb478f7c4caf3a73364128892f002b26b1301bbfe44a0ebdb0b5c
SimHash 4a15d8806015

Groups

googlebot

Rule Path
Disallow /cgi-bin/iqshop2.cgi
Disallow /winkelmandje
Disallow /vergelijken
Disallow /html/intern
Disallow /pdf
Disallow /ingram
Disallow /logfiles
Disallow /bestel

googlebot-image

Rule Path
Disallow /cgi-bin/iqshop2.cgi
Disallow /winkelmandje
Disallow /vergelijken
Disallow /html/intern
Disallow /pdf
Disallow /ingram
Disallow /logfiles
Disallow /bestel

*

Rule Path
Disallow /cgi-bin/iqshop2.cgi
Disallow /winkelmandje
Disallow /vergelijken
Disallow /html/intern
Disallow /pdf
Disallow /ingram
Disallow /logfiles
Disallow /bestel

adsbot/3.1

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

teracent-feed-processing

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ingrid

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

Comments

  • dissalow crawlers
  • delay