museedelhistoire.ca
robots.txt

Robots Exclusion Standard data for museedelhistoire.ca

Resource Scan

Scan Details

Site Domain museedelhistoire.ca
Base Domain museedelhistoire.ca
Scan Status Ok
Last Scan2024-11-11T20:02:10+00:00
Next Scan 2024-12-11T20:02:10+00:00

Last Scan

Scanned2024-11-11T20:02:10+00:00
URL https://www.museedelhistoire.ca/robots.txt
Domain IPs 2600:9000:2024:1200:8:aae5:fd00:93a1, 2600:9000:2024:1a00:8:aae5:fd00:93a1, 2600:9000:2024:2e00:8:aae5:fd00:93a1, 2600:9000:2024:7200:8:aae5:fd00:93a1, 2600:9000:2024:b000:8:aae5:fd00:93a1, 2600:9000:2024:b400:8:aae5:fd00:93a1, 2600:9000:2024:be00:8:aae5:fd00:93a1, 2600:9000:2024:f400:8:aae5:fd00:93a1, 65.9.112.116, 65.9.112.17, 65.9.112.41, 65.9.112.84
Response IP 18.165.140.70
Found Yes
Hash 72c87fd04c46fe98ecfe0e12cd8bfe865c1c9b47e7dd7322b5b63f4aa8bb3dd2
SimHash 5622fa202ba2

Groups

voltron

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

*

Rule Path
Disallow /*.json
Disallow /*.xml
Disallow /*.embed
Disallow /collections/gallery
Disallow /collections/galerie
Disallow /wp-admin/
Disallow /*/wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /*/wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10

Comments

  • 80legs
  • 80legs' new crawler

Warnings

  • 2 invalid lines.