Log In Sign Up

aid.no
robots.txt

Robots Exclusion Standard data for aid.no

Resource Scan

Scan Details

Site Domain aid.no
Base Domain aid.no
Scan Status Ok
Last Scan2024-09-24T01:39:47+00:00
Next Scan 2024-10-08T01:39:47+00:00

Last Scan

Scanned2024-09-24T01:39:47+00:00
URL https://aid.no/robots.txt
Redirect https://www.aid.no/robots.txt
Redirect Domain www.aid.no
Redirect Base aid.no
Domain IPs 2a02:c0:ac::a49d:1, 87.238.38.9
Redirect IPs 2a02:c0:ac::a49d:1, 87.238.38.9
Response IP 87.238.38.9
Found Yes
Hash de30d10387eba997e8cc1f8dc1ad82646ea5a0704b237bb0c8549ba1ae34337b
SimHash 404ec8c0e513

Groups

mediapartners-google

Rule Path
Allow /

*

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /
Back to top

Comments

  • robots.txt
  • Be nice.
  • Start AI crawler block
  • End AI crawler block
Back to top