euroman.dk
robots.txt

Robots Exclusion Standard data for euroman.dk

Resource Scan

Scan Details

Site Domain euroman.dk
Base Domain euroman.dk
Scan Status Ok
Last Scan2024-11-14T09:30:12+00:00
Next Scan 2024-11-21T09:30:12+00:00

Last Scan

Scanned2024-11-14T09:30:12+00:00
URL https://euroman.dk/robots.txt
Redirect https://www.euroman.dk/robots.txt
Redirect Domain www.euroman.dk
Redirect Base euroman.dk
Domain IPs 52.157.217.51
Redirect IPs 52.157.217.51
Response IP 52.157.217.51
Found Yes
Hash 3622464bf9884232d3e856191e1d48a13e4a08fe5c9345a4ad680def93a4652f
SimHash 6004c8e0e422

Groups

*

Rule Path
Disallow /Util/
Disallow */content$
Disallow */Content$

gptbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

Comments

  • start AI crawler block
  • Common crawl
  • OpenAI (ChatGPT realtime search)
  • Anthropic
  • end AI crawler block