Log In Sign Up

findkollegie.dk
robots.txt

Robots Exclusion Standard data for findkollegie.dk

Resource Scan

Scan Details

Site Domain findkollegie.dk
Base Domain findkollegie.dk
Scan Status Ok
Last Scan2025-11-13T05:27:55+00:00
Next Scan 2025-12-13T05:27:55+00:00

Last Scan

Scanned2025-11-13T05:27:55+00:00
URL https://findkollegie.dk/robots.txt
Domain IPs 104.21.87.176, 172.67.170.124, 2606:4700:3033::6815:57b0, 2606:4700:3035::ac43:aa7c
Response IP 172.67.170.124
Found Yes
Hash 6fd620049e617c70817b7ff7506d168b518d6bdb73e58e6f70da64fd4550c2d0
SimHash 2aad0c8d81f0

Groups

ia_archiver

Rule Path
Disallow /

*

Rule Path
Disallow /kollegie/*/contact-landlord
Disallow /rentals-in-*
Disallow /brugere/login
Disallow /abuse/
Disallow /?keywords=
Disallow /en/*
Disallow /en/rentals/
Disallow /en/abuse/
Disallow /misbrug*
Disallow /*/contact-landlord
Disallow /*/show-contact-details
Disallow /en/tenants*
Disallow *%26*
Back to top

Other Records

Field Value
sitemap https://findkollegie.dk/sitemaps/sitemap.xml.gz
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • User-agent: *
  • Disallow: /
Back to top