Log In Sign Up

koldfront.dk
robots.txt

Robots Exclusion Standard data for koldfront.dk

Resource Scan

Scan Details

Site Domain koldfront.dk
Base Domain koldfront.dk
Scan Status Ok
Last Scan2024-10-24T00:12:42+00:00
Next Scan 2024-11-23T00:12:42+00:00

Last Scan

Scanned2024-10-24T00:12:42+00:00
URL https://koldfront.dk/robots.txt
Domain IPs 212.237.178.116, 2a06:4001:d06f:0:b62e:99ff:fe41:be49
Response IP 212.237.178.116
Found Yes
Hash 486ee59c293402b878cea89987465dfbc25ff7c05cca9c6b5bfcd42672e6072f
SimHash 181c5972d998

Groups

*

Rule Path
Disallow /calendar/
Disallow /text/netexpres-2.4.7.html
Disallow /text/netexpres-2.4.0-test4.html
Disallow /text/netexpres-2.3.39.html
Disallow /company/
Disallow /text/emailplease.html
Disallow /cam/
Disallow /log/
Disallow /staff/
Disallow /darcs/
Disallow /darcsweb
Disallow /debian/
Disallow /git/
Disallow /gitweb
Disallow /text/handel/

turnitinbot

Rule Path
Disallow /

girafabot

Rule Path
Disallow /

npbot

Rule Path
Disallow /

hostharvest

Rule Path
Disallow /

bloglines

Rule Path
Disallow /log.rss

feedster crawler

Rule Path
Disallow /log.rss

syndic8

Rule Path
Disallow /log.rss

everyfeed-spider

Rule Path
Disallow /log.rss

statastico

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

alphaseobot-sa

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

domaincrawler

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

linguee

Rule Path
Disallow /

blexbot

Rule Path
Disallow /calendar/

dataforseobot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /
Back to top

Comments

  • Contains an infinite number of "pages":
  • These are obsolete:
  • These have moved:
  • My darcs repos have been deprecated in favour of git:
  • No need to advertise my locally built packages:
  • Nor my local git repo:
  • I don't want aggregators to advertise next to/around my content:
  • Does not respect *:
Back to top