Log In Sign Up

indiegames.biz
robots.txt

Robots Exclusion Standard data for indiegames.biz

Resource Scan

Scan Details

Site Domain indiegames.biz
Base Domain indiegames.biz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-11-05T20:30:31+00:00
Next Scan 2024-11-06T20:30:31+00:00

Last Successful Scan

Scanned2024-08-21T10:51:01+00:00
URL http://indiegames.biz/robots.txt
Domain IPs 130.211.24.203
Response IP 130.211.24.203
Found Yes
Hash 85acb7460fbfd7a34b6aa786ec297fc7bfac756d8bd3e8021acc3abf1411b201
SimHash 692f8668a8f3

Groups

*

Rule Path
Disallow /api/
Disallow /*admin/
Disallow /cgi-bin/
Disallow /datastore/
Disallow /system/
Disallow /login/
Disallow /logout/
Disallow /profile/*

Other Records

Field Value
crawl-delay 60
Back to top

Other Records

Field Value
sitemap https://gamedev.net/sitemap.xml
Back to top

Comments

  • Disallow: /applications/
  • Disallow: /plugins/
  • Disallow files
  • Disallow: /403error.php
  • Disallow: /404error.php
  • Disallow: /500error.php
  • Disallow: /Credits.txt
  • Disallow: /error.php
  • Disallow: /upgrading.html
  • Querystring
  • Disallow: /?tab=*
  • Disallow: /*?app=*
  • Disallow: /*sortby=*
  • Disallow: /*/?do=download
  • Disallow: /profile/*/?do=*
  • Disallow: /profile/*/content/
  • Disallow: /*?do=add
  • Disallow: /*?do=email
  • Disallow: /*?do=getNewComment
  • Disallow: /*?do=getLastComment
  • Disallow: /*?do=getLastComment
  • Disallow: /*?do=reportComment
  • Allow specific parts
  • Allow: /applications/core/interface/imageproxy/imageproxy.php?img=*
Back to top