Log In Sign Up

nightowl.gg
robots.txt

Robots Exclusion Standard data for nightowl.gg

Resource Scan

Scan Details

Site Domain nightowl.gg
Base Domain nightowl.gg
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-05-02T20:12:00+00:00
Next Scan 2024-07-31T20:12:00+00:00

Last Successful Scan

Scanned2023-06-15T19:04:56+00:00
URL https://nightowl.gg/robots.txt
Redirect https://www.nightowl.gg/robots.txt
Redirect Domain www.nightowl.gg
Redirect Base nightowl.gg
Domain IPs 13.226.2.118, 13.226.2.122, 13.226.2.45, 13.226.2.66, 2600:9000:21f8:1c00:b:4505:3d80:93a1, 2600:9000:21f8:400:b:4505:3d80:93a1, 2600:9000:21f8:6200:b:4505:3d80:93a1, 2600:9000:21f8:9e00:b:4505:3d80:93a1, 2600:9000:21f8:aa00:b:4505:3d80:93a1, 2600:9000:21f8:bc00:b:4505:3d80:93a1, 2600:9000:21f8:d000:b:4505:3d80:93a1, 2600:9000:21f8:e000:b:4505:3d80:93a1
Redirect IPs 143.204.9.17, 143.204.9.6, 143.204.9.67, 143.204.9.96, 2600:9000:248d:2200:b:4505:3d80:93a1, 2600:9000:248d:2600:b:4505:3d80:93a1, 2600:9000:248d:4e00:b:4505:3d80:93a1, 2600:9000:248d:7e00:b:4505:3d80:93a1, 2600:9000:248d:800:b:4505:3d80:93a1, 2600:9000:248d:8e00:b:4505:3d80:93a1, 2600:9000:248d:a400:b:4505:3d80:93a1, 2600:9000:248d:b600:b:4505:3d80:93a1
Response IP 143.204.98.80
Found Yes
Hash 8eb429f90e4d98118d00e0a9faf2cc0abd4095df863e9797dddeab263a703520
SimHash 38304d114c9c

Groups

*

Rule Path
Disallow /startTopic/
Disallow /discover/unread/
Disallow /markallread/
Disallow /staff/
Disallow /online/
Disallow /discover/
Disallow /leaderboard/
Disallow /search/
Disallow /tags/
Disallow /*?advancedSearchForm=
Disallow /register/
Disallow /lostpassword/
Disallow /login/
Disallow /*?sortby=
Disallow /*?filter=
Disallow /*?tab=
Disallow /*?do=
Disallow /*ref%3D
Disallow /*?forumId*
Disallow /profile/
Back to top

Other Records

Field Value
sitemap https://www.nightowl.gg/sitemap.php
Back to top

Comments

  • Rules for Invision Community (https://invisioncommunity.com)
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links
  • Sitemap URL
Back to top