dev.gamereactor.no
robots.txt

Robots Exclusion Standard data for dev.gamereactor.no

Resource Scan

Scan Details

Site Domain dev.gamereactor.no
Base Domain gamereactor.no
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-29T07:37:05+00:00
Next Scan 2025-09-12T07:37:05+00:00

Last Successful Scan

Scanned2025-07-21T03:16:22+00:00
URL https://dev.gamereactor.no/robots.txt
Domain IPs 104.21.31.56, 172.67.175.3, 2606:4700:3036::ac43:af03, 2606:4700:3037::6815:1f38
Response IP 104.21.31.56
Found Yes
Hash 66a7acb32de84647f98a78d0e048750e20ea5fdec8487e57bc4ff6ee63d36b2b
SimHash e2491a085691

Groups

*

Rule Path
Disallow /admin/
Disallow /569313408/
Disallow /grtv/embed/
Disallow /video/embed/
Disallow /grtv2/admin2/
Disallow /grtv2/insert_grtv_stats.php
Disallow /lay/agreements/
Disallow /*banComment*
Disallow /webshop/
Disallow /vcards.php
Disallow /partnerstuff/screenshots_js.php*
Disallow /academy/
Disallow /betakey/
Disallow /konkurranser/
Disallow /poll/
Disallow /blimedlem/
Disallow /la-oss-spille/
Disallow /bye.php*
Disallow /media/vtt/
Disallow /ajaxstuff/fblogin.php

screaming frog seo spider

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mediatoolkitbot

Rule Path
Disallow /

zoombot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

trendkite-akashic-crawler

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

claudebot/1.0

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

gptbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 3

Other Records

Field Value
sitemap https://www.gamereactor.no//media/sitemaps/sitemap_no_texts_2025_07.xml
sitemap https://www.gamereactor.no//media/sitemaps/sitemap_no_texts_2025_06.xml
sitemap https://www.gamereactor.no//media/sitemaps/sitemap_no_videos_2025_07.xml
sitemap https://www.gamereactor.no//media/sitemaps/sitemap_no_shorts_2025_07.xml
sitemap https://www.gamereactor.no//googlepingfeed.php