treningsforum.no
robots.txt

Robots Exclusion Standard data for treningsforum.no

Resource Scan

Scan Details

Site Domain treningsforum.no
Base Domain treningsforum.no
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-18T02:57:09+00:00
Next Scan 2024-11-17T02:57:09+00:00

Last Successful Scan

Scanned2024-06-28T02:55:03+00:00
URL https://treningsforum.no/robots.txt
Domain IPs 13.69.228.3
Response IP 13.69.228.3
Found Yes
Hash dbf91d79ffab7ddd4fb7c7be37dfb4afd5a8ce8dee15f8e704432f27a8ce0f40
SimHash 434dd4b27d00

Groups

*

Rule Path
Disallow /abonnement/
Disallow /extra/
Disallow /magasinet/

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

appcodescrawler

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

compspybot

Rule Path
Disallow /

gwm-gsa-crawler

Rule Path
Disallow /

linguee

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

fatbot 2.0

Rule Path
Disallow /

ahc/1.0

Rule Path
Disallow /

ahc/2.0

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

sogou blog

Rule Path
Disallow /

sogou inst spider

Rule Path
Disallow /

sogou news spider

Rule Path
Disallow /

sogou orion spider

Rule Path
Disallow /

sogou spider2

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

bluechipbacklinks

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /