Log In Sign Up

gamemania.be
robots.txt

Robots Exclusion Standard data for gamemania.be

Resource Scan

Scan Details

Site Domain gamemania.be
Base Domain gamemania.be
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-09-13T19:19:58+00:00
Next Scan 2024-12-12T19:19:58+00:00

Last Successful Scan

Scanned2022-09-30T10:27:52+00:00
URL https://www.gamemania.be/robots.txt
Response IP 172.66.42.233, 172.66.41.23
Found Yes
Hash a1882698d51649c1207ccd2efc1b797b449872823e9af1bafc09bb79247959c2
SimHash 38419c70b273

Groups

*

Rule Path
Disallow /lucee

baiduspider

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

yandexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /nl/internal
Disallow /fr/internal
Disallow /en/internal
Disallow /de/internal
Disallow /nl/admin
Disallow /fr/admin
Disallow /en/admin
Disallow /de/admin

Other Records

Field Value
crawl-delay 60
Back to top

Comments

  • internal & admin page
Back to top