player.gamespot.com
robots.txt

Robots Exclusion Standard data for player.gamespot.com

Resource Scan

Scan Details

Site Domain player.gamespot.com
Base Domain gamespot.com
Scan Status Ok
Last Scan2024-11-11T01:52:07+00:00
Next Scan 2024-11-18T01:52:07+00:00

Last Scan

Scanned2024-11-11T01:52:07+00:00
URL https://player.gamespot.com/robots.txt
Domain IPs 199.232.208.194, 199.232.212.194
Response IP 199.232.44.194
Found Yes
Hash 6938249f7ca20dc102c858867a284c83cb06577c6f8c42486759b2d70e271831
SimHash 504671406821

Groups

ahrefsbot

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

*

Rule Path
Disallow */?comment_page=*
Allow /api/$
Disallow /api/
Disallow /chat/
Disallow /community/
Disallow /email/
Disallow /feeds/*
Disallow /forgot-password/
Disallow /g00/*
Disallow /genre/
Disallow /images/
Disallow /jsonsearch/
Disallow /latest-amp-article/
Disallow /login/
Disallow /login-signup/
Disallow /notifications/
Disallow /pages/
Disallow /platform/
Disallow /postRender
Disallow /profile/*/blog/
Disallow /profile/*/reviews/
Disallow /profile/*/trophies/
Disallow /search/
Disallow /signup/
Disallow /tags/
Disallow /test/
Disallow /theme/
Disallow /trax/
Disallow /users/

Comments

  • robots.txt for https://www.gamespot.com/