Log In Sign Up

vallsupdate.gamerchatforum.com
robots.txt

Robots Exclusion Standard data for vallsupdate.gamerchatforum.com

Resource Scan

Scan Details

Site Domain vallsupdate.gamerchatforum.com
Base Domain gamerchatforum.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-03-11T21:38:29+00:00
Next Scan 2024-06-09T21:38:29+00:00

Last Successful Scan

Scanned2023-02-06T15:43:10+00:00
URL https://vallsupdate.gamerchatforum.com/robots.txt
Domain IPs 107.21.35.214, 18.213.166.18
Response IP 18.213.166.18
Found Yes
Hash fab1e9f0bbe7f9e3bb0bc24d955af8f94c0a1e2cab608ab1ab7603ae3e4182b5
SimHash 6842d8dcdcc0

Groups

bubing
alphaseobot
ltx71
companybook-crawler
bdcbot
spbot
semrushbot
ahrefsbot
mj12bot
dotbot
omgili
blexbot
magpie-crawler
extlinksbot
netseer
weborama-fetcher
linkfluence
sentibot
seokicks
barkrowler
ccbot
trendictionbot
amazonbot
serpstatbot
petalbot
dataforseobot
censysinspect

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /
Allow /site/images/support/

facebookexternalhit

Rule Path
Disallow /images/
Disallow /pm
Disallow /album
Disallow /mbactions
Disallow /register
Disallow /email
Disallow /search
Disallow /upload
Disallow /printthread
Disallow /post/show_single_post
Disallow /post/printadd
Disallow /post/upost
Disallow /post/hpt
Disallow /subscribe
Disallow /calendar
Disallow /calendar/newevent
Disallow /calendar/daydetail?*&nav=
Disallow /calendar/display*view%3Dweekly
Disallow /calendar/display*view%3Dmonthly
Disallow /calendar/showbirthday
Disallow /external
Disallow /tool/view/gb/private
Disallow /tool/view/gb/email
Disallow /tool/pm
Disallow /tool/members/
Disallow /tool/ticket/
Disallow /cgi/view/poll.cgi
Disallow /cgi/view/topsites.cgi
Disallow /cgi/view/member.cgi
Disallow /cgi/view/out.cgi
Disallow /?authtoken=
Disallow *?*&sort=
Disallow *?sort=
Allow /site/images/support/

Other Records

Field Value
crawl-delay 15

mediapartners-google

Rule Path
Disallow /images/
Disallow /pm
Disallow /album
Disallow /mbactions
Disallow /register
Disallow /email
Disallow /search
Disallow /profile
Disallow /file
Disallow /thumb
Disallow /upload
Disallow /printthread
Disallow /post/show_single_post
Disallow /post/printadd
Disallow /post/upost
Disallow /post/hpt
Disallow /subscribe
Disallow /calendar
Disallow /calendar/newevent
Disallow /calendar/daydetail?*&nav=
Disallow /calendar/display*view%3Dweekly
Disallow /calendar/display*view%3Dmonthly
Disallow /calendar/showbirthday
Disallow /external
Disallow /tool/view/gb/private
Disallow /tool/view/gb/email
Disallow /tool/pm
Disallow /tool/members/
Disallow /tool/ticket/
Disallow /cgi/view/poll.cgi
Disallow /cgi/view/topsites.cgi
Disallow /cgi/view/member.cgi
Disallow /cgi/view/out.cgi
Disallow /?authtoken=

Other Records

Field Value
crawl-delay 15

*

Rule Path
Disallow /images/
Disallow /pm
Disallow /album
Disallow /mbactions
Disallow /register
Disallow /email
Disallow /search
Disallow /profile
Disallow /file
Disallow /thumb
Disallow /upload
Disallow /printthread
Disallow /post/show_single_post
Disallow /post/printadd
Disallow /post/upost
Disallow /post/hpt
Disallow /subscribe
Disallow /calendar
Disallow /calendar/newevent
Disallow /calendar/daydetail?*&nav=
Disallow /calendar/display*view%3Dweekly
Disallow /calendar/display*view%3Dmonthly
Disallow /calendar/showbirthday
Disallow /external
Disallow /tags
Disallow /tool/view/gb/private
Disallow /tool/view/gb/email
Disallow /tool/pm
Disallow /tool/members/
Disallow /tool/ticket/
Disallow /cgi/view/poll.cgi
Disallow /cgi/view/topsites.cgi
Disallow /cgi/view/member.cgi
Disallow /cgi/view/out.cgi
Disallow /?authtoken=
Disallow /contact?*subject=
Disallow /post*?goto=
Disallow /post*%26goto%3D
Disallow /post*?id=
Disallow *?*&sort=
Disallow *?sort=
Disallow /*.jpg$
Disallow /*.jpeg$
Disallow /*.gif$
Disallow /*.png$
Disallow /tool/members/login?action=logout
Allow /site/images/support/
Allow /tool/members/signup
Allow /tool/members/login

Other Records

Field Value
crawl-delay 15
Back to top

Other Records

Field Value
sitemap https://vallsupdate.gamerchatforum.com/sitemap.xml
Back to top

Comments

  • Most desirable images are hosted on S3. These images will just be icons and stuff, so block them.
  • allowing access to image scripts and topic pages for proper sharing
  • allowing access to topic pages for proper context ads
  • Disallow pages with very little content, duplicate content, or different links pointing to the same content
Back to top