dlgamer.com
robots.txt

Robots Exclusion Standard data for dlgamer.com

Resource Scan

Scan Details

Site Domain dlgamer.com
Base Domain dlgamer.com
Scan Status Ok
Last Scan2024-09-20T04:48:04+00:00
Next Scan 2024-10-20T04:48:04+00:00

Last Scan

Scanned2024-09-20T04:48:04+00:00
URL https://dlgamer.com/robots.txt
Redirect https://www.dlgamer.com/go/robots.txt
Redirect Domain www.dlgamer.com
Redirect Base dlgamer.com
Domain IPs 104.26.14.138, 104.26.15.138, 172.67.71.177, 2606:4700:20::681a:e8a, 2606:4700:20::681a:f8a, 2606:4700:20::ac43:47b1
Redirect IPs 104.26.14.138, 104.26.15.138, 172.67.71.177, 2606:4700:20::681a:e8a, 2606:4700:20::681a:f8a, 2606:4700:20::ac43:47b1
Response IP 104.26.15.138
Found Yes
Hash 023e6bdb44e22978baf53fcbd98b643e5164aedde63ed059daf81750224be6f2
SimHash 8881f2a1ccd3

Groups

*

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 600

googlebot
googlebot-image
google page speed insights
mediapartners-google
msnbot
msnbot-media
slurp
yahoo-blogs
yahoo-mmcrawler

Rule Path
Disallow /*/ban/
Disallow /ban/
Disallow /*/stats/
Disallow /stats/
Disallow /*/cart
Disallow /cart
Disallow /*/cart/confirm
Disallow /cart/confirm
Disallow /*/cart/payment
Disallow /cart/payment
Disallow /*/search
Disallow /search
Disallow /*/images/flux/
Disallow /images/flux/
Disallow /*/images/banners/
Disallow /images/banners/
Disallow /*/images/charts/
Disallow /images/charts/
Disallow /*/images/icons/
Disallow /images/icons/
Disallow /*/images/old/
Disallow /images/old/
Disallow /*/images/stars/
Disallow /images/stars/
Disallow /*/images/topventes/
Disallow /images/topventes/
Disallow /*/includes/
Disallow /includes/
Disallow /*/misc/
Disallow /misc/
Disallow /*/modules/
Disallow /modules/
Disallow /*/profiles/
Disallow /profiles/
Disallow /*/sites/
Disallow /sites/
Disallow /*/themes/
Disallow /themes/
Disallow /*/CHANGELOG.txt
Disallow /CHANGELOG.txt
Disallow /*/cron.php
Disallow /cron.php
Disallow /*/INSTALL.mysql.txt
Disallow /INSTALL.mysql.txt
Disallow /*/INSTALL.pgsql.txt
Disallow /INSTALL.pgsql.txt
Disallow /*/install.php
Disallow /install.php
Disallow /*/INSTALL.txt
Disallow /INSTALL.txt
Disallow /*/LICENSE.txt
Disallow /LICENSE.txt
Disallow /*/MAINTAINERS.txt
Disallow /MAINTAINERS.txt
Disallow /*/update.php
Disallow /update.php
Disallow /*/UPGRADE.txt
Disallow /UPGRADE.txt
Disallow /*/xmlrpc.php
Disallow /xmlrpc.php
Disallow /*/shopping_cart.php
Disallow /shopping_cart.php
Disallow /*/advanced_search_result.php
Disallow /advanced_search_result.php
Disallow /*/shopping_cart.php
Disallow /shopping_cart.php
Disallow /*/blockedAccount.php
Disallow /blockedAccount.php
Disallow /*/admin/
Disallow /admin/
Disallow /*/comment/reply/
Disallow /comment/reply/
Disallow /*/contact/
Disallow /contact/
Disallow /*/logout/
Disallow /logout/
Disallow /*/node/add/
Disallow /node/add/
Disallow /*/search/
Disallow /search/
Disallow /*/opensearch/
Disallow /opensearch/
Disallow /*/user/register/
Disallow /user/register/
Disallow /*/user/password/
Disallow /user/password/
Disallow /*/user/login/
Disallow /user/login/
Disallow /*/?q=admin%2F
Disallow /?q=admin%2F
Disallow /*/?q=comment%2Freply%2F
Disallow /?q=comment%2Freply%2F
Disallow /*/?q=contact%2F
Disallow /?q=contact%2F
Disallow /*/?q=logout%2F
Disallow /?q=logout%2F
Disallow /*/?q=node%2Fadd%2F
Disallow /?q=node%2Fadd%2F
Disallow /*/?q=search%2F
Disallow /?q=search%2F
Disallow /*/search?keywords=
Disallow /*/?q=user%2Fpassword%2F
Disallow /?q=user%2Fpassword%2F
Disallow /*/?q=user%2Fregister%2F
Disallow /?q=user%2Fregister%2F
Disallow /*/?q=user%2Flogin%2F
Disallow /?q=user%2Flogin%2F

Other Records

Field Value
sitemap https://www.dlgamer.com/sitemap/sitemap_index.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-fr.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-de.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-en.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-es.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-it.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-eu.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-us.xml
sitemap https://www.dlgamer.com/sitemap/sitemap-nl.xml

Comments

  • disallow all
  • but allow only important bots
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)