themouthpiece.com
robots.txt

Robots Exclusion Standard data for themouthpiece.com

Resource Scan

Scan Details

Site Domain themouthpiece.com
Base Domain themouthpiece.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-14T20:10:37+00:00
Next Scan 2025-01-12T20:10:37+00:00

Last Successful Scan

Scanned2023-09-20T21:30:58+00:00
URL https://themouthpiece.com/robots.txt
Redirect https://www.themouthpiece.com/robots.txt
Redirect Domain www.themouthpiece.com
Redirect Base themouthpiece.com
Domain IPs 104.16.248.51, 104.16.249.51
Redirect IPs 104.17.97.5, 104.17.98.5, 2606:4700::6811:6105, 2606:4700::6811:6205
Response IP 104.17.97.5
Found Yes
Hash 93f0b00d8048794d7803fa0e74d1024ef778879f38fa1193bb8f96470de0a067
SimHash cd485f904bf2

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /forum/register/
Disallow /forum/login/
Disallow /forum/account/
Disallow /forum/lost-password/
Disallow /forum/admin.php
Disallow /forum/search/
Disallow /forum/online/
Disallow /forum/misc/contact
Allow /

Other Records

Field Value
sitemap https://www.themouthpiece.com/sitemap.php

Comments

  • robots.txt for http://www.themouthpiece.com/
  • Last changed 08/28/2015
  • Adsense can crawl anything