mydogtag.com
robots.txt

Robots Exclusion Standard data for mydogtag.com

Resource Scan

Scan Details

Site Domain mydogtag.com
Base Domain mydogtag.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-01T00:19:07+00:00
Next Scan 2026-01-30T00:19:07+00:00

Last Successful Scan

Scanned2024-03-18T06:09:39+00:00
URL https://mydogtag.com/robots.txt
Redirect https://www.mydogtag.com/robots.txt
Redirect Domain www.mydogtag.com
Redirect Base mydogtag.com
Domain IPs 104.26.6.89, 104.26.7.89, 172.67.73.216, 2606:4700:20::681a:659, 2606:4700:20::681a:759, 2606:4700:20::ac43:49d8
Redirect IPs 104.26.6.89, 104.26.7.89, 172.67.73.216, 2606:4700:20::681a:659, 2606:4700:20::681a:759, 2606:4700:20::ac43:49d8
Response IP 104.26.6.89
Found Yes
Hash dacd8f35396a7c2f5b82e7a5bcc331e5e990bb59693da4c3807c2afb3148751a
SimHash e80c509489b2

Groups

*

Rule Path
Disallow /thanks.php
Disallow /thanks_feedback.php
Disallow /instagram.php
Disallow /cart.php
Disallow /payments.php
Disallow /payment_instructions.php
Disallow /mydogtags.php
Disallow /*?*example=
Disallow /*?*phrase=
Disallow /*?*t1=*

twitterbot

Rule Path
Disallow

ia_archiver

Rule Path
Disallow /

yandex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

thefind.com/crawler

Rule Path
Disallow /

fatbot

Rule Path
Disallow /

mozilla/5.0 (compatible; fatbot 2.0; http://www.thefind.com/crawler)

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mydogtag.com/sitemap.xml

Comments

  • Disallow: /sitesearch.php
  • Disallow: /*?*lang=
  • User-agent: Googlebot-image
  • Disallow:
  • TheFind
  • TheFind
  • TheFind