goldwtapp.com
robots.txt

Robots Exclusion Standard data for goldwtapp.com

Resource Scan

Scan Details

Site Domain goldwtapp.com
Base Domain goldwtapp.com
Scan Status Ok
Last Scan2025-10-29T14:03:51+00:00
Next Scan 2025-11-05T14:03:51+00:00

Last Scan

Scanned2025-10-29T14:03:51+00:00
URL https://goldwtapp.com/robots.txt
Domain IPs 104.21.44.38, 172.67.194.166, 2606:4700:3033::6815:2c26, 2606:4700:3036::ac43:c2a6
Response IP 104.21.44.38
Found Yes
Hash c99d7d9f8158b089d049965001382b3ed8fd218b3815c0cc9916f75ec9706b06
SimHash 4b04de8276b7

Groups

googlebot

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

googlebot-news

Rule Path
Disallow

googlebot-video

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

sogou

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

spbot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

*

Rule Path
Disallow /cdn-cgi/*
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://goldwtapp.com/sitemap_index.xml

Comments

  • robots.txt for WordPress Blog
  • Allow all Google bots
  • Block common bad bots
  • General rules for all crawlers
  • Sitemap