jdwel.com
robots.txt

Robots Exclusion Standard data for jdwel.com

Resource Scan

Scan Details

Site Domain jdwel.com
Base Domain jdwel.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-25T17:48:39+00:00
Next Scan 2024-09-23T17:48:39+00:00

Last Successful Scan

Scanned2023-03-03T19:46:40+00:00
URL https://jdwel.com/robots.txt
Domain IPs 104.26.8.64, 104.26.9.64, 172.67.74.142
Response IP 104.26.9.64
Found Yes
Hash 2ca2f0c552c243a9a69c9de7c3ce05fdc3dd1e9e1530210aa33a304c257bf294
SimHash 6055d5716521

Groups

*

Rule Path
Disallow /?s=*
Disallow /match/
Disallow /matches/

googlebot
googlebot-mobile
googlebot-image
googlebot-news
googlebot-video
bingbot
applebot
slurp
yandex
baiduspider

Rule Path
Allow /match/
Allow /matches/

ahrefsbot
ahrefssiteaudit
blexbot
dataforseobot
caliperbot
dotbot
hubspot
mj12bot
rogerbot
semrushbot

Rule Path
Disallow /wp-json/

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://jdwel.com/sitemap_index.xml

Comments

  • Global rules
  • Search Engines
  • SEO Tools