thedesk.matthewkeys.net
robots.txt

Robots Exclusion Standard data for thedesk.matthewkeys.net

Resource Scan

Scan Details

Site Domain thedesk.matthewkeys.net
Base Domain matthewkeys.net
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-12-06T02:35:15+00:00
Next Scan 2025-12-07T02:35:15+00:00

Last Successful Scan

Scanned2025-11-05T17:41:12+00:00
URL https://thedesk.matthewkeys.net/robots.txt
Redirect https://thedesk.net/robots.txt
Redirect Domain thedesk.net
Redirect Base thedesk.net
Domain IPs 67.222.31.239
Redirect IPs 104.21.25.125, 172.67.134.59, 2606:4700:3032::6815:197d, 2606:4700:3034::ac43:863b
Response IP 172.67.134.59
Found Yes
Hash b686ccc0481ccb167e0c65f8f7888f216f78e930c5d5d3c540c4e0d7947d9e90
SimHash c828be422e8b

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /wp-content/uploads/.*

ahrefsbot

Rule Path
Disallow /

*

Rule Path
Disallow /feed/
Disallow */feed
Disallow */feed$
Disallow /feed/$
Disallow /comments/feed
Disallow /?feed=
Disallow /wp-feed

semrushbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

Other Records

Field Value
sitemap https://thedesk.net/news.xml
sitemap https://thedesk.net/post-sitemap1.xml
sitemap https://thedesk.net/post-sitemap2.xml
sitemap https://thedesk.net/post-sitemap3.xml
sitemap https://thedesk.net/post-sitemap4.xml
sitemap https://thedesk.net/post-sitemap5.xml
sitemap https://thedesk.net/post-sitemap6.xml
sitemap https://thedesk.net/post-sitemap7.xml
sitemap https://thedesk.net/page-sitemap1.xml
sitemap https://thedesk.net/category-sitemap1.xml
sitemap https://thedesk.net/post_tag-sitemap1.xml
sitemap https://thedesk.net/post_tag-sitemap2.xml
sitemap https://thedesk.net/author.xml
sitemap https://thedesk.net/sitemaps.xml

Comments

  • Default WP rules
  • Block AhrefsBot
  • Block RSS feeds
  • Block SemrushBot