therichardrosereport.com
robots.txt

Robots Exclusion Standard data for therichardrosereport.com

Resource Scan

Scan Details

Site Domain therichardrosereport.com
Base Domain therichardrosereport.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-07-20T20:45:25+00:00
Next Scan 2025-10-18T20:45:25+00:00

Last Successful Scan

Scanned2024-03-05T05:33:48+00:00
URL https://therichardrosereport.com/robots.txt
Domain IPs 104.26.14.187, 104.26.15.187, 172.67.68.190, 2606:4700:20::681a:ebb, 2606:4700:20::681a:fbb, 2606:4700:20::ac43:44be
Response IP 104.26.15.187
Found Yes
Hash e269eca25b6852bb2ff22d8197565dc1a5ca7fd6289bb94236620e551e2e1a69
SimHash de151d4ad652

Groups

scrapy

Rule Path
Allow /

scrapy

Rule Path
Allow /

mozilla/5.0 (compatible; sitemap generator; windows nt 10.0)

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow /wp-login.php
Disallow /wp-signup.php
Disallow /press-this.php
Disallow /remote-login.php
Disallow /activate/
Disallow /?print=print%2F
Disallow /cgi-bin/
Disallow /mshots/v1/
Disallow /next/
Disallow /public.api/
Disallow /feed/
Disallow /trackback/
Disallow /wp-includes/
Disallow /xmlrpc.php
Disallow /wp-
Disallow /wp-content/
Disallow /wp-upload/
Disallow /wp-uploads/
Disallow /you-landed-here-eh/
Disallow /checkout/
Disallow /purchase-confirmation/
Disallow /heres-your-download/
Disallow /purchase-history/
Disallow /signup/
Disallow /subscribe/
Disallow /search_results/
Disallow /transaction-failed/
Disallow /amp/
Disallow /cgi-bin/
Disallow /private/
Disallow /tmp/
Disallow /wpforms-preview/
Disallow /search/
Disallow /success/
Disallow /edd/
Disallow /users/
Disallow /?print=print
Disallow /foo*/

Other Records

Field Value
sitemap https://therichardrosereport.com/sitemap_index.xml
sitemap https://therichardrosereport.com/post-sitemap.xml
sitemap https://therichardrosereport.com/xmlsitemap.xml
sitemap https://therichardrosereport.com/TRRRsitemap.xml

Warnings

  • 11 invalid lines.