betterleap.com
robots.txt

Robots Exclusion Standard data for betterleap.com

Resource Scan

Scan Details

Site Domain betterleap.com
Base Domain betterleap.com
Scan Status Ok
Last Scan2025-07-27T18:15:41+00:00
Next Scan 2025-08-03T18:15:41+00:00

Last Scan

Scanned2025-07-27T18:15:41+00:00
URL https://betterleap.com/robots.txt
Domain IPs 104.26.12.50, 104.26.13.50, 172.67.73.8, 2606:4700:20::681a:c32, 2606:4700:20::681a:d32, 2606:4700:20::ac43:4908
Response IP 104.26.13.50
Found Yes
Hash f6fbbc4233c749fc95958a04b3ab743fea0758e1d75c3cf0a4d58db403307835
SimHash e2054a5aca12

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Disallow /trackback/
Disallow /xmlrpc.php
Disallow /comments/feed/
Disallow /category/*/*
Disallow */trackback/
Disallow */xmlrpc.php
Disallow */comments/feed/
Disallow /wp-json/
Disallow /*?*

Other Records

Field Value
sitemap https://betterleap.com/sitemap_index.xml