davidrojas.net
robots.txt

Robots Exclusion Standard data for davidrojas.net

Resource Scan

Scan Details

Site Domain davidrojas.net
Base Domain davidrojas.net
Scan Status Ok
Last Scan2025-03-29T15:10:27+00:00
Next Scan 2025-04-05T15:10:27+00:00

Last Scan

Scanned2025-03-29T15:10:27+00:00
URL https://davidrojas.net/robots.txt
Redirect https://www.davidrojas.net/robots.txt
Redirect Domain www.davidrojas.net
Redirect Base davidrojas.net
Domain IPs 162.241.217.90
Redirect IPs 162.241.217.90
Response IP 162.241.217.90
Found Yes
Hash 1cd1161177829c2503abf7548090cbff93f1bd1a0e86fff6b3e75c43f46f1ac8
SimHash 48005a008793

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php

Other Records

Field Value
sitemap https://www.davidrojas.net/index.php?xml_sitemap=params=