dev.ngo
robots.txt

Robots Exclusion Standard data for dev.ngo

Resource Scan

Scan Details

Site Domain dev.ngo
Base Domain dev.ngo
Scan Status Ok
Last Scan2025-10-04T14:37:25+00:00
Next Scan 2025-11-03T14:37:25+00:00

Last Scan

Scanned2025-10-04T14:37:25+00:00
URL https://dev.ngo/robots.txt
Redirect https://www.dev.ngo/robots.txt
Redirect Domain www.dev.ngo
Redirect Base dev.ngo
Domain IPs 109.74.193.121, 2a01:7e00:e001:1f00:80::1
Redirect IPs 109.74.193.121, 2a01:7e00:e001:1f00:80::1
Response IP 109.74.193.121
Found Yes
Hash d077064010445865473c0fef351389b28e0de111dab0db0b7c764c2ce7c589b6
SimHash 610cdc52c393

Groups

*

Rule Path
Allow /
Disallow /search/

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.dev.ngo/sitemap.xml