headlinegen.com
robots.txt

Robots Exclusion Standard data for headlinegen.com

Resource Scan

Scan Details

Site Domain headlinegen.com
Base Domain headlinegen.com
Scan Status Ok
Last Scan2026-03-08T08:09:25+00:00
Next Scan 2026-03-15T08:09:25+00:00

Last Scan

Scanned2026-03-08T08:09:25+00:00
URL https://headlinegen.com/robots.txt
Domain IPs 2.17.106.168, 2.17.106.179, 2600:1413:5000:3c::1735:76ea, 2600:1413:5000:3c::1735:76f5
Response IP 23.59.80.186
Found Yes
Hash a282a65ed376e036c97294bb9cf0ec2ce3ae30847376f5672c2acb44a2ff7af1
SimHash cc10ee02ef12

Groups

*

Rule Path
Disallow /*?
Disallow /api/