gatsbytester.xyz
robots.txt

Robots Exclusion Standard data for gatsbytester.xyz

Resource Scan

Scan Details

Site Domain gatsbytester.xyz
Base Domain gatsbytester.xyz
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-08-28T14:57:20+00:00
Next Scan 2025-11-26T14:57:20+00:00

Last Successful Scan

Scanned2025-05-01T14:56:24+00:00
URL https://gatsbytester.xyz/robots.txt
Domain IPs 104.21.55.86, 172.67.146.91, 2606:4700:3032::ac43:925b, 2606:4700:3037::6815:3756
Response IP 172.67.146.91
Found Yes
Hash bfa9f488cb67bd5931b27419ddea9b8fa19db11ce45bc18020d93949a20d74b5
SimHash 6205490c8d93

Groups

*

Rule Path
Disallow */tags/
Disallow */categories/

Other Records

Field Value
sitemap https://gatsby-advanced-starter-demo.netlify.com/sitemap.xml