mycompiler.io
robots.txt

Robots Exclusion Standard data for mycompiler.io

Resource Scan

Scan Details

Site Domain mycompiler.io
Base Domain mycompiler.io
Scan Status Ok
Last Scan2024-05-24T20:50:52+00:00
Next Scan 2024-06-23T20:50:52+00:00

Last Scan

Scanned2024-05-24T20:50:52+00:00
URL https://mycompiler.io/robots.txt
Redirect https://www.mycompiler.io/robots.txt
Redirect Domain www.mycompiler.io
Redirect Base mycompiler.io
Domain IPs 152.199.42.208, 2606:2800:247:444:226b:9a6a:ae83:fa03
Redirect IPs 152.199.42.208, 2606:2800:247:444:226b:9a6a:ae83:fa03
Response IP 152.199.42.208
Found Yes
Hash 205b873d7baaffb2f83287cbc42cd05619b521ac939c73f0c5dff2149e809bc3
SimHash 2a4409000537

Groups

*

Rule Path
Disallow /unsupported-browser
Disallow /download/*
Disallow /new/*?fork=*

ccbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.mycompiler.io/sitemap.xml
sitemap https://www.mycompiler.io/recent-sitemap.xml
sitemap https://www.mycompiler.io/guides/sitemap.xml