defure.com
robots.txt

Robots Exclusion Standard data for defure.com

Resource Scan

Scan Details

Site Domain defure.com
Base Domain defure.com
Scan Status Ok
Last Scan2024-10-21T10:15:23+00:00
Next Scan 2024-11-20T10:15:23+00:00

Last Scan

Scanned2024-10-21T10:15:23+00:00
URL https://defure.com/robots.txt
Redirect https://www.defure.com/robots.txt
Redirect Domain www.defure.com
Redirect Base defure.com
Domain IPs 104.21.92.95, 172.67.190.227, 2606:4700:3031::ac43:bee3, 2606:4700:3036::6815:5c5f
Redirect IPs 104.21.92.95, 172.67.190.227, 2606:4700:3031::ac43:bee3, 2606:4700:3036::6815:5c5f
Response IP 104.21.92.95
Found Yes
Hash 43dce9145d407d9ff1f0371c5c386f18577fc3143de26a082f4cd9526d4b0568
SimHash 79bdc964ce9b

Groups

*

Rule Path
Disallow /mtos/
Disallow /img/
Disallow /images/
Disallow /template/
Disallow /cgi-bin/
Disallow /inc/
Disallow /include/

megalodon

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

mappy

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

qwantify/bleriot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

smtbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.defure.com/sitemap.xml