amitie.fr
robots.txt

Robots Exclusion Standard data for amitie.fr

Resource Scan

Scan Details

Site Domain amitie.fr
Base Domain amitie.fr
Scan Status Ok
Last Scan2024-11-05T03:36:44+00:00
Next Scan 2024-11-12T03:36:44+00:00

Last Scan

Scanned2024-11-05T03:36:44+00:00
URL https://amitie.fr/robots.txt
Redirect https://www.amitie.fr/robots.txt
Redirect Domain www.amitie.fr
Redirect Base amitie.fr
Domain IPs 185.40.101.43
Redirect IPs 185.40.101.43
Response IP 185.40.101.43
Found Yes
Hash ffc336a8c14a7e79173e0752c369ec913e0777e3d8e83d7f389d1706a5332643
SimHash 6154c3100653

Groups

mj12bot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

geekystats.com crawler

Rule Path
Disallow /

jugendschutzprogramm-crawler; info: http://www.jugendschutzprogramm.de

Rule Path
Disallow /

hybridbot (hybrid.ru/about. if our bot caused problems please contact us. contact email: m.lyashkov@targetix.net)

Rule Path
Disallow /

mozilla/5.0 (compatible; grapeshotcrawler/2.0; +http://www.grapeshot.co.uk/crawler.php)

Rule Path
Disallow /

proximic

Rule Path
Disallow /

sirdatabot (+https://semantic-api.docs.sirdata.net/contextual-api/contextual-api/introduction)

Rule Path
Disallow /Account/Activate/

*

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.amitie.fr/sitemap.xml