safti.fr
robots.txt

Robots Exclusion Standard data for safti.fr

Resource Scan

Scan Details

Site Domain safti.fr
Base Domain safti.fr
Scan Status Ok
Last Scan2024-09-25T03:03:22+00:00
Next Scan 2024-10-25T03:03:22+00:00

Last Scan

Scanned2024-09-25T03:03:22+00:00
URL https://safti.fr/robots.txt
Redirect https://www.safti.fr/robots.txt
Redirect Domain www.safti.fr
Redirect Base safti.fr
Domain IPs 217.70.184.55
Redirect IPs 104.18.16.194, 104.18.17.194, 2606:4700::6812:10c2, 2606:4700::6812:11c2
Response IP 104.18.17.194
Found Yes
Hash 827a4aa05b866143565d79183337cf379e0ee71ff9993dd91c415ee7b66f81f8
SimHash 671444502e8b

Groups

*

Rule Path
Disallow /bien-indisponible
Disallow /recherche
Disallow *?*project=
Disallow /votre-conseiller-safti/*/contact*
Disallow /votre-conseiller-safti/jean-luc-crobeddu
Disallow /*?featured
Disallow /*?from
Disallow /*?about
Disallow /*?sortOrder
Disallow /*?location
Disallow /*?fbclid
Disallow /*?adSummary
Disallow /*?sort

nutch

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

seekport crawler

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.safti.fr/sitemap.xml
sitemap https://www.safti.fr/sitemaps/sitemap.agents.xml
sitemap https://www.safti.fr/sitemaps/sitemap.bien.available.xml
sitemap https://www.safti.fr/sitemaps/sitemap.bien.sale.xml