topsantemedecine.com
robots.txt

Robots Exclusion Standard data for topsantemedecine.com

Resource Scan

Scan Details

Site Domain topsantemedecine.com
Base Domain topsantemedecine.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonCouldn't connect to server.
Last Scan2024-08-20T04:47:14+00:00
Next Scan 2024-11-18T04:47:14+00:00

Last Successful Scan

Scanned2023-10-26T01:16:41+00:00
URL https://topsantemedecine.com/robots.txt
Redirect https://www.topsantemedecine.com/robots.txt
Redirect Domain www.topsantemedecine.com
Redirect Base topsantemedecine.com
Domain IPs 212.83.158.154
Redirect IPs 212.83.158.154
Response IP 212.83.158.154
Found Yes
Hash 3c22082d33c9c3d33a3ad7367acf69539be3f10d55493bf0b72ef538a53d354e
SimHash ab5cdc0266b2

Groups

mauibot (crawler.feedback+wc@gmail.com)

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

httrack

Rule Path
Disallow /

blexbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seekport crawler

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Allow /
Disallow /storage/do_xml/id/

Other Records

Field Value
sitemap https://www.topsantemedecine.com/sitemap.xml