Log In Sign Up

anibis.ch
robots.txt

Robots Exclusion Standard data for anibis.ch

Resource Scan

Scan Details

Site Domain anibis.ch
Base Domain anibis.ch
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-06-01T21:04:14+00:00
Next Scan 2024-06-08T21:04:14+00:00

Last Successful Scan

Scanned2023-11-09T15:36:17+00:00
URL https://www.anibis.ch/robots.txt
Domain IPs 104.18.42.195, 172.64.145.61
Response IP 172.64.145.61
Found Yes
Hash 491a78cb8ccf79d93e0d2436bf0ca45e6c6880f39cf1a28fde39fc69191018d2
SimHash c9d471c6db34

Groups

*

Rule Path
Disallow /services/
Disallow /member/
Disallow *login.aspx*
Disallow *sct%3D*
Disallow *aral%3D*
Disallow *aidl%3D*
Disallow *ps%3D*
Disallow *p%3D*
Disallow *ualf%3D*
Disallow *sdc%3D*
Disallow *mid%3D*
Disallow */localite-*
Disallow */localita-*
Disallow */ort-*
Allow /fr/c/toutes-les-rubriques/localite-*
Allow /de/c/alle-kategorien/ort-*
Allow /it/c/tutte-le-categorie/localita-*
Allow /fr/c/animaux-accessoires*/localite-*
Allow /fr/c/emploi*/localite-*
Allow /fr/c/automobiles*/localite-*
Allow /fr/c/immobilier*/localite-*
Allow /fr/c/moto-velo*/localite-*
Allow /fr/c/sante-beaute*/localite-*
Allow /fr/ce/erotique*/localite-*
Allow /de/c/tiere-tierzubehoer*/ort-*
Allow /de/c/job*/ort-*
Allow /de/c/autos-autozubehoer*/ort-*
Allow /de/c/immobilien*/ort-*
Allow /de/c/motorrad-velo*/ort-*
Allow /de/c/gesundheit-beauty*/ort-*
Allow /de/ce/erotik*/ort-*
Allow /it/c/animali-e-accessori*/localita-*
Allow /it/c/impiego*/localita-*
Allow /it/c/automobili*/localita-*
Allow /it/c/immobili*/localita-*
Allow /it/c/moto-biciclette*/localita-*
Allow /it/c/salute-bellezza*/localita-*
Allow /it/ce/erotismo*/localita-*

grapeshot

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.anibis.ch/sitemap/sitemapindex.xml.gz
Back to top

Comments

  • prevent crawling of all locations (catchall)
  • include localized listing overview pages
  • allow crawling of specific locations (exceptions)
Back to top