vivastreet.co.uk
robots.txt

Robots Exclusion Standard data for vivastreet.co.uk

Resource Scan

Scan Details

Site Domain vivastreet.co.uk
Base Domain vivastreet.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-10-07T14:21:59+00:00
Next Scan 2024-12-06T14:21:59+00:00

Last Successful Scan

Scanned2024-08-09T14:20:43+00:00
URL https://vivastreet.co.uk/robots.txt
Redirect https://www.vivastreet.co.uk/robots.txt
Redirect Domain www.vivastreet.co.uk
Redirect Base vivastreet.co.uk
Domain IPs 104.18.4.18, 104.18.5.18
Redirect IPs 104.18.4.18, 104.18.5.18
Response IP 104.18.5.18
Found Yes
Hash f62a6711049e7c043b76208b84d7a05512c49c016c8d32bb89d6bb58a4245dc3
SimHash 203298c49491

Groups

petalbot

Rule Path
Disallow /

*

Rule Path
Disallow /account/classifieds
Disallow /account_classifieds.php
Disallow /post
Disallow /account_reset_password.php
Disallow /authorize.php
Disallow /api/*
Disallow /*?view=*
Disallow /*?type=*
Disallow /*category_id%3D*
Disallow /*change_geo%3D*
Disallow */r$
Disallow */r?*
Disallow */n$
Disallow */n%2B*
Disallow */g%2B*
Disallow */particulier$
Disallow */particulier%2B*
Disallow */individual$
Disallow */individual%2B*
Disallow */privat$
Disallow */privat%2B*
Disallow */privati$
Disallow */privati%2B*
Disallow */particular$
Disallow */particular%2B*
Disallow */professionnel$
Disallow */professionnel%2B*
Disallow */professional$
Disallow */professional%2B*
Disallow */gewerbliche-anbieter$
Disallow */gewerbliche-anbieter%2B*
Disallow */professionisti$
Disallow */professionisti%2B*
Disallow */profesional$
Disallow */profesional%2B*
Disallow */profissional$
Disallow */profissional%2B*
Disallow */professional%2B*
Disallow */gewerbliche-anbieter$
Disallow */gewerbliche-anbieter%2B*
Disallow /s/ad-limit
Disallow */profile%2B*

Other Records

Field Value
sitemap https://www.vivastreet.co.uk/sitemap/GB/sitemapindex-locations.xml
sitemap https://www.vivastreet.co.uk/sitemap/GB/sitemapindex-adverts.xml
sitemap https://www.vivastreet.co.uk/sitemap/GB/sitemapindex-q.xml
sitemap https://www.vivastreet.co.uk/sitemap/GB/sitemap-static.xml

Comments

  • Block Chinese scraper