pazaruvaj.com
robots.txt

Robots Exclusion Standard data for pazaruvaj.com

Resource Scan

Scan Details

Site Domain pazaruvaj.com
Base Domain pazaruvaj.com
Scan Status Ok
Last Scan2024-11-15T03:39:06+00:00
Next Scan 2024-11-22T03:39:06+00:00

Last Scan

Scanned2024-11-15T03:39:06+00:00
URL https://pazaruvaj.com/robots.txt
Domain IPs 104.18.34.26, 172.64.153.230
Response IP 104.18.34.26
Found Yes
Hash 8446c75e5a261de40535dad30ad1eeccc7009f7e14547beed1bd469abfab32fe
SimHash 6509f515930b

Groups

baiduspider

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

lcc

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

bubing

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /Jump.php
Disallow /JumpAuction.php
Disallow /outgoing/
Disallow /Ajax
Disallow /AddProductReview
Disallow /cgi-bin/
Disallow /*q%3A*
Disallow /direct/mapa/*
Disallow /direct/ajax/*
Disallow /direct/bannery/*
Disallow /direct/js/*
Disallow /unsecured-data-feeds/
Disallow /unsecured-data-feeds/*.xml$
Disallow /sravnenie-na-produkt/
Disallow /GetPriceChart.php*
Disallow *?searchedQuery=*
Disallow */config/slots-definitions$
Disallow */slots-definitions$
Disallow */config/utils$
Disallow */utils$
Disallow */config/config$
Disallow */config/*
Disallow */config$
Disallow */f%3A*%3A*%2C*
Disallow */f%3A*%3Bl%3A*
Disallow */f%3Al%3A*%3B*
Disallow */f%3Ar%3A*
Disallow */f%3A*r%3A*
Allow */f%3Ar%3A1/$
Disallow /?o=*
Disallow /*?o=*
Disallow /*?*&o=*
Allow /direct/i/*

Other Records

Field Value
sitemap https://www.pazaruvaj.com/sitemaps/sitemap_index.xml

Comments

  • Price-chart
  • SearchRelated
  • Bugs
  • Filters
  • Rating
  • Ordering

Warnings

  • 1 invalid line.