pazaruvaj.com
robots.txt

Robots Exclusion Standard data for pazaruvaj.com

Resource Scan

Scan Details

Site Domain pazaruvaj.com
Base Domain pazaruvaj.com
Scan Status Ok
Last Scan2024-06-12T02:50:31+00:00
Next Scan 2024-06-19T02:50:31+00:00

Last Scan

Scanned2024-06-12T02:50:31+00:00
URL https://pazaruvaj.com/robots.txt
Domain IPs 104.18.41.219, 172.64.146.37
Response IP 172.64.146.37
Found Yes
Hash 23d4e19c7f68ab93106828f02b1f9bdb0d988464873c4dd2abb7a152288f685a
SimHash 6509f511930b

Groups

baiduspider

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

lcc

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

seoscanners.net

Rule Path
Disallow /

bubing

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

seekportbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

*

Rule Path
Disallow /Jump.php
Disallow /JumpAuction.php
Disallow /outgoing/
Disallow /Ajax
Disallow /AddProductReview
Disallow /cgi-bin/
Disallow /*q%3A*
Disallow /direct/mapa/*
Disallow /direct/ajax/*
Disallow /direct/bannery/*
Disallow /direct/js/*
Disallow /exit-click-web?*
Disallow /unsecured-data-feeds/
Disallow /unsecured-data-feeds/*.xml$
Disallow /sravnenie-na-produkt/
Disallow /GetPriceChart.php*
Disallow *?searchedQuery=*
Disallow */config/slots-definitions$
Disallow */slots-definitions$
Disallow */config/utils$
Disallow */utils$
Disallow */config/config$
Disallow */config/*
Disallow */config$
Disallow */f%3A*%3A*%2C*
Disallow */f%3A*%3Bl%3A*
Disallow */f%3Al%3A*%3B*
Disallow */f%3Ar%3A*
Disallow */f%3A*r%3A*
Allow */f%3Ar%3A1/$
Disallow /?o=*
Disallow /*?o=*
Disallow /*?*&o=*
Allow /direct/i/*

Other Records

Field Value
sitemap https://www.pazaruvaj.com/sitemaps/sitemap_index.xml

Comments

  • Price-chart
  • SearchRelated
  • Bugs
  • Filters
  • Rating
  • Ordering

Warnings

  • 1 invalid line.