dez.ro
robots.txt

Robots Exclusion Standard data for dez.ro

Resource Scan

Scan Details

Site Domain dez.ro
Base Domain dez.ro
Scan Status Ok
Last Scan2024-09-27T21:29:28+00:00
Next Scan 2024-10-04T21:29:28+00:00

Last Scan

Scanned2024-09-27T21:29:28+00:00
URL https://dez.ro/robots.txt
Domain IPs 88.99.250.173
Response IP 88.99.250.173
Found Yes
Hash b488665b111a3a91f94d481699fa78987f3e69617118cada0152f80df2bb46b2
SimHash 501c1160a1a3

Groups

*

Rule Path
Allow /
Disallow /orders/
Disallow /contul-meu/
Disallow /oferta/pachete/abonament-nou.html
Disallow /*?pricemin=
Disallow /*pricemax%3D

yandex

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

synthesio

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

amazonbot

Product Comment
amazonbot Amazon's user agent
Rule Path Comment
Disallow / disallow this directory

bytespider

Product Comment
bytespider Amazon's user agent
Rule Path Comment
Disallow / disallow this directory

dotbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dez.ro/anunturi.xml
sitemap https://www.dez.ro/sitemap.xml
sitemap https://www.dez.ro/blog/sitemap_index.xml

Warnings

  • 2 invalid lines.