horecamag.ro
robots.txt

Robots Exclusion Standard data for horecamag.ro

Resource Scan

Scan Details

Site Domain horecamag.ro
Base Domain horecamag.ro
Scan Status Ok
Last Scan2024-10-29T07:41:54+00:00
Next Scan 2024-11-28T07:41:54+00:00

Last Scan

Scanned2024-10-29T07:41:54+00:00
URL https://horecamag.ro/robots.txt
Domain IPs 89.40.32.100
Response IP 89.40.32.100
Found Yes
Hash a4846e5076340ff016a588b455b963183f4644b19bfa7e7e26d3b60dfca60702
SimHash 2d004b76e292

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Allow /

*

Rule Path
Disallow /*add-to-cart%3D*
Disallow /cgi-bin/
Disallow /finalizare-comanda/
Disallow /wp-admin/
Disallow /wp-admin/
Disallow /archives/
Disallow *?replytocom
Disallow /comments/feed/
Allow /wp-admin/admin-ajax.php

mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

baiduspider

Rule Path
Disallow /

baiduspdeir-image

Rule Path
Disallow /

baiduspider-video

Rule Path
Disallow /

baiduspider-news

Rule Path
Disallow /

baiduspider-favo

Rule Path
Disallow /

baiduspider-cpro

Rule Path
Disallow /

baiduspider-sfkr

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.horecamag.ro/sitemap-index.xml
sitemap https://www.horecamag.ro/sitemap-images.xml
sitemap https://www.horecamag.ro/sitemap-videos.xml

Comments

  • disallow all files in these directories