Log In Sign Up

aryafoods.ir
robots.txt

Robots Exclusion Standard data for aryafoods.ir

Resource Scan

Scan Details

Site Domain aryafoods.ir
Base Domain aryafoods.ir
Scan Status Ok
Last Scan2024-11-11T23:36:11+00:00
Next Scan 2024-12-11T23:36:11+00:00

Last Scan

Scanned2024-11-11T23:36:11+00:00
URL https://aryafoods.ir/robots.txt
Domain IPs 185.94.97.37
Response IP 185.94.97.37
Found Yes
Hash 650680207f908f4ae0243ff10f78db4adaf0ffe438f27702dd09a98d582c3198
SimHash 2451d44045b5

Groups

roverdog

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

duggmirror

Rule Path
Disallow

litefinder

Rule Path
Disallow

slurp

Rule Path
Disallow

baidu

Rule Path
Disallow

googlemediabot*

Rule Path
Disallow

gulliver

Rule Path
Disallow

fast

Rule Path
Disallow

scooter

Rule Path
Disallow

vscooter

Rule Path
Disallow

fast-webcrawler

Rule Path
Disallow

lycos_spider_(t-rex)

Rule Path
Disallow

bingbot

Rule Path
Disallow

msnptc

Rule Path
Disallow

msnbot-media/1.0

Rule Path
Disallow

architextspider

Rule Path
Disallow

zyborg

Rule Path
Disallow

surveybot

Rule Path
Disallow

alexabot

Rule Path
Disallow

yandex

Rule Path
Disallow

jikespider

Rule Path
Disallow

googlebot-image

Rule Path
Disallow

googlebot-mobile

Rule Path
Disallow

msnbot

Rule Path
Disallow

teoma

Rule Path
Disallow

twiceler

Rule Path
Disallow

gigabot

Rule Path
Disallow

scrubby

Rule Path
Disallow

robozilla

Rule Path
Disallow

nutch

Rule Path
Disallow

ia_archiver

Rule Path
Disallow

baiduspider

Rule Path
Disallow

naverbot

Rule Path
Disallow

yeti

Rule Path
Disallow

yahoo-mmcrawler

Rule Path
Disallow

psbot

Rule Path
Disallow

asterias

Rule Path
Disallow

yahoo-blogs/v3.9

Rule Path
Disallow

teleport

Rule Path
Disallow

teleportpro

Rule Path
Disallow

emailcollector

Rule Path
Disallow

emailsiphon

Rule Path
Disallow

webbandit

Rule Path
Disallow

webzip

Rule Path
Disallow

webreaper

Rule Path
Disallow

webstripper

Rule Path
Disallow

web downloader

Rule Path
Disallow

webcopier

Rule Path
Disallow

offline explorer pro

Rule Path
Disallow

httrack website copier

Rule Path
Disallow

offline commander

Rule Path
Disallow

leech

Rule Path
Disallow

websnake

Rule Path
Disallow

blackwidow

Rule Path
Disallow

http weazel

Rule Path
Disallow

googlebot

Rule Path
Disallow

*

Rule Path
Disallow
Disallow sites/plugins
Disallow sites/themes
Disallow sites/cache
Disallow wp-admin
Disallow wp-includes
Disallow wp-content/plugins
Disallow wp-content/cache
Disallow wp-content/themes
Disallow trackback
Disallow wget/
Disallow httpd/
Disallow cgi-bin
Disallow images/
Disallow search
Back to top

Other Records

Field Value
sitemap https://www.aryafoods.ir/sitemap.xml
Back to top

Comments

  • Google AdSense
  • digg mirror
  • too many repeated hits, too quick
  • Yahoo. too many repeated hits, too quick
  • too many repeated hits, too quick
  • Google Image
  • Directory
  • Sitemap
Back to top