shabekhoobpizza.com
robots.txt

Robots Exclusion Standard data for shabekhoobpizza.com

Resource Scan

Scan Details

Site Domain shabekhoobpizza.com
Base Domain shabekhoobpizza.com
Scan Status Ok
Last Scan2024-10-02T15:58:37+00:00
Next Scan 2024-11-01T15:58:37+00:00

Last Scan

Scanned2024-10-02T15:58:37+00:00
URL https://shabekhoobpizza.com/robots.txt
Domain IPs 185.73.114.139
Response IP 185.73.114.139
Found Yes
Hash da6c9b09d0f5a8786f269ff3d9b2fb74062e1016ed156f5557121abc8cb20d30
SimHash 245452825714

Groups

*

Rule Path
Disallow /stat
Disallow /crm

ninjabot

Rule Path
Allow /
Disallow /stat
Disallow /crm

googlebot

Rule Path
Allow /
Disallow /stat
Disallow /crm

googlebot-mobile

Rule Path
Allow /
Disallow /stat
Disallow /crm

googlebot-image

Rule Path
Allow /
Disallow /stat
Disallow /crm

mediapartners-google

Rule Path
Allow /
Disallow /stat
Disallow /crm

adsbot-google

Rule Path
Allow /
Disallow /stat
Disallow /crm

bingbot

Rule Path
Allow /
Disallow /stat
Disallow /crm

slurp

Rule Path
Allow /
Disallow /stat
Disallow /crm

teoma

Rule Path
Allow /
Disallow /stat
Disallow /crm

twiceler

Rule Path
Allow /
Disallow /stat
Disallow /crm

gigabot

Rule Path
Allow /
Disallow /stat
Disallow /crm

scrubby

Rule Path
Allow /
Disallow /stat
Disallow /crm

robozilla

Rule Path
Allow /
Disallow /stat
Disallow /crm

ia_archiver

Rule Path
Allow /
Disallow /stat
Disallow /crm

baiduspider

Rule Path
Allow /
Disallow /stat
Disallow /crm

naverbot

Rule Path
Allow /
Disallow /stat
Disallow /crm

yeti

Rule Path
Allow /
Disallow /stat
Disallow /crm

yahoo-mmcrawler

Rule Path
Allow /
Disallow /stat
Disallow /crm

psbot

Rule Path
Allow /
Disallow /stat
Disallow /crm

asterias

Rule Path
Allow /
Disallow /stat
Disallow /crm

yahoo-blogs/v3.9

Rule Path
Allow /
Disallow /stat
Disallow /crm

user-agent

Rule Path
Allow /
Disallow /stat
Disallow /crm

Other Records

Field Value
sitemap http://localhost//sitemap.xml

Warnings

  • 2 invalid lines.