lawinery.com
robots.txt

Robots Exclusion Standard data for lawinery.com

Resource Scan

Scan Details

Site Domain lawinery.com
Base Domain lawinery.com
Scan Status Ok
Last Scan2024-10-06T17:48:27+00:00
Next Scan 2024-11-05T17:48:27+00:00

Last Scan

Scanned2024-10-06T17:48:27+00:00
URL https://lawinery.com/robots.txt
Domain IPs 63.247.137.218
Response IP 63.247.137.218
Found Yes
Hash e6e0d33017838e2d5e78b5c1229898eb6a2a029031eca53e9ea519bcfa9bfd9f
SimHash 43774b1e6e1b

Groups

*

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

*

Rule Path
Disallow /cgi-bin/
Disallow /wp-includes/
Disallow /recommended/
Disallow /comments/feed/
Disallow /trackback/
Disallow /index.php
Disallow /xmlrpc.php
Disallow /wp-content/plugins/
Disallow /tag
Disallow /category
Disallow /tag
Disallow /author
Disallow /search
Disallow /feed
Disallow /cgi-bin/
Disallow /wp-content/plugins/
Disallow /wp-content/cache/
Disallow /wp-content/themes/
Disallow /wp-login.php
Disallow /wp-content/themes/DCA/
Disallow /wp-register.php

qwantify

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

bingbot

Rule Path
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

twitterbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

pagesinventory

Rule Path
Disallow /

ninjabot

Rule Path
Allow /

googlebot

Rule Path
Allow /

mediapartners-google*

Rule Path
Allow /

googlebot-image

Rule Path
Allow /wp-content/uploads/

adsbot-google

Rule Path
Allow /

googlebot-mobile

Rule Path
Allow /

mj12bot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

bender

Rule Path
Disallow /

discobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

searchwebengine.net

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sindicebot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

findfiles.net

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

goodzer

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

fast enterprise crawler 6

Rule Path
Disallow /

sensis.com.au web crawler

Rule Path
Disallow /

worio bot heritrix

Rule Path
Disallow /

trovitbot

Rule Path
Disallow /

scoutjet

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

mj12bot

Rule Path
Disallow /

swebot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

exabot

Rule Path
Disallow /

sitebot

Rule Path
Disallow /

twengabot-discover

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

bender

Rule Path
Disallow /

discobot

Rule Path
Disallow /

panscient.com

Rule Path
Disallow /

searchwebengine.net

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

nextgensearchbot

Rule Path
Disallow /

speedy

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

nerdbynature.bot

Rule Path
Disallow /

wbsearchbot

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sindicebot

Rule Path
Disallow /

plukkie

Rule Path
Disallow /

findfiles.net

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

goodzer

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

edisterbot

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

pagepeeker

Rule Path
Disallow /