hyrenbostad.se
robots.txt

Robots Exclusion Standard data for hyrenbostad.se

Resource Scan

Scan Details

Site Domain hyrenbostad.se
Base Domain hyrenbostad.se
Scan Status Ok
Last Scan2024-05-30T20:34:47+00:00
Next Scan 2024-06-29T20:34:47+00:00

Last Scan

Scanned2024-05-30T20:34:47+00:00
URL https://hyrenbostad.se/robots.txt
Redirect https://www.hyrenbostad.se/robots.txt
Redirect Domain www.hyrenbostad.se
Redirect Base hyrenbostad.se
Domain IPs 104.21.30.106, 172.67.172.193
Redirect IPs 104.21.30.106, 172.67.172.193
Response IP 104.21.30.106
Found Yes
Hash ba2b34aa88ecb30de4eabd927d4ec80cb4f15af5898c7dea4802fddd0c782993
SimHash 131e5dd3cc20

Groups

adsbot

Rule Path
Disallow /

ahrefsbot
ahrefssiteaudit

Rule Path
Disallow /

alphabot
alphaseobot
alphaseobot-sa

Rule Path
Disallow /

applebot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

amazonbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

awariorssbot
awariosmartbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bubing

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dk.binaryconstructors.iapa

Rule Path
Disallow /

domains project

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

geedobot

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

huaweisymantecspider

Rule Path
Disallow /

ia_archiver

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

imagesiftbot

Rule Path
Disallow /

iisbot

Rule Path
Disallow /

keybot translation-search-machine

Rule Path
Disallow /

kingkevinbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

lunabot

Rule Path
Disallow /

linguee

Rule Path
Disallow /

lumtelbot

Rule Path
Disallow /

mauibot

Rule Path
Disallow /

megaindex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

morningscore

Rule Path
Disallow /

rankactivelinkbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

semrushbot
semrushbot-ba
semrushbot-bm
semrushbot-ct
semrushbot-sa
semrushbot-si
semrushbot-swa

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

siteliner

Rule Path
Disallow /

sogou web spider

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

spiderling

Rule Path
Disallow /

spbot

Rule Path
Disallow /

statastico

Rule Path
Disallow /

toweyabot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

turnitinbot

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

xovibot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

yakazbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 30

*

Rule Path
Disallow /kort/
Disallow /kort
Disallow /*kort$
Disallow /lejeboliger/favoritkort
Disallow /Content/Docs/
Disallow /Handler/
Disallow /Hunter/GetOriginalDescription
Disallow /Hunter/LeaseTipModal
Disallow /Hunter/MoreResults
Disallow /Hunter/ReportLeaseModal
Disallow /Hunter/ResetSearch
Disallow /Hunter/SetFavorite
Disallow /Landlord/DeleteLeaseImage
Disallow /Landlord/ChangeLeaseImageType
Disallow /Landlord/CreateLeaseCallMe
Disallow /Landlord/ReceiveLeaseImage
Disallow /Payment/
Disallow /Shared/SwitchCulture

Other Records

Field Value
sitemap /handler-staticsitemapxml.xml
sitemap /handler-searchsitemapxml.xml