douleutaras.gr
robots.txt

Robots Exclusion Standard data for douleutaras.gr

Resource Scan

Scan Details

Site Domain douleutaras.gr
Base Domain douleutaras.gr
Scan Status Ok
Last Scan2024-08-31T20:31:42+00:00
Next Scan 2024-09-30T20:31:42+00:00

Last Scan

Scanned2024-08-31T20:31:42+00:00
URL https://douleutaras.gr/robots.txt
Redirect https://www.douleutaras.gr/robots.txt
Redirect Domain www.douleutaras.gr
Redirect Base douleutaras.gr
Domain IPs 104.26.2.224, 104.26.3.224, 172.67.70.52, 2606:4700:20::681a:2e0, 2606:4700:20::681a:3e0, 2606:4700:20::ac43:4634
Redirect IPs 104.26.2.224, 104.26.3.224, 172.67.70.52, 2606:4700:20::681a:2e0, 2606:4700:20::681a:3e0, 2606:4700:20::ac43:4634
Response IP 104.26.3.224
Found Yes
Hash c60a4ec9278ab6dc5abd7e943ad962f7b50f695b192d97ca40b6ba5884f4edbf
SimHash 4d11404047b4

Groups

npbot-1/2.0

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider-ads

Rule Path
Disallow /

npbot

Rule Path
Disallow /

yahoo!-adcrawler

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

adidxbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

adsbot-google

Rule Path
Allow /

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 3

*

Rule Path
Allow /jobs/create/signup/*/iframe/*
Allow /api/public/category_form/*
Allow /api/public/city/*
Allow /api/public/pricing_csv/*
Disallow /jobs/create_unauthed/*
Disallow /jobs/search*
Disallow /portfolio_items/*
Disallow /sellers/*
Disallow /sellers/search*
Disallow /jobs/create/*
Disallow /site/assets*
Disallow /makedonia/*
Disallow /buyers/*
Disallow /dashboard*
Disallow /test/*
Disallow /partials/header_panel
Disallow /?lkws_11794
Disallow /anatoliki-makedonia-kai-thraki/
Disallow /boreio-aigaio/
Disallow /ipeiros/
Disallow /ionioi-nisoi/
Disallow /notio-aigaio/
Disallow /accounts/
Disallow /pro/
Disallow /profile/sunthetika-koufomata/
Disallow /?utm_source
Disallow /admin/
Disallow /renovation_guide_download/*
Disallow /jobs/*
Disallow /api/*
Disallow /r/*
Disallow /en/*
Disallow /activate-language/*

twitterbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://www.douleutaras.gr/sitemap.xml

Warnings

  • `host` is not a known field.