elektormagazine.nl
robots.txt

Robots Exclusion Standard data for elektormagazine.nl

Resource Scan

Scan Details

Site Domain elektormagazine.nl
Base Domain elektormagazine.nl
Scan Status Ok
Last Scan2024-09-23T12:58:51+00:00
Next Scan 2024-10-23T12:58:51+00:00

Last Scan

Scanned2024-09-23T12:58:51+00:00
URL https://elektormagazine.nl/robots.txt
Redirect https://www.elektormagazine.nl/robots.txt
Redirect Domain www.elektormagazine.nl
Redirect Base elektormagazine.nl
Domain IPs 198.211.126.214, 2a03:b0c0:0:1010::5c:1
Redirect IPs 198.211.126.214, 2a03:b0c0:0:1010::5c:1
Response IP 198.211.126.214
Found Yes
Hash a174e888614c1e1690e836f1579cdf600a4ad042ecc74551766aafbee27cd7e2
SimHash 39d65d01c0d1

Groups

*

Rule Path
Disallow /portal/
Disallow /embed/
Disallow /search/
Disallow /banners-elektor/
Disallow /waitress/
Disallow /admin/
Disallow /account/mssc/
Disallow /files/
Disallow /includes/
Disallow /*/*/vote
Disallow /search
Disallow /*%26q%3D*
Disallow /*.pdf$
Disallow /*.PDF$
Disallow /*.zip$
Disallow /*.ZIP$
Disallow /assets/upload/files/EN2014120381.pdf

megaindex.ru

Rule Path
Disallow /

megaindex.com

Rule Path
Disallow /

megaindex.ru/2.0

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

jyxobot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

msnbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

rogerbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 2

baiduspider

Rule Path
Disallow /

cazoodlebot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.elektormagazine.nl/sitemap
sitemap https://www.elektormagazine.nl/sitemap/general
sitemap https://www.elektormagazine.nl/sitemap/news
sitemap https://www.elektormagazine.nl/sitemap/articles
sitemap https://www.elektormagazine.nl/sitemap/magazines
sitemap https://www.elektormagazine.nl/sitemap/categories
sitemap https://www.elektormagazine.nl/sitemap/tags
sitemap https://www.elektormagazine.nl/sitemap/selects
sitemap https://www.elektormagazine.nl/sitemap/newsrooms

Comments

  • Disallowed Sub-Directories
  • Disallowed File Types
  • For specific bots