androidplanet.nl
robots.txt

Robots Exclusion Standard data for androidplanet.nl

Resource Scan

Scan Details

Site Domain androidplanet.nl
Base Domain androidplanet.nl
Scan Status Ok
Last Scan2024-11-11T07:11:13+00:00
Next Scan 2024-11-18T07:11:13+00:00

Last Scan

Scanned2024-11-11T07:11:13+00:00
URL https://androidplanet.nl/robots.txt
Redirect https://www.androidplanet.nl/robots.txt
Redirect Domain www.androidplanet.nl
Redirect Base androidplanet.nl
Domain IPs 104.26.12.3, 104.26.13.3, 172.67.72.74, 2606:4700:20::681a:c03, 2606:4700:20::681a:d03, 2606:4700:20::ac43:484a
Redirect IPs 104.26.12.3, 104.26.13.3, 172.67.72.74, 2606:4700:20::681a:c03, 2606:4700:20::681a:d03, 2606:4700:20::ac43:484a
Response IP 104.26.13.3
Found Yes
Hash 12b914312f83a34a9d8a88eedddd7f50ee492fb8f6483d5c0d605b162d0a13f8
SimHash d09a59b3e442

Groups

*

Rule Path
Disallow /api/
Disallow /app/
Disallow /apple/
Disallow /go/
Disallow /naar-winkel/
Disallow /naar-shop/
Disallow /out/
Disallow /pagina/
Disallow /search/
Disallow /top-apps/
Disallow /wp-content/themes/bigspark/ajax/dynamic-content.php
Disallow /wp/wp-admin/
Disallow /auteur/*/archief/
Disallow /thema/*/archief/
Disallow /*/*/nieuws/archief/
Disallow /*/*/tips/archief/
Disallow /?key=*
Disallow */?s=
Disallow /*/*/prijzen/refurbished/
Disallow /*/pagina/*/?*
Disallow /wp-content/themes/bigspark/ajax/device-url.php
Disallow /featured/
Disallow /wp-json/
Disallow /?rest_route=
Disallow */?categorieen=
Disallow */?type=
Disallow */?themas=
Disallow */?productcategorieen=
Disallow */?merk=
Disallow */?product=

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

eknip

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.androidplanet.nl/sitemap_index.xml
sitemap https://www.androidplanet.nl/google-news-sitemap.xml
sitemap https://www.androidplanet.nl/advisor-sitemap.php

Comments

  • General directives
  • Ban AI bots
  • Ban other bots

Warnings

  • 2 invalid lines.