dekazeta.net
robots.txt

Robots Exclusion Standard data for dekazeta.net

Resource Scan

Scan Details

Site Domain dekazeta.net
Base Domain dekazeta.net
Scan Status Ok
Last Scan2024-09-23T13:20:13+00:00
Next Scan 2024-09-30T13:20:13+00:00

Last Scan

Scanned2024-09-23T13:20:13+00:00
URL https://dekazeta.net/robots.txt
Redirect https://www.dekazeta.net/robots.txt
Redirect Domain www.dekazeta.net
Redirect Base dekazeta.net
Domain IPs 89.46.109.43
Redirect IPs 89.46.109.43
Response IP 89.46.109.43
Found Yes
Hash 2bf060678df25cc3534d1ea52536d1e97cecf1342a414d8bee9ebda822a1c672
SimHash ab12611e48a1

Groups

*

Rule Path
Allow /wp-admin/admin-ajax.php
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /?s=

compspybot
curious george
cybeye.com
docomo
exb language crawler
ezooms
flamingo_searchengine
genieo
genio
lwnutch
lexxebot
openwebindex
rediffnewsbot
seoengworldbot
scanmine
screaming frog seo spider
shopwiki
showyoubot
sosospider
wocbot
yeti
yeti
youdaobot
daumoa
gsa-crawler
libcrawl
linkdex
magpie-crawler
repparser
rogerbot
sindice-site-manager
sogou spider
sogou
woriobot
yacybot
yolinkbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

*

Rule Path
Disallow /foro/startTopic/
Disallow /foro/*?do=add
Disallow /foro/*?do=submit
Disallow /foro/discover/unread/
Disallow /foro/markallread/
Disallow /foro/staff/
Disallow /foro/online/
Disallow /foro/discover/
Disallow /foro/leaderboard/
Disallow /foro/search/
Disallow /foro/*?advancedSearchForm=
Disallow /foro/register/
Disallow /foro/lostpassword/
Disallow /foro/login/
Disallow /foro/*?sortby=
Disallow /foro/*?filter=
Disallow /foro/*?tab=comments
Disallow /foro/*?do=email
Disallow /foro/*?do=findComment
Disallow /foro/*?do=getLastComment
Disallow /foro/*?do=getNewComment
Disallow /foro/profile/

Other Records

Field Value
sitemap https://www.dekazeta.net/sitemap_index.xml
sitemap https://www.dekazeta.net/foro/sitemap.php

Comments

  • Reglas tipicas para WordPress
  • Bots chungos o que consumen ancho de banda
  • No queremos regalar contenido a la IA de ChatGPT
  • Reglas para foro IPB
  • Block pages with no unique content
  • Block faceted pages and 301 redirect pages
  • Block profile pages as these have little unique value, consume a lot of crawl time and contain hundreds of 301 links