Log In Sign Up

emacswiki.org
robots.txt

Robots Exclusion Standard data for emacswiki.org

Resource Scan

Scan Details

Site Domain emacswiki.org
Base Domain emacswiki.org
Scan Status Ok
Last Scan2024-09-28T23:01:07+00:00
Next Scan 2024-10-28T23:01:07+00:00

Last Scan

Scanned2024-09-28T23:01:07+00:00
URL https://emacswiki.org/robots.txt
Redirect https://www.emacswiki.org/robots.txt
Redirect Domain www.emacswiki.org
Redirect Base emacswiki.org
Domain IPs 178.209.50.237, 2a02:418:6a04:178:209:50:237:1
Redirect IPs 178.209.50.237, 2a02:418:6a04:178:209:50:237:1
Response IP 178.209.50.237
Found Yes
Hash 0bd988601deecd02d86d3ad5f0b4e2649e7c8bf6933c9ea8f5486fb803e22c74
SimHash 17d6d0b06ba0

Groups

x28-job-bot
velenpublicwebcrawler
omgilitbot
google-extended
chatgpt-user
gptbot
magpie-crawler
seokicks
dataforseobot
adsbot
ccbot
barkrowler
petalbot
buck
femtosearchbot
gigabot
mauibot
blexbot
mj12bot
ahrefsbot
seznambot
daumoa
sputnikbot
wbsearchbot
yeti
sogou spider
naverbot
sistrix
spbot
wiederfreibot/1.0
dotbot
xovibot
linguee
seoscanners.net
bubing
serpstatbot

Rule Path
Disallow /

*

Rule Path
Disallow /wiki
Disallow /emacs.pl
Disallow /cgi-bin/emacs
Disallow /cgi-bin/wiki?action=
Disallow /cgi-bin/wiki?search=
Disallow /emacs?action=
Disallow /emacs/?action=
Disallow /emacs-de?action=
Disallow /emacs-en?action=
Disallow /emacs-es?action=
Disallow /emacs-fr?action=
Disallow /emacs-gr?action=
Disallow /emacs-ja?action=
Disallow /emacs-jp?action=
Disallow /emacs-ko?action=
Disallow /emacs-pl?action=
Disallow /emacs-pt?action=
Disallow /emacs-ru?action=
Disallow /emacs-se?action=
Disallow /emacs/download/
Disallow /cgi-bin/latin-1.pl
Disallow /cgi-bin/wiki.pl?action=
Disallow /cgi-bin/wiki.pl?search=
Disallow /cgi-bin/wiki?action=
Disallow /cgi-bin/wiki?search=
Disallow /cgi-bin/wiki/download/
Disallow /cgit

Other Records

Field Value
crawl-delay 240
Back to top

Warnings

  • 1 invalid line.
Back to top