Log In Sign Up

amb.org
robots.txt

Robots Exclusion Standard data for amb.org

Resource Scan

Scan Details

Site Domain amb.org
Base Domain amb.org
Scan Status Ok
Last Scan2024-11-08T08:23:14+00:00
Next Scan 2024-12-08T08:23:14+00:00

Last Scan

Scanned2024-11-08T08:23:14+00:00
URL https://amb.org/robots.txt
Redirect https://www.amb.org/robots.txt
Redirect Domain www.amb.org
Redirect Base amb.org
Domain IPs 50.211.207.106
Redirect IPs 50.211.207.106
Response IP 50.211.207.106
Found Yes
Hash ba710d55f74ebbc986fee8daf8c0a8f6095c3eca2614e272e8cc3a55398217c5
SimHash 77adca444eea

Groups

ahrefsbot

Rule Path
Disallow /

aihitbot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

aranea web corpora project

Rule Path
Disallow /

balibot

Rule Path
Disallow /

bananabot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

betabot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

bpimagewalker

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

charityengine bot

Rule Path
Disallow /

cincraw

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

coccocbot-image

Rule Path
Disallow /

coccocbot-web

Rule Path
Disallow /

dataprovider.com

Rule Path
Disallow /

daum

Rule Path
Disallow /

df bot

Rule Path
Disallow /

dispatch

Rule Path
Disallow /

domainstatsbot

Rule Path
Disallow /

domains project

Rule Path
Disallow /

domcopbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ecatch

Rule Path
Disallow /

evc-batch

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

garlikcrawler

Rule Path
Disallow /

getleft

Rule Path
Disallow /

gigabot

Rule Path
Disallow /

haosouspider

Rule Path
Disallow /

hatena::fetcher

Rule Path
Disallow /

heritrix

Rule Path
Disallow /

httrack

Rule Path
Disallow /

hubspot crawler

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

ichiro

Rule Path
Disallow /

jobo

Rule Path
Disallow /

kocmohabt

Rule Path
Disallow /

konturbot

Rule Path
Disallow /

lemurwebcrawler

Rule Path
Disallow /

linkdexbot

Rule Path
Disallow /

linkwalker

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mail.ru

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

mojeekbot

Rule Path
Disallow /

mrsputnik

Rule Path
Disallow /

nbertaupete95

Rule Path
Disallow /

obot

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

pandalytics

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

python-requests

Rule Path
Disallow /

python-urllib

Rule Path
Disallow /

riddler

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

sitesucker

Rule Path
Disallow /

spiderling

Rule Path
Disallow /

superbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

sistrix

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

teleport pro

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

web2map

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

webzip

Rule Path
Disallow /

web downloader

Rule Path
Disallow /

website extractor

Rule Path
Disallow /

wget

Rule Path
Disallow /

windows-live-social-object-extractor

Rule Path
Disallow /

woorankreview

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wysigot

Rule Path
Disallow /

xaldon webspider

Rule Path
Disallow /

xovibot

Rule Path
Disallow /

yacybot

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandexaccessibilitybot

Rule Path
Disallow /

yandexantivirus

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

yandexmetrika

Rule Path
Disallow /

yandexmobilebot

Rule Path
Disallow /

yandexsearch

Rule Path
Disallow /

yandexsearchbrowser

Rule Path
Disallow /

yeti

Rule Path
Disallow /

yisouspider

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

linkchecker

Rule Path
Disallow /cgi-bin
Disallow /common
Disallow /download
Disallow /datasheets
Disallow /discog
Disallow /audio/results.html
Disallow /forum

*

Rule Path
Disallow /cgi-bin
Disallow /common
Disallow /download
Disallow /datasheets
Disallow /discog
Disallow /audio/results.html

Other Records

Field Value
crawl-delay 15
Back to top

Comments

  • @(#)robots.txt 1.21 24/08/06 www.amb.org
  • Disallow: /forum/topic*.html
  • Disallow: /forum/viewforum.php?f=
  • Disallow: /forum/viewtopic.php?p=
  • Disallow: /forum/viewtopic.php?=&p=
  • Disallow: /forum/viewtopic.php?t=
  • Disallow: /forum/viewtopic.php?start=
  • Disallow: /forum/*&view=previous
  • Disallow: /forum/*&view=next
  • Disallow: /forum/*&sid=
  • Disallow: /forum/*&p=
  • Disallow: /forum/*&sd=a
Back to top