my-adaptor.com
robots.txt

Robots Exclusion Standard data for my-adaptor.com

Resource Scan

Scan Details

Site Domain my-adaptor.com
Base Domain my-adaptor.com
Scan Status Ok
Last Scan2025-10-03T02:04:20+00:00
Next Scan 2025-11-02T02:04:20+00:00

Last Scan

Scanned2025-10-03T02:04:20+00:00
URL https://my-adaptor.com/robots.txt
Redirect https://www.my-adaptor.com/robots.txt
Redirect Domain www.my-adaptor.com
Redirect Base my-adaptor.com
Domain IPs 162.214.5.130
Redirect IPs 162.214.5.130
Response IP 162.214.5.130
Found Yes
Hash 3de6d66333f46488474d133a9f32386c3d8fc293edb518fab1a29b223c73f164
SimHash d351577cc610

Groups

*

Rule Path
Disallow /cart/
Disallow /sale_records.php?item=
Disallow /search.php?keywords=
Disallow /cache/
Disallow /api/
Disallow /plugins/
Disallow /data/
Disallow /plus/
Disallow /admin/
Disallow /config/
Disallow /vendor/
Disallow /install/
Disallow /wp/
Disallow /wp-login.php

bingbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

google

Rule Path
Allow /

applebot

Rule Path
Allow /

bingpreview

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

bingpreview

Rule Path
Allow /

pinterest/0.2 (+http://www.pinterest.com/)

Rule Path
Allow /

gptbot

Rule Path
Allow /

archive.org_bot

Rule Path
Allow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

senutobot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

barkrowler

Rule Path
Disallow /

almaden

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

istellabot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

aspseek

Rule Path
Disallow /

axmo

Rule Path
Disallow /

booch

Rule Path
Disallow /

crawler

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

dts agent

Rule Path
Disallow /

downloader

Rule Path
Disallow /

emailcollector

Rule Path
Disallow /

emailsiphon

Rule Path
Disallow /

emailwolf

Rule Path
Disallow /

expired domain sleuth

Rule Path
Disallow /

franklin locator

Rule Path
Disallow /

facebook

Rule Path
Disallow /

gaisbot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

grub

Rule Path
Disallow /

hughcrawler

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

lcabotaccept

Rule Path
Disallow /

iconsurf

Rule Path
Disallow /

iltrovatore-setaccio

Rule Path
Disallow /

indy library

Rule Path
Disallow /

iupui

Rule Path
Disallow /

kittiecentral

Rule Path
Disallow /

iaea.org

Rule Path
Disallow /

larbin

Rule Path
Disallow /

lwp-trivial

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

missigua locator

Rule Path
Disallow /

netresearchserver

Rule Path
Disallow /

nextgensearch

Rule Path
Disallow /

npbot

Rule Path
Disallow /

nutch

Rule Path
Disallow /

objectssearch

Rule Path
Disallow /

oracle ultra search

Rule Path
Disallow /

openai.com

Rule Path
Disallow /

peerbot

Rule Path
Disallow /

pictureofinternet

Rule Path
Disallow /

plantynet

Rule Path
Disallow /

quepasacreep

Rule Path
Disallow /

scspider

Rule Path
Disallow /

soft411

Rule Path
Disallow /

spider.acont.de

Rule Path
Disallow /

sqworm

Rule Path
Disallow /

ssm agent

Rule Path
Disallow /

tamu

Rule Path
Disallow /

theusefulbot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

tutorial crawler

Rule Path
Disallow /

tutorgig

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

webzip

Rule Path
Disallow /

zipppbot

Rule Path
Disallow /

xenu

Rule Path
Disallow /

wotbox

Rule Path
Disallow /

wget

Rule Path
Disallow /

mozdex

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

sogou web spider

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

seokicks

Rule Path
Disallow /

megaindex.ru

Rule Path
Disallow /

velenpublicwebcrawler

Rule Path
Disallow /

backlinkcrawler

Rule Path
Disallow /

exabot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

yellowbrandprotectionbot

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

tinytestbot

Rule Path
Disallow /

my-tiny-bot

Rule Path
Disallow /

fidget-spinner-bot

Rule Path
Disallow /

thesis-research-bot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

crawler

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

python

Rule Path
Disallow /

Warnings

  • 2 invalid lines.