swurl.com
robots.txt
Robots Exclusion Standard data for swurl.com
Resource Scan
Scan Details
| Site Domain | swurl.com |
| Base Domain | swurl.com |
| Scan Status | Ok |
| Last Scan | 2024-11-01T14:43:11+00:00 |
| Next Scan | 2024-12-01T14:43:11+00:00 |
Last Scan
| Scanned | 2024-11-01T14:43:11+00:00 |
| URL | http://swurl.com/robots.txt |
| Redirect | https://picclick.com/robots.php |
| Redirect Domain | picclick.com |
| Redirect Base | picclick.com |
| Domain IPs | 54.176.32.72 |
| Redirect IPs | 54.176.32.72 |
| Response IP | 54.176.32.72 |
| Found | Yes |
| Hash | 3fb9f90bedd3fbd82c567badfae9a81a82908a95d6514561d957f687995ddf1e |
| SimHash | d24e7951c053 |
Groups
*
| Rule | Path |
|---|---|
| Disallow | /searchamazon.php |
| Disallow | /amazonsearchresults/ |
| Disallow | /ebaysearchresults/ |
| Disallow | /seller/ |
| Disallow | /item/ |
| Disallow | /track/ |
| Disallow | /pict/ |
| Disallow | /d/ |
| Disallow | /Collectibles/play/ |
| Disallow | /*?q= |
| Disallow | /*?query= |
| Disallow | /*?iframe= |
| Disallow | /*?link= |
| Disallow | /*?perma& |
| Disallow | /*?itemsort= |
| Disallow | /*?type= |
| Disallow | /*?sort= |
| Disallow | /*?categoryId= |
| Disallow | /*?filters= |
| Disallow | /*?directbuy= |
| Disallow | /description.php |
| Disallow | /feed.php |
| Disallow | /amazon.php |
| Disallow | /search.php |
| Disallow | /tools/ |
| Disallow | /popularhtml.php |
| Disallow | /sitemaphtml.php |
| Disallow | /sitemapindex.php |
| Disallow | /sitemap.php |
chatglm-spider
netestate ne crawler
megaindex
megaindex.ru
megaindex.com
skyworkspider
senutobot
bluechipbacklinks
stractbot
ezoicbot
openindexspider
surdotlybot
seokicks
serpstatbot
mbcrawler
proximic
anthropic-ai
claude-web
pyspider
ioncrawl
scoop.it
mojeekbot
ccbot
timpibot
zoominfobot
cincraw
sitecheckerbotcrawler
yak
searchmetricsbot
trendictionbot
gnowitnewsbot
wellknownbot
bytespider
hawaiibot
mediatoolkitbot
scoop.it
seekportbot
semanticscholarbot
velenpublicwebcrawler
keys-so-bot
brightbot
exabot
friendly_crawler
friendlycrawler
gnowitnewsbot/2.0
uptimerobot/2.0
awariorssbot
awariosmartbot
brightedge crawler
claudebot
imagesiftbot
geedoproductsearch
turnitinbot
turnitin robot
seznambot
gptbot
yandex
yandexbot
dataforseobot
barkrowler
neevabot
screaming frog seo spider
qwantify
lamarkbot
yacybot
blexbot
colly
nuclei
petalbot
sogou
sogou spider
sogou web spider
ahrefsbot
mj12bot
domaincrawler
garlikcrawler
dotbot
semrushbot
semrushbot-sa
siteauditbot
semrushbot-ba
semrushbot-si
semrushbot-swa
semrushbot-ct
semrushbot-bm
splitsignalbot
semrushbot-coub
httrack
scrapy
cloudflare
zoombot
brandverity
spiderling
cloudflare always online
cloudflare alwaysonline
cloudflare-traffic-manager
cloudflare-alwaysonline
buck
tigerbot
cloudflare-amp
ltx71
ltx71 - (http://ltx71.com/)
linkfluence
bot@linkfluence.net
evc-batch
zgrab
adscanner
| Rule | Path |
|---|---|
| Disallow | / |
Other Records
| Field | Value |
|---|---|
| sitemap | https://picclick.com/sitemapindex.xml |
Warnings
- 1 invalid line.