Log In Sign Up

askly.pl
robots.txt

Robots Exclusion Standard data for askly.pl

Resource Scan

Scan Details

Site Domain askly.pl
Base Domain askly.pl
Scan Status Ok
Last Scan2025-06-03T14:56:17+00:00
Next Scan 2025-06-10T14:56:17+00:00

Last Scan

Scanned2025-06-03T14:56:17+00:00
URL https://askly.pl/robots.txt
Domain IPs 104.21.1.166, 172.67.129.163, 2606:4700:3033::6815:1a6, 2606:4700:3036::ac43:81a3
Response IP 104.21.1.166
Found Yes
Hash 3009693f147201fa0878af795fc6dc8bd70d96d6d5f46c725ab642007bed3ec6
SimHash ac38965264f3

Groups

*

Rule Path
Allow /ads.txt

semrushbot-sa

Rule Path Comment
Disallow / Semrush
Allow /ads.txt -

semrushbot

Rule Path Comment
Disallow / Semrush
Allow /ads.txt -

rogerbot

Rule Path Comment
Disallow / MOZ
Allow /ads.txt -

dotbot

Rule Path Comment
Disallow / MOZ
Allow /ads.txt -

blexbot

Rule Path Comment
Disallow / Webmeup.com
Allow /ads.txt -

spbot

Rule Path Comment
Disallow / Openlinkprofiler
Allow /ads.txt -

seodiver

Rule Path Comment
Disallow / SEOdiver
Allow /ads.txt -

dataprovider

Rule Path Comment
Disallow / DataProvider.com
Allow /ads.txt -

magpie-crawler

Rule Path Comment
Disallow / BrandWatch.com
Allow /ads.txt -

getintent crawler

Rule Path
Disallow /
Allow /ads.txt

grapeshot

Rule Path
Disallow

doubleverify

Rule Path
Disallow

white ops

Rule Path
Disallow

moatbot

Rule Path
Disallow

ias_crawler

Rule Path
Disallow

forensiq

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

leikibot

Rule Path
Disallow

baidu-yunguance-scanbot(ce.baidu.com)

Rule Path
Disallow

baidu-yunguance-slabot(ce.baidu.com)

Rule Path
Disallow

baidu-yunguance-perfbot(ce.baidu.com)

Rule Path
Disallow

baidu-yunguance-vsbot(ce.baidu.com)

Rule Path
Disallow

sogou web spider

Rule Path
Disallow /
Allow /ads.txt

baiduspider

Rule Path
Disallow /
Allow /ads.txt

naverbot

Rule Path
Disallow /
Allow /ads.txt

yeti

Rule Path
Disallow /
Allow /ads.txt

coccocbot-web

Rule Path
Disallow /
Allow /ads.txt

qwantify

Rule Path
Disallow /
Allow /ads.txt

exabot

Rule Path
Disallow /
Allow /ads.txt

linguee

Rule Path Comment
Disallow / Language tool
Allow /ads.txt -

surdotlybot

Rule Path Comment
Disallow / Sur.ly
Allow /ads.txt -

bubing

Rule Path Comment
Disallow / Bubing academic crawler
Allow /ads.txt -

twitterbot

Rule Path
Disallow

mediapartners-google

Rule Path
Disallow

facebot

Rule Path
Disallow

applebot-extended

No rules defined. All paths allowed.

Back to top

Comments

  • Disallow exotic search engine crawlers
  • Disallow other crawlers
  • Good bots whitelisting:
Back to top

Warnings

  • 3 invalid lines.
Back to top