selkiess.com
robots.txt

Robots Exclusion Standard data for selkiess.com

Resource Scan

Scan Details

Site Domain selkiess.com
Base Domain selkiess.com
Scan Status Ok
Last Scan2025-08-23T07:52:18+00:00
Next Scan 2025-09-22T07:52:18+00:00

Last Scan

Scanned2025-08-23T07:52:18+00:00
URL https://selkiess.com/robots.txt
Domain IPs 179.61.189.111, 2a02:4780:15:2a1:743a:1d:a7:d13d, 2a02:4780:16:274e:d1fb:124c:1cbe:ed8a, 91.108.100.142
Response IP 93.127.187.173
Found Yes
Hash b232a79897fce0333fd37444a8f543262e650e78c3cf321059848a5e4e491044
SimHash 48fe5d03a6a3

Groups

*

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Disallow /tag/
Disallow /page/
Disallow /page/*/?s=
Disallow /*?page=
Disallow /amp/
Disallow /*?amp=
Disallow /rss/
Disallow /category/feed/
Disallow /tag/feed/
Disallow /*?feed=
Disallow /wp-json/
Disallow /?rest_route=
Disallow /*.htm$
Disallow /*.html$
Disallow /wp-login.php?*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$

googlebot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

googlebot-mobile

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

bingbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oai searchbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

gptbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

chatgpt-user

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oncrawl

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

crawlbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

facebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

linkedinbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

pinterestbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

instagram

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.selkiess.com/sitemap.xml
sitemap https://www.selkiess.com/post-sitemap.xml

Comments

  • START BLOCK
  • ---------------------------
  • Allow social media crawlers