clingal.com
robots.txt

Robots Exclusion Standard data for clingal.com

Resource Scan

Scan Details

Site Domain clingal.com
Base Domain clingal.com
Scan Status Ok
Last Scan3/15/2025, 8:57:05 PM
Next Scan 4/14/2025, 8:57:05 PM

Last Scan

Scanned3/15/2025, 8:57:05 PM
URL https://clingal.com/robots.txt
Domain IPs 2a02:4780:16:f264:5746:ca63:90b9:1492, 93.127.196.236
Response IP 93.127.201.10
Found Yes
Hash de3018840a9b2b1510c26e624554833f57051e031ebec7d80bb71736a4473bc0
SimHash 68f65d03a6a2

Groups

*

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Disallow /tag/
Disallow /page/
Disallow /page/*/?s=
Disallow /*?page=
Disallow /amp/
Disallow /*?amp=
Disallow /rss/
Disallow /category/feed/
Disallow /tag/feed/
Disallow /*?feed=
Disallow /wp-json/
Disallow /?rest_route=
Disallow /*.htm$
Disallow /*.html$
Disallow /wp-login.php?*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$

googlebot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

googlebot-mobile

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

bingbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oai searchbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

gptbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

chatgpt-user

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oncrawl

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

crawlbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

facebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

linkedinbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

pinterestbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

instagram

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.clingal.com/sitemap.xml
sitemap https://www.clingal.com/post-sitemap.xml

Comments

  • START BLOCK
  • ---------------------------
  • Allow social media crawlers