quthum.com
robots.txt

Robots Exclusion Standard data for quthum.com

Resource Scan

Scan Details

Site Domain quthum.com
Base Domain quthum.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-08-21T18:43:27+00:00
Next Scan 2025-09-04T18:43:27+00:00

Last Successful Scan

Scanned2025-07-14T14:54:53+00:00
URL https://quthum.com/robots.txt
Domain IPs 191.101.228.207, 2a02:4780:16:43d5:2192:3507:95b4:61cf, 2a02:4780:39:7813:f34c:42d2:b5ed:9b8e, 91.108.100.73
Response IP 77.37.75.24
Found Yes
Hash f3400db5bdcbaf2af1c3cd5b323f4653ffe9c032931268517bd91a1670074365
SimHash 68be5c02a6a3

Groups

*

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Disallow /tag/
Disallow /page/
Disallow /page/*/?s=
Disallow /*?page=
Disallow /amp/
Disallow /*?amp=
Disallow /rss/
Disallow /category/feed/
Disallow /tag/feed/
Disallow /*?feed=
Disallow /wp-json/
Disallow /?rest_route=
Disallow /*.htm$
Disallow /*.html$
Disallow /wp-login.php?*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$

googlebot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

googlebot-mobile

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

bingbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oai searchbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

gptbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

chatgpt-user

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oncrawl

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

crawlbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

facebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

linkedinbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

pinterestbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

instagram

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.quthum.com/sitemap.xml
sitemap https://www.quthum.com/post-sitemap.xml

Comments

  • START BLOCK
  • ---------------------------
  • Allow social media crawlers