fieroversus.com
robots.txt

Robots Exclusion Standard data for fieroversus.com

Resource Scan

Scan Details

Site Domain fieroversus.com
Base Domain fieroversus.com
Scan Status Ok
Last Scan2025-03-16T05:46:04+00:00
Next Scan 2025-04-15T05:46:04+00:00

Last Scan

Scanned2025-03-16T05:46:04+00:00
URL https://fieroversus.com/robots.txt
Domain IPs 2a02:4780:16:c578:e87f:ee44:3346:b4f2, 77.37.66.48
Response IP 191.101.228.140
Found Yes
Hash badfd9e9f39864dbb69873e0d484146511a47ab6473a1371da4f280bad547745
SimHash 48be5d03a6a2

Groups

*

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Disallow /tag/
Disallow /page/
Disallow /page/*/?s=
Disallow /*?page=
Disallow /amp/
Disallow /*?amp=
Disallow /rss/
Disallow /category/feed/
Disallow /tag/feed/
Disallow /*?feed=
Disallow /wp-json/
Disallow /?rest_route=
Disallow /*.htm$
Disallow /*.html$
Disallow /wp-login.php?*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$

googlebot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

googlebot-mobile

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

bingbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oai searchbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

gptbot

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

chatgpt-user

Rule Path
Disallow */feed/
Disallow /feed/$
Disallow /*?
Disallow /*?=
Disallow /search/
Disallow *?s=*
Allow /*.js
Allow /*.css
Allow /*.jpg
Allow /*.png$
Allow /*.jpeg$
Allow /*.gif$

oncrawl

Rule Path
Disallow /

alexabot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

crawlbot

Rule Path
Disallow /

linkpadbot

Rule Path
Disallow /

sistrix crawler

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

sogou

Rule Path
Disallow /

exabot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

facebot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

twitterbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 30

linkedinbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

pinterestbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

instagram

Rule Path
Allow /

Other Records

Field Value
crawl-delay 10

Other Records

Field Value
sitemap https://www.fieroversus.com/sitemap.xml
sitemap https://www.fieroversus.com/post-sitemap.xml

Comments

  • START BLOCK
  • ---------------------------
  • Allow social media crawlers