if-sakerhet.se
robots.txt

Robots Exclusion Standard data for if-sakerhet.se

Resource Scan

Scan Details

Site Domain if-sakerhet.se
Base Domain if-sakerhet.se
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2025-11-20T00:19:51+00:00
Next Scan 2025-12-04T00:19:51+00:00

Last Successful Scan

Scanned2025-10-11T19:55:28+00:00
URL https://if-sakerhet.se/robots.txt
Redirect https://www.stgeorge.se/robots.txt
Redirect Domain www.stgeorge.se
Redirect Base stgeorge.se
Domain IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Redirect IPs 151.101.1.124, 151.101.129.124, 151.101.193.124, 151.101.65.124
Response IP 151.101.1.124
Found Yes
Hash 6f87184cac31378d6e8483b0a9b22bee00b3d96fa930185b60ae2ad66e4cae5e
SimHash 3934b953de0a

Groups

googlebot

Rule Path
Allow /

googlebot-image

Rule Path
Allow /

adsbot-google

Rule Path
Allow /

adsbot

Rule Path
Allow /

bingbot

Rule Path
Allow /

adidxbot

Rule Path
Allow /

msnbot-media

Rule Path
Allow /

msnbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

gptbot

Rule Path
Allow /

ia_archiver

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

rogerbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

siteauditbot

Rule Path
Disallow /

semrushbot-ba

Rule Path
Disallow /

semrushbot-si

Rule Path
Disallow /

semrushbot-swa

Rule Path
Disallow /

semrushbot-ct

Rule Path
Disallow /

semrushbot-bm

Rule Path
Disallow /

splitsignalbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

megaindexbot

Rule Path
Disallow /

seo-powersuite-bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

*

Rule Path
Allow /

*

Rule Path Comment
Allow /media/ -
Allow /static/ -
Allow /*.js$ -
Allow /*.css$ -
Allow /*.woff$ -
Allow /*.woff2$ -
Allow /*?p= -
Allow /*?page= -
Disallow /catalog/ för att undvika o-omskrivna kategorier/produkter
Disallow /catalogsearch/ -
Disallow /checkout/ -
Disallow /control/ -
Disallow /customer/ -
Disallow /customize/ -
Disallow /index.php/ -
Disallow /newsletter/ -
Disallow /poll/ -
Disallow /review/ -
Disallow /search/advanced/ -
Disallow /static/ -
Disallow /stores/ -
Disallow /stores/store/ -
Disallow /sendfriend/ -
Disallow /tag/ -
Disallow /wishlist/ -
Disallow /*.php$ blockera direktåtkomst till PHP-skript
Disallow /api/ -
Disallow /app/ -
Disallow /bin/ -
Disallow /errors/ -
Disallow /dev/ -
Disallow /lib/ -
Disallow /phpserver/ -
Disallow /pkginfo/ -
Disallow /pub/ -
Disallow /report/ -
Disallow /setup/ -
Disallow /update/ -
Disallow /var/ -
Disallow /vendor/ -
Disallow /*?*SID= -
Disallow /*?*___SID= -
Disallow /*?*___from_store= -
Disallow /*?*___store= -
Disallow /*?*utm_ -
Disallow /*?*gclid= -
Disallow /*?*msclkid= -
Disallow /*?*fbclid= -
Disallow /*?*dir= -
Disallow /*?*order= -
Disallow /*?*mode= -
Disallow /*?*limit= -
Disallow /*?*price= -
Disallow /*?*manufacturer= -
Disallow /*?*color= -
Disallow /*?*size= -

Other Records

Field Value
sitemap https://www.stgeorge.se/media/sitemap/sitemap_stg_se.xml
sitemap https://www.stgeorge.se/media/sitemap/sitemap_stg_se.xml

Comments

  • robots.txt for stgeorge.se (Magento 2)
  • ALLOWED BOTS
  • Google Search Bot
  • Google Image Bot
  • Google Ads Bot
  • Google Ads Bot
  • Bing's main search crawler
  • Bing's advertising/product crawler (used for Shopping & Ads)
  • Microsoft’s image crawler (if you want product images indexed)
  • Microsoft’s generic crawler (covers older or fallback bot IDs)
  • ChatGPT Main Search Crawler
  • ChatGPT User
  • ChatGPT AI Training
  • DISALLOWED USER AGENTS
  • Alexa: https://support.alexa.com/hc/en-us/articles/200450194-Alexa-s-Web-and-Site-Audit-Crawlers
  • Ahrefs: https://ahrefs.com/robot
  • MOZ: https://moz.com/help/moz-procedures/crawlers/rogerbot
  • MOZ: https://moz.com/help/moz-procedures/crawlers/dotbot
  • Semrush: https://www.semrush.com/bot/
  • Majestic: https://mj12bot.com/
  • SerpStat: https://serpstatbot.com/
  • MegaIndex: https://ru.megaindex.com/blog/seo-bot-detection
  • SEO-PowerSuite-bot: https://www.link-assistant.com/seo-workflow/site-audit.html
  • ALLOWED IMPORTANT RESOURCES
  • Behåll pagination & sidnummer
  • BLOCK INTERNAL / DUPLICATE PATHS
  • BLOCK INTERNAL / SYSTEM CATALOGUE (IF PUBLIC)
  • BLOCK PARAMETERS
  • Sessions/Store-switch
  • Spårningsparametrar
  • Facetter/sortering som skapar tunna dubletter
  • (Lägg till fler egna filterparametrar vid behov)
  • --- Sitemaps ---