sysjolt.com
robots.txt

Robots Exclusion Standard data for sysjolt.com

Archived Snapshots

Resource Scan

Scan Details

Site Domain	sysjolt.com
Base Domain	sysjolt.com
Scan Status	Ok
Last Scan	2025-08-30T03:55:12+00:00
Next Scan	2025-09-06T03:55:12+00:00

Last Scan

Scanned	2025-08-30T03:55:12+00:00
URL	https://sysjolt.com/robots.txt
Domain IPs	2607:f1c0:100f:f000::2fe, 74.208.236.43
Response IP	74.208.236.43
Found	Yes
Hash	ce2bca0a72fc9a421a06bbf7699aa161a0381ef8bd894779859d3bced94f1334
SimHash	7b970901c3c6

Groups

ai2bot

Rule	Path
Disallow	/

Rule

Path

Disallow

ai2bot-dolma

Rule	Path
Disallow	/

Rule

Path

Disallow

anthropic-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

applebot-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

brightbot 1.0

Rule	Path
Disallow	/

Rule

Path

Disallow

bytespider

Rule	Path
Disallow	/

Rule

Path

Disallow

ccbot

Rule	Path
Disallow	/

Rule

Path

Disallow

chatgpt-user

Rule	Path
Disallow	/

Rule

Path

Disallow

claude-web

Rule	Path
Disallow	/

Rule

Path

Disallow

claudebot

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-ai

Rule	Path
Disallow	/

Rule

Path

Disallow

cohere-training-data-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

crawlspace

Rule	Path
Disallow	/

Rule

Path

Disallow

diffbot

Rule	Path
Disallow	/

Rule

Path

Disallow

facebookbot

Rule	Path
Disallow	/

Rule

Path

Disallow

friendlycrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

google-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

gptbot

Rule	Path
Disallow	/

Rule

Path

Disallow

icc-crawler

Rule	Path
Disallow	/

Rule

Path

Disallow

imagesiftbot

Rule	Path
Disallow	/

Rule

Path

Disallow

img2dataset

Rule	Path
Disallow	/

Rule

Path

Disallow

isscyberriskcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

kangaroo bot

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalagent

Rule	Path
Disallow	/

Rule

Path

Disallow

meta-externalfetcher

Rule	Path
Disallow	/

Rule

Path

Disallow

oai-searchbot

Rule	Path
Disallow	/

Rule

Path

Disallow

omgili

Rule	Path
Disallow	/

Rule

Path

Disallow

omgilibot

Rule	Path
Disallow	/

Rule

Path

Disallow

pangubot

Rule	Path
Disallow	/

Rule

Path

Disallow

perplexitybot

Rule	Path
Disallow	/

Rule

Path

Disallow

petalbot

Rule	Path
Disallow	/

Rule

Path

Disallow

scrapy

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-ocob

Rule	Path
Disallow	/

Rule

Path

Disallow

semrushbot-swa

Rule	Path
Disallow	/

Rule

Path

Disallow

sidetrade indexer bot

Rule	Path
Disallow	/

Rule

Path

Disallow

timpibot

Rule	Path
Disallow	/

Rule

Path

Disallow

velenpublicwebcrawler

Rule	Path
Disallow	/

Rule

Path

Disallow

webzio-extended

Rule	Path
Disallow	/

Rule

Path

Disallow

youbot

Rule	Path
Disallow	/

Rule

Path

Disallow

ia_archiver

Rule	Path
Disallow	/

Rule

Path

Disallow

archive.org_bot

Rule	Path
Disallow	/

Rule

Path

Disallow

*

Rule	Path
Disallow	/analytics/
Disallow	/images/

Rule

Path

Disallow

/analytics/

Disallow

/images/

Other Records

Field	Value
sitemap	https://www.sysjolt.com/sitemap_index.xml

Field

Value

sitemap

https://www.sysjolt.com/sitemap_index.xml

sysjolt.comrobots.txt

Resource Scan

Scan Details

Last Scan

Groups

ai2bot

ai2bot-dolma

anthropic-ai

applebot-extended

brightbot 1.0

bytespider

ccbot

chatgpt-user

claude-web

claudebot

cohere-ai

cohere-training-data-crawler

crawlspace

diffbot

facebookbot

friendlycrawler

google-extended

gptbot

icc-crawler

imagesiftbot

img2dataset

isscyberriskcrawler

kangaroo bot

meta-externalagent

meta-externalfetcher

oai-searchbot

omgili

omgilibot

pangubot

perplexitybot

petalbot

scrapy

semrushbot-ocob

semrushbot-swa

sidetrade indexer bot

timpibot

velenpublicwebcrawler

webzio-extended

youbot

ia_archiver

archive.org_bot

*

Other Records

sysjolt.com
robots.txt