Log In Sign Up

timeoff.sedgwick.com
robots.txt

Robots Exclusion Standard data for timeoff.sedgwick.com

Resource Scan

Scan Details

Site Domain timeoff.sedgwick.com
Base Domain sedgwick.com
Scan Status Ok
Last Scan2025-05-25T04:12:41+00:00
Next Scan 2025-06-24T04:12:41+00:00

Last Scan

Scanned2025-05-25T04:12:41+00:00
URL https://timeoff.sedgwick.com/robots.txt
Redirect https://timeoff.sedgwick.com/Home/Index?ReturnUrl=%2frobots.txt
Domain IPs 74.116.230.199
Response IP 74.116.230.199
Found Yes
Hash 27d0bcf11b71be06eb3d5ad51b5ec7ee0943e031e5d883016dff8a84514eb28e
SimHash 785419019ed3

Groups

*

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

hunter.io

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

similartechbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

meltwaterbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

serpstatbot

Rule Path
Disallow /

siteimprovebot

Rule Path
Disallow /

aspiegelbot

Rule Path
Disallow /

httrack

Rule Path
Disallow /

wget

Rule Path
Disallow /

curl

Rule Path
Disallow /

googlebot

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /

googlebot-news

Rule Path
Disallow /

googlebot-video

Rule Path
Disallow /

googlebot-mobile

Rule Path
Disallow /

adsbot-google

Rule Path
Disallow /

storebot-google

Rule Path
Disallow /

google-inspectiontool

Rule Path
Disallow /

googleother

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

msnbot

Rule Path
Disallow /

duckduckbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

facebot

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

linkedinbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

mozbot

Rule Path
Disallow /

screaming frog seo spider

Rule Path
Disallow /

majestic-eightybot

Rule Path
Disallow /

uptimerobot

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

agentic

Rule Path
Disallow /

ai article writer

Rule Path
Disallow /

ai content detector

Rule Path
Disallow /

ai dungeon

Rule Path
Disallow /

ai search engine

Rule Path
Disallow /

ai seo crawler

Rule Path
Disallow /

ai writer

Rule Path
Disallow /

ai21 labs

Rule Path
Disallow /

ai2bot

Rule Path
Disallow /

aibot

Rule Path
Disallow /

aimatrix

Rule Path
Disallow /

aisearchbot

Rule Path
Disallow /

ai training

Rule Path
Disallow /

aitraining

Rule Path
Disallow /

alexa

Rule Path
Disallow /

alpha ai

Rule Path
Disallow /

alphaai

Rule Path
Disallow /

amazon bedrock

Rule Path
Disallow /

amazon-kendra

Rule Path
Disallow /

amazon lex

Rule Path
Disallow /

amazon comprehend

Rule Path
Disallow /

amazon sagemaker

Rule Path
Disallow /

amazon silk

Rule Path
Disallow /

amazon textract

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

amelia

Rule Path
Disallow /

anderspinkbot

Rule Path
Disallow /

anthropic

Rule Path
Disallow /

anypicker

Rule Path
Disallow /

anyword

Rule Path
Disallow /

aria browse

Rule Path
Disallow /

articoolo

Rule Path
Disallow /

automated writer

Rule Path
Disallow /

awariorssbot

Rule Path
Disallow /

awariosmartbot

Rule Path
Disallow /

azure

Rule Path
Disallow /

bardbot

Rule Path
Disallow /

brave leo

Rule Path
Disallow /

bytedance

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

catboost

Rule Path
Disallow /

cc-crawler

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatglm

Rule Path
Disallow /

chinchilla

Rule Path
Disallow /

claude

Rule Path
Disallow /

clearscope

Rule Path
Disallow /

cohere

Rule Path
Disallow /

common crawl

Rule Path
Disallow /

commoncrawl

Rule Path
Disallow /

content harmony

Rule Path
Disallow /

content king

Rule Path
Disallow /

content optimizer

Rule Path
Disallow /

content samurai

Rule Path
Disallow /

contentatscale

Rule Path
Disallow /

contentbot

Rule Path
Disallow /

contentedge

Rule Path
Disallow /

conversion ai

Rule Path
Disallow /

copilot

Rule Path
Disallow /

copyai

Rule Path
Disallow /

copymatic

Rule Path
Disallow /

copyscape

Rule Path
Disallow /

cotoyogi

Rule Path
Disallow /

crawlq ai

Rule Path
Disallow /

crawlspace

Rule Path
Disallow /

crew ai

Rule Path
Disallow /

crewai

Rule Path
Disallow /

dall-e

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

dataprovider

Rule Path
Disallow /

deepai

Rule Path
Disallow /

deepl

Rule Path
Disallow /

deepmind

Rule Path
Disallow /

deepseek

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

doubao ai

Rule Path
Disallow /

duckassistbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

falcon

Rule Path
Disallow /

firecrawl

Rule Path
Disallow /

flyriver

Rule Path
Disallow /

frase ai

Rule Path
Disallow /

friendlycrawler

Rule Path
Disallow /

gemini

Rule Path
Disallow /

gemma

Rule Path
Disallow /

genai

Rule Path
Disallow /

genspark

Rule Path
Disallow /

glm

Rule Path
Disallow /

goose

Rule Path
Disallow /

gpt

Rule Path
Disallow /

grammarly

Rule Path
Disallow /

grendizer

Rule Path
Disallow /

grok

Rule Path
Disallow /

gt bot

Rule Path
Disallow /

gtbot

Rule Path
Disallow /

hemingway editor

Rule Path
Disallow /

hugging face

Rule Path
Disallow /

hypotenuse ai

Rule Path
Disallow /

iaskspider

Rule Path
Disallow /

icc-crawler

Rule Path
Disallow /

imagegen

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

img2dataset

Rule Path
Disallow /

ink editor

Rule Path
Disallow /

inkforall

Rule Path
Disallow /

intelliseek

Rule Path
Disallow /

inferkit

Rule Path
Disallow /

isscyberriskcrawler

Rule Path
Disallow /

jasperai

Rule Path
Disallow /

kafkai

Rule Path
Disallow /

kangaroo

Rule Path
Disallow /

keyword density ai

Rule Path
Disallow /

knowledge

Rule Path
Disallow /

komobot

Rule Path
Disallow /

llama

Rule Path
Disallow /

llms

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

marketmuse

Rule Path
Disallow /

meltwater

Rule Path
Disallow /

meta ai

Rule Path
Disallow /

meta-external

Rule Path
Disallow /

metaai

Rule Path
Disallow /

metatagbot

Rule Path
Disallow /

mistral

Rule Path
Disallow /

narrative

Rule Path
Disallow /

neevabot

Rule Path
Disallow /

neural text

Rule Path
Disallow /

neuralseo

Rule Path
Disallow /

nova act

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

open ai

Rule Path
Disallow /

openai

Rule Path
Disallow /

openbot

Rule Path
Disallow /

opentext ai

Rule Path
Disallow /

operator

Rule Path
Disallow /

outwrite

Rule Path
Disallow /

page analyzer ai

Rule Path
Disallow /

pangubot

Rule Path
Disallow /

paperlibot

Rule Path
Disallow /

paraphraser.io

Rule Path
Disallow /

perplexity

Rule Path
Disallow /

petalbot

Rule Path
Disallow /

phindbot

Rule Path
Disallow /

piplbot

Rule Path
Disallow /

prowritingaid

Rule Path
Disallow /

quillbot

Rule Path
Disallow /

robotspider

Rule Path
Disallow /

rytr

Rule Path
Disallow /

saplingai

Rule Path
Disallow /

scalenut

Rule Path
Disallow /

scraper

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

scriptbook

Rule Path
Disallow /

seo content machine

Rule Path
Disallow /

seo robot

Rule Path
Disallow /

sentibot

Rule Path
Disallow /

sidetrade

Rule Path
Disallow /

simplified ai

Rule Path
Disallow /

sitefinity

Rule Path
Disallow /

skydancer

Rule Path
Disallow /

slickwrite

Rule Path
Disallow /

sonic

Rule Path
Disallow /

spin rewriter

Rule Path
Disallow /

spinbot

Rule Path
Disallow /

stability

Rule Path
Disallow /

stablediffusionbot

Rule Path
Disallow /

sudowrite

Rule Path
Disallow /

super agent

Rule Path
Disallow /

surfer ai

Rule Path
Disallow /

text blaze

Rule Path
Disallow /

textcortex

Rule Path
Disallow /

the knowledge ai

Rule Path
Disallow /

timpibot

Rule Path
Disallow /

vidnami ai

Rule Path
Disallow /

webzio

Rule Path
Disallow /

whisper

Rule Path
Disallow /

wordai

Rule Path
Disallow /

wordtune

Rule Path
Disallow /

wormsgtp

Rule Path
Disallow /

wpbot

Rule Path
Disallow /

writecream

Rule Path
Disallow /

writerzen

Rule Path
Disallow /

writescope

Rule Path
Disallow /

writesonic

Rule Path
Disallow /

xai

Rule Path
Disallow /

xbot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

zero gtp

Rule Path
Disallow /

zerochat

Rule Path
Disallow /

zhipu

Rule Path
Disallow /

zimm

Rule Path
Disallow /
Disallow /bin/
Disallow /Content/
Disallow /LeaveRequestForms/
Disallow /Properties/
Disallow /Styles/
Disallow /Scripts/
Disallow /Views/
Disallow /WebReferences/
Disallow /*.sql$
Disallow /*.log$
Disallow /*.ini$
Disallow /*.json$
Disallow /*.yaml$
Disallow /*.xml$
Disallow /*.config$
Back to top

Comments

  • Specific disallow rules for known scraping bots
  • Search Engine Bots
  • Google:
  • Bing:
  • DuckDuckGo:
  • Baidu:
  • Yandex:
  • Apple:
  • Social Media Bots:
  • Facebook:
  • Twitter:
  • LinkedIn:
  • SEO and Analytics Bots:
  • AI and LLM Crawlers:
  • Disallow access to specific sensitive directories
  • Disallow access to specific file types
Back to top

Warnings

  • 3 invalid lines.
  • `<pre style="word-wrap` is not a known field.
Back to top