Log In Sign Up

s.kym-cdn.com
robots.txt

Robots Exclusion Standard data for s.kym-cdn.com

Resource Scan

Scan Details

Site Domain s.kym-cdn.com
Base Domain kym-cdn.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2024-07-13T14:59:47+00:00
Next Scan 2024-10-11T14:59:47+00:00

Last Successful Scan

Scanned2023-02-27T10:51:56+00:00
URL https://s.kym-cdn.com/robots.txt
Domain IPs 117.121.250.56
Response IP 117.121.250.56
Found Yes
Hash f61edcbfb77aa5eb2e501bd7c807938359afb59a0eed16d3eb7005a66448b18d
SimHash 73166e4085b9

Groups

mediapartners-google

Rule Path
Disallow /search

mediapartners

Rule Path
Disallow /search

slurp

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

yahoo! slurp

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

yandex

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

yandexbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

netseer

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

maxpointcrawler

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

blexbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

bingbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 1

msnbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

webindex

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

scrapy

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

baiduspider

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

applebot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

steeler

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

archive.org_bot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

ia_archiver

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

sputnikbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

nutch

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

wesee_bot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

scoutjet

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

spbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

getintent crawler

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

ntentbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

leikibot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

mozilla/5.0 (compatible;contxbot/1.0)

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

contxbot

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

yeti

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

qwantify/vr

Rule Path
Disallow /search

Other Records

Field Value
crawl-delay 30

ccbot

Rule Path
Disallow /

genieo

Rule Path
Disallow /

sogou

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

riddler

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

*

Rule Path
Disallow /search
Disallow /edits
Disallow /*/edits$
Disallow /*/edits?*$
Disallow /*/edits/*
Disallow /users/*/forum_posts
Back to top

Other Records

Field Value
sitemap https://knowyourmeme.com/sitemap.xml
sitemap https://knowyourmeme.com/memes/confirmed/sitemap.xml
sitemap https://knowyourmeme.com/memes/researching/sitemap.xml
sitemap https://knowyourmeme.com/memes/most-viewed/sitemap.xml
sitemap https://knowyourmeme.com/photos/trending/sitemap.xml
sitemap https://knowyourmeme.com/photos/most-liked/sitemap.xml
sitemap https://knowyourmeme.com/photos/most-viewed/sitemap.xml
sitemap https://knowyourmeme.com/photos/most-commented/sitemap.xml
sitemap https://knowyourmeme.com/photos/most-favorited/sitemap.xml
sitemap https://knowyourmeme.com/videos/trending/sitemap.xml
sitemap https://knowyourmeme.com/videos/most-liked/sitemap.xml
sitemap https://knowyourmeme.com/videos/most-viewed/sitemap.xml
sitemap https://knowyourmeme.com/videos/most-commented/sitemap.xml
sitemap https://knowyourmeme.com/videos/most-favorited/sitemap.xml
sitemap https://knowyourmeme.com/episodes/sitemap.xml
Back to top