up.codes
robots.txt

Robots Exclusion Standard data for up.codes

Resource Scan

Scan Details

Site Domain up.codes
Base Domain up.codes
Scan Status Ok
Last Scan2024-09-10T22:10:39+00:00
Next Scan 2024-10-10T22:10:39+00:00

Last Scan

Scanned2024-09-10T22:10:39+00:00
URL https://up.codes/robots.txt
Domain IPs 108.156.2.11, 108.156.2.13, 108.156.2.20, 108.156.2.69, 2600:9000:2024:3200:1d:a008:8780:93a1, 2600:9000:2024:3600:1d:a008:8780:93a1, 2600:9000:2024:5e00:1d:a008:8780:93a1, 2600:9000:2024:6c00:1d:a008:8780:93a1, 2600:9000:2024:7600:1d:a008:8780:93a1, 2600:9000:2024:9e00:1d:a008:8780:93a1, 2600:9000:2024:cc00:1d:a008:8780:93a1, 2600:9000:2024:f800:1d:a008:8780:93a1
Response IP 65.9.112.39
Found Yes
Hash 4a0a7a12315f1e9f377c5d91fb1b31c427ed0edc827bb817751b85caf4728379
SimHash 741d9910c441

Groups

*

Rule Path
Disallow /

googlebot

Rule Path
Allow /
Disallow /billing
Disallow /bookmark-export
Disallow /illustrations
Disallow /diagrams
Disallow /illustration_images
Disallow /login
Disallow /msearch
Disallow /payment
Disallow /account-creation-confirmation
Disallow /profile
Disallow /project/
Disallow /projects
Disallow /search
Disallow /sign-up
Disallow /users
Disallow /viewer-export/
Disallow /get-copilot
Disallow /get-copilot-ca
Disallow /get-copilot-nyc

bingbot

Rule Path
Allow /
Disallow /billing
Disallow /bookmark-export
Disallow /illustrations
Disallow /diagrams
Disallow /illustration_images
Disallow /login
Disallow /msearch
Disallow /payment
Disallow /account-creation-confirmation
Disallow /profile
Disallow /project/
Disallow /projects
Disallow /search
Disallow /sign-up
Disallow /users
Disallow /viewer-export/
Disallow /get-copilot
Disallow /get-copilot-ca
Disallow /get-copilot-nyc

duckduckbot

Rule Path
Allow /
Disallow /billing
Disallow /bookmark-export
Disallow /illustrations
Disallow /diagrams
Disallow /illustration_images
Disallow /login
Disallow /msearch
Disallow /payment
Disallow /account-creation-confirmation
Disallow /profile
Disallow /project/
Disallow /projects
Disallow /search
Disallow /sign-up
Disallow /users
Disallow /viewer-export/
Disallow /get-copilot
Disallow /get-copilot-ca
Disallow /get-copilot-nyc

amazonbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

awariorssbot
awariosmartbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /*?*smid=

friendlycrawler

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

newsnow

Rule Path
Disallow /

news-please

Rule Path
Disallow /

oai-searchbot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

peer39_crawler
peer39_crawler/1.0

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

quora-bot

Rule Path
Disallow /

scrapy

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

twitterbot

Rule Path
Allow /*?*smid=

yandex

Rule Path
Disallow /

ahrefssiteaudit

Rule Path
Allow /

ahrefsbot

Rule Path
Allow /

Other Records

Field Value
sitemap https://up.codes/sitemap.xml

Comments

  • Yahoo Search should follow Bing rules
  • AI and other bots
  • Technically not required given the above blanket disallow, but keeping to be explicit