Log In Sign Up

temp-mail.lol
robots.txt

Robots Exclusion Standard data for temp-mail.lol

Resource Scan

Scan Details

Site Domain temp-mail.lol
Base Domain temp-mail.lol
Scan Status Ok
Last Scan2026-01-31T07:30:50+00:00
Next Scan 2026-02-07T07:30:50+00:00

Last Scan

Scanned2026-01-31T07:30:50+00:00
URL https://temp-mail.lol/robots.txt
Domain IPs 216.24.57.1
Response IP 216.24.57.1
Found Yes
Hash 129bc32d33aa9e083ecba141331134635f220c969deda8a11d3fe7040149c80f
SimHash 6a389900e6b0

Groups

*

Rule Path
Allow /
Disallow /api/
Disallow /admin/
Disallow /private/
Disallow /temp/
Disallow /*.json$
Disallow /dist/
Disallow /node_modules/
Disallow /.git/

googlebot

Rule Path
Allow /
Allow /*.js$
Allow /*.css$
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$
Allow /*.woff$
Allow /*.woff2$
Allow /*.ico$

Other Records

Field Value
crawl-delay 1

bingbot

Rule Path
Allow /

Other Records

Field Value
crawl-delay 1

googlebot-image

Rule Path
Allow /
Allow /*.png$
Allow /*.jpg$
Allow /*.jpeg$
Allow /*.gif$
Allow /*.webp$
Allow /*.svg$

googlebot-mobile

Rule Path
Allow /

googlebot-news

Rule Path
Allow /blog/
Allow /

googlebot-video

Rule Path
Disallow /

facebookexternalhit

Rule Path
Allow /

twitterbot

Rule Path
Allow /

linkedinbot

Rule Path
Allow /

gptbot

Rule Path
Allow /
Allow /llm.txt

chatgpt-user

Rule Path
Allow /
Allow /llm.txt

claude-web

Rule Path
Allow /
Allow /llm.txt

anthropic-ai

Rule Path
Allow /
Allow /llm.txt

google-extended

Rule Path
Allow /
Allow /llm.txt

ccbot

Rule Path
Allow /
Allow /llm.txt

omgilibot

Rule Path
Allow /
Allow /llm.txt

facebookbot

Rule Path
Allow /
Allow /llm.txt

applebot-extended

Rule Path
Allow /
Allow /llm.txt

perplexitybot

Rule Path
Allow /
Allow /llm.txt

diffbot

Rule Path
Allow /
Allow /llm.txt

cohere-ai

Rule Path
Allow /
Allow /llm.txt

mj12bot

Rule Path
Disallow /

ahrefsbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10
crawl-delay 3
Back to top

Other Records

Field Value
sitemap https://temp-mail.lol/sitemap.xml
sitemap https://temp-mail.lol/sitemap-index.xml
Back to top

Comments

  • robots.txt for temp-mail.lol - Optimized for SEO
  • Last updated: 2025-08-16
  • Default rules for all crawlers
  • High-priority crawl bots with optimized settings
  • Google's specialized crawlers
  • Social media crawlers
  • LLM and AI Crawlers - Allow for training and context
  • Block aggressive crawlers that can impact performance
  • Sitemap locations
  • LLM context file for AI agents
  • Learn more: https://llmstxt.org/
  • Default crawl delay for other bots
  • Host directive to specify preferred domain
Back to top

Warnings

  • `host` is not a known field.
  • `llm` is not a known field.
Back to top