Log In Sign Up

taskmonk.ai
robots.txt

Robots Exclusion Standard data for taskmonk.ai

Resource Scan

Scan Details

Site Domain taskmonk.ai
Base Domain taskmonk.ai
Scan Status Ok
Last Scan2026-03-22T18:45:03+00:00
Next Scan 2026-04-21T18:45:03+00:00

Last Scan

Scanned2026-03-22T18:45:03+00:00
URL https://www.taskmonk.ai/robots.txt
Domain IPs 2620:cb:2000::1, 75.2.70.75
Response IP 75.2.70.75
Found Yes
Hash 500b921125408a0c087c3cb26211734ef368a7807bdb229df16f47fc9c73026e
SimHash 6010990525c6

Groups

*

Rule Path
Disallow

googlebot

Rule Path
Disallow

bingbot

Rule Path
Disallow

slurp

Rule Path
Disallow

duckduckbot

Rule Path
Disallow

baiduspider

Rule Path
Disallow

yandexbot

Rule Path
Disallow

applebot

Rule Path
Disallow

gptbot

Rule Path
Disallow

chatgpt-user

Rule Path
Disallow

claudebot

Rule Path
Disallow

perplexitybot

Rule Path
Disallow

google-extended

Rule Path
Disallow

ccbot

Rule Path
Disallow

scaleserpbot

Rule Path
Disallow

amazonbot

Rule Path
Disallow

youbot

Rule Path
Disallow

facebookbot

Rule Path
Disallow

copilotbot

Rule Path
Disallow
Back to top

Other Records

Field Value
sitemap https://www.taskmonk.ai/sitemap.xml
sitemap https://www.taskmonk.ai/sitemap.xml
Back to top

Comments

  • robots.txt for https://www.taskmonk.ai/
  • Full Access Version — allows all search engines and AI crawlers.
  • --- General Allow ---
  • --- Major Search Engine Crawlers ---
  • --- Major LLM / AI Crawlers (as of 2025) ---
  • --- Sitemap for Indexing ---
Back to top