thesaurus.plus
robots.txt

Robots Exclusion Standard data for thesaurus.plus

Resource Scan

Scan Details

Site Domain thesaurus.plus
Base Domain thesaurus.plus
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-10-11T21:40:24+00:00
Next Scan 2026-01-09T21:40:24+00:00

Last Successful Scan

Scanned2025-06-14T19:25:05+00:00
URL https://thesaurus.plus/robots.txt
Domain IPs 94.198.55.224
Response IP 94.198.55.224
Found Yes
Hash 460af9fc5b4f09ef360ece0eaf5565a2ab8842089a60ec6eb677a5697d95183c
SimHash c21c1460c7f1

Groups

ahrefsbot

Rule Path
Disallow /

admantx

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ccbot/2.0

Rule Path
Disallow /

criteobot/0.1

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

clickagy intelligence bot v2

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

facebookexternalhit

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

keys-so-bot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

proximic

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

*

Rule Path
Disallow /redirect/*

Warnings

  • `host` is not a known field.