Log In Sign Up

amphetamines.com
robots.txt

Robots Exclusion Standard data for amphetamines.com

Resource Scan

Scan Details

Site Domain amphetamines.com
Base Domain amphetamines.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-11-08T19:24:03+00:00
Next Scan 2026-01-07T19:24:03+00:00

Last Successful Scan

Scanned2025-09-09T13:42:19+00:00
URL https://amphetamines.com/robots.txt
Domain IPs 104.21.48.8, 172.67.175.42, 2606:4700:3032::6815:3008, 2606:4700:3035::ac43:af2a
Response IP 172.67.175.42
Found Yes
Hash 2e8cceffc65b07367bce06a4bf54301aeb9b70ba68d00fa141eab3c732644246
SimHash 6024db498c07

Groups

*

Product Comment
* Default rules for Wp
Rule Path Comment
Disallow /wp-admin/ Default rules for Wp
Allow /wp-admin/admin-ajax.php Default rules for Wp

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

claudebot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

applebot

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

imagesiftbot

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

youbot

Rule Path
Disallow /

ia_archiver

Rule Path Comment
Disallow / Block Wayback.org from caching.
Back to top

Comments

  • Amphetamines.com: robots.txt
  • https://amphetamines.com
  • Last modified 23.23 am GMT, 1st Jun 2018
  • >
  • Block GPTBot, an AI-related bot.
  • Block ChatGPT-User, a specific user-agent for ChatGPT interactions.
  • Block Google-Extended, which includes additional Google bots beyond standard crawlers.
  • Block PerplexityBot, used by Perplexity AI.
  • Block Amazonbot, Amazon's web crawling bot.
  • Block ClaudeBot, associated with Anthropic's AI.
  • Block Omgilibot, a bot used for crawling by Omgili.
  • Block FacebookBot, Facebook's web crawler.
  • Block Applebot, used by Apple for web services.
  • Block anthropic-ai, another bot by Anthropic.
  • Block Bytespider, used by various services for crawling.
  • Block Claude-Web, a specific bot for Anthropic's web-related services.
  • Block Diffbot, a bot for extracting structured data from websites.
  • Block ImagesiftBot, likely related to image processing or crawling.
  • Duplicate entry for Omgilibot, keep only one instance.
  • Block Omgili, a bot related to web crawling by Omgili.
  • Block YouBot, another general bot.
  • >
Back to top