Log In Sign Up

wagento.com
robots.txt

Robots Exclusion Standard data for wagento.com

Resource Scan

Scan Details

Site Domain wagento.com
Base Domain wagento.com
Scan Status Ok
Last Scan2025-06-22T10:30:36+00:00
Next Scan 2025-07-22T10:30:36+00:00

Last Scan

Scanned2025-06-22T10:30:36+00:00
URL https://wagento.com/robots.txt
Redirect https://www.wagento.com/robots.txt
Redirect Domain www.wagento.com
Redirect Base wagento.com
Domain IPs 104.21.67.171, 172.67.178.251, 2606:4700:3035::ac43:b2fb, 2606:4700:3037::6815:43ab
Redirect IPs 104.21.67.171, 172.67.178.251, 2606:4700:3035::ac43:b2fb, 2606:4700:3037::6815:43ab
Response IP 104.21.67.171
Found Yes
Hash bb8b8b562acb07b911e986d7f525fb35f05a4167266786ae4de7800b9870305f
SimHash 5d247817c6c3

Groups

*

Rule Path
Allow /

*

Rule Path
Disallow /wp-login.php/
Disallow /thank-you/
Disallow /thank-you-for-plan-your-project/

*

Rule Path
Disallow /wp-content/uploads/wp-import-export-lite/
Disallow /wp-admin/
Disallow /panel/

oai-searchbot

Rule Path
Allow /

bingbot

Rule Path
Allow /

googlebot

Rule Path
Allow /

perplexitybot

Rule Path
Allow /

youbot

Rule Path
Allow /

claudebot

Rule Path
Allow /
Back to top

Other Records

Field Value
sitemap https://www.wagento.com/sitemap_index.xml
Back to top

Comments

  • WP Import Export Rule
  • Allow OpenAI's SearchBot
  • Allow Bingbot for Bing/ChatGPT indexing
  • Allow Googlebot
  • Allow Perplexity.ai's crawler
  • Allow You.com's bot
  • Allow Anthropic's ClaudeBot
Back to top