Log In Sign Up

thepexcel.com
robots.txt

Robots Exclusion Standard data for thepexcel.com

Resource Scan

Scan Details

Site Domain thepexcel.com
Base Domain thepexcel.com
Scan Status Ok
Last Scan2025-12-12T11:48:16+00:00
Next Scan 2025-12-19T11:48:16+00:00

Last Scan

Scanned2025-12-12T11:48:16+00:00
URL https://thepexcel.com/robots.txt
Redirect https://www.thepexcel.com/robots.txt
Redirect Domain www.thepexcel.com
Redirect Base thepexcel.com
Domain IPs 203.170.190.138
Redirect IPs 203.170.190.138
Response IP 203.170.190.138
Found Yes
Hash 1da21fac1210ee6bae452652e589d52dfd9587bfaa46c9a0c5a95685250b120f
SimHash 683419d54b23

Groups

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

gptbot

Rule Path
Disallow /

*

Rule Path
Disallow /wp-admin/
Disallow /wp-includes/

*

Rule Path
Allow /wp-admin/admin-ajax.php
Back to top

Other Records

Field Value
sitemap https://www.thepexcel.com/sitemap_index.xml
Back to top

Comments

  • Allow OpenAI Search Bot
  • Allow ChatGPT User Bot (for answering queries)
  • Disallow GPTBot from training on the content
  • Disallow all bots from accessing wp-admin and wp-includes
  • Allow specific access to admin-ajax.php for AJAX functionality in WordPress
  • Basic Sitemap
Back to top