Log In Sign Up

visualwebtechnologies.com
robots.txt

Robots Exclusion Standard data for visualwebtechnologies.com

Resource Scan

Scan Details

Site Domain visualwebtechnologies.com
Base Domain visualwebtechnologies.com
Scan Status Ok
Last Scan2025-12-12T18:52:10+00:00
Next Scan 2026-01-11T18:52:10+00:00

Last Scan

Scanned2025-12-12T18:52:10+00:00
URL https://visualwebtechnologies.com/robots.txt
Domain IPs 104.21.1.246, 172.67.128.104, 2606:4700:3033::6815:1f6, 2606:4700:3033::ac43:8068
Response IP 104.21.1.246
Found Yes
Hash b30ba3ca052bf3616256a4688f33e9884311214cbbecf2a94cde43234d5ef1e9
SimHash 5c15f930cb75

Groups

googlebot

Rule Path
Allow /*.js*
Allow /*.css*

gptbot

Rule Path
Allow /

oai-searchbot

Rule Path
Allow /

chatgpt-user

Rule Path
Allow /

google-extended

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Disallow /status-updates/wp-admin/
Disallow /author/
Disallow /Category/
Disallow /wp-login.php
Back to top

Other Records

Field Value
sitemap https://visualwebtechnologies.com/page-sitemap.xml
sitemap https://visualwebtechnologies.com/post-sitemap.xml
sitemap https://visualwebtechnologies.com/blog/post-sitemap1.xml
Back to top

Comments

  • Allow ChatGPT / OpenAI crawlers
  • (Optional) allow Google's AI extension if you want
Back to top