oxfordwebservices.com
robots.txt

Robots Exclusion Standard data for oxfordwebservices.com

Resource Scan

Scan Details

Site Domain oxfordwebservices.com
Base Domain oxfordwebservices.com
Scan Status Ok
Last Scan2025-04-09T06:45:20+00:00
Next Scan 2025-05-09T06:45:20+00:00

Last Scan

Scanned2025-04-09T06:45:20+00:00
URL https://www.oxfordwebservices.com/robots.txt
Domain IPs 172.66.40.140, 172.66.43.116, 2606:4700:3108::ac42:288c, 2606:4700:3108::ac42:2b74
Response IP 172.66.43.116
Found Yes
Hash 7e13ad54d2c64b6c3d3cc1989d7754c9ec29e8574b31c2cd0ff0a0de9de41798
SimHash 69864b06acb0

Groups

googlebot

Rule Path
Allow /

bingbot

Rule Path
Allow /

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Allow /*.css$
Allow /*.js$
Disallow /wp-includes/
Disallow /wp-content/plugins/
Disallow /wp-content/themes/
Allow /wp-includes/*.css
Allow /wp-includes/*.js
Allow /wp-content/plugins/*.css
Allow /wp-content/plugins/*.js
Allow /wp-content/themes/*.css
Allow /wp-content/themes/*.js
Allow /wp-content/uploads/
Disallow /*?s=
Disallow /search/
Disallow /*.php$
Disallow /*.inc$
Disallow /wp-sitemap-posts-pixpopup-1.xml
Disallow /wp-sitemap-posts-pixheader-1.xml
Disallow /wp-sitemap-posts-pixfooter-1.xml

Other Records

Field Value
sitemap https://www.oxfordwebservices.com/wp-sitemap.xml
sitemap https://www.oxfordwebservices.com/wp-sitemap-posts-post-1.xml
sitemap https://www.oxfordwebservices.com/wp-sitemap-posts-page-1.xml
sitemap https://www.oxfordwebservices.com/wp-sitemap-taxonomies-post_tag-1.xml
sitemap https://www.oxfordwebservices.com/wp-sitemap-taxonomies-category-1.xml

Comments

  • Start Robots.txt
  • Google-specific crawlers
  • Bing-specific crawlers
  • All other crawlers
  • WordPress system files and directories
  • Allow media in wp-content uploads for SEO purposes
  • Block query strings to avoid duplicate content
  • Block specific file types from being indexed
  • Block specific sitemaps that do not need indexing
  • Sitemaps declarations for efficient indexing
  • End Robots.txt