Log In Sign Up

wpengine.co
robots.txt

Robots Exclusion Standard data for wpengine.co

Resource Scan

Scan Details

Site Domain wpengine.co
Base Domain wpengine.co
Scan Status Ok
Last Scan2025-09-08T03:20:27+00:00
Next Scan 2025-09-22T03:20:27+00:00

Last Scan

Scanned2025-09-08T03:20:27+00:00
URL https://www.wpengine.co/robots.txt
Redirect https://wpengine.com/robots.txt
Redirect Domain wpengine.com
Redirect Base wpengine.com
Domain IPs 104.18.37.43, 172.64.150.213
Redirect IPs 104.18.37.43, 172.64.150.213
Response IP 104.18.37.43
Found Yes
Hash 6eb0ac26eeca7613e251c159e1f007d3627a7c5a3f01f918d8befd94d2b4dc74
SimHash db70e305e9b8

Groups

*

Rule Path
Disallow /wp-admin/
Allow /wp-admin/admin-ajax.php
Disallow */*/audio/
Disallow /site.webmanifest
Disallow /wp-json/wpe/v1/chat-group
Disallow /wp-json/wpe/v1/opt-in/
Disallow /search/
Disallow /solution-center/tag/
Disallow /*/2023-
Disallow /*/day/
Disallow /*eventDisplay
Disallow /*/?__hstc=
Disallow /*/?utm_
Disallow /*/?nabe=
Disallow /*/?print
Disallow /*/?wtime=
Disallow /*/?coupon=
Disallow /*/?wvideo=
Disallow /*/?_hsenc=
Disallow /*/?local-download=
Disallow /*/?_ga=
Disallow /*/?es_p=
Disallow /*/?w_agcid=
Disallow /*/?kaid=
Disallow /*/?tribe-bar-date=
Disallow /*/?amp=
Disallow /*/?fl_rand_seed=
Disallow /*/?clientId=
Disallow /*/?budget=
Disallow /*/?ss-track=
Disallow /*/?language=
Disallow /*/?inf_contact_key=
Disallow /*/?ref=
Disallow /*/?s=
Disallow /*/?sa=
Disallow /*/?a=
Disallow /*/?p=
Disallow /*/?_gl=
Disallow /*/?cid=
Disallow /*/?o=
Disallow /page/
Back to top

Other Records

Field Value
sitemap https://wpengine.com/sitemap_index.xml
sitemap https://wpengine.com/builders/sitemap_index.xml
sitemap https://wpengine.com/sitemap-au.xml
sitemap https://wpengine.com/sitemap-ca.xml
sitemap https://wpengine.com/sitemap-dk.xml
sitemap https://wpengine.com/sitemap-fi.xml
sitemap https://wpengine.com/sitemap-fr.xml
sitemap https://wpengine.com/sitemap-gb.xml
sitemap https://wpengine.com/sitemap-ie.xml
sitemap https://wpengine.com/sitemap-nl.xml
sitemap https://wpengine.com/sitemap-us.xml
Back to top

Comments

  • Route: Next.js for US
  • Apply to all bots
  • Block the admin area but allow ajax callbacks
  • WEB-3507 Rationale unknown
  • Block API endpoints that don't have indexable content
  • WEB-5082 Google has a number of junk URLs to both of these paths
  • Prevent builder calendar filters to be crawled
  • To optimize crawl budget for SEO
  • Block 404s coming from other sites (added 6 Nov 24)
  • Link to Yoast sitemaps
  • Pages still on 2.0 that wont be in a Yoast Sitemap
Back to top

Warnings

  • 27 invalid lines.
Back to top