Log In Sign Up

ecoloads.com.au
robots.txt

Robots Exclusion Standard data for ecoloads.com.au

Resource Scan

Scan Details

Site Domain ecoloads.com.au
Base Domain ecoloads.com.au
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan2024-05-24T10:11:52+00:00
Next Scan 2024-07-23T10:11:52+00:00

Last Successful Scan

Scanned2024-03-03T10:08:27+00:00
URL https://ecoloads.com.au/robots.txt
Domain IPs 108.157.254.115, 108.157.254.33, 108.157.254.88, 108.157.254.9
Response IP 108.157.254.88
Found Yes
Hash 9b67e372a003767c3153e381278d2820a7069304ba54dd050cb5a5398660832c
SimHash bc53975ecef4

Groups

*

Rule Path
Allow /landing-pages/*.xml
Allow /landing-pages/wp-includes/*
Allow /landing-pages/wp-content/*
Disallow /landing-pages
Disallow /ssr
Allow /
Back to top

Other Records

Field Value
sitemap https://ecoloads.com.au/sitemap_index.xml
Back to top

Comments

  • This should be permanently disallowed as these pages are duplicates of landing pages accessed through WP
  • Besides, this cannot be blocked by password as these pages should continue being accessible
  • because wp-admin, wp-login.php and static assets are still accessed through /landing-pages path
  • Alternative representation of pages server-side rendered.
  • These are accessed only by prerender service as a custom origin and responses are proxied back
  • though standard URLs. E.g. / -> /ssr, /green-fleet -> /ssr/green-fleet, /car-transport -> /ssr/car-transport
Back to top