Log In Sign Up

newbasenewlife.co.uk
robots.txt

Robots Exclusion Standard data for newbasenewlife.co.uk

Resource Scan

Scan Details

Site Domain newbasenewlife.co.uk
Base Domain newbasenewlife.co.uk
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a server error.
Last Scan5/10/2025, 6:56:23 PM
Next Scan 7/9/2025, 6:56:23 PM

Last Successful Scan

Scanned3/12/2025, 12:16:27 PM
URL https://newbasenewlife.co.uk/robots.txt
Redirect https://www.newbasenewlife.co.uk/robots.txt
Redirect Domain www.newbasenewlife.co.uk
Redirect Base newbasenewlife.co.uk
Domain IPs 104.21.91.38, 172.67.209.132, 2606:4700:3034::ac43:d184, 2606:4700:3035::6815:5b26
Redirect IPs 104.21.91.38, 172.67.209.132, 2606:4700:3034::ac43:d184, 2606:4700:3035::6815:5b26
Response IP 104.21.91.38
Found Yes
Hash 75c2b0f2eddbce9a8e54ade7d7dc17bb9e13130f524f5f929349a38c8d5333e0
SimHash 1284a8057c71

Groups

*

Rule Path
Allow /ads.txt

petalbot

Rule Path
Disallow /locations/

semrushbot

Rule Path
Disallow /locations/

grapeshot

Rule Path
Disallow /locations/

grapeshotcrawler

Rule Path
Disallow /locations/
Back to top

Comments

  • See http://www.robotstxt.org/robotstxt.html for documentation on how to use the robots.txt file
  • To ban all spiders from the entire site uncomment the next two lines:
  • Sitemap: https://www.newbasenewlife.com/sitemap-root.xml
  • Sitemap: https://www.newbasenewlife.com/sitemap-root2.xml
  • Sitemap: https://www.newbasenewlife.com/sitemap-root3.xml
  • Sitemap: https://www.newbasenewlife.com/sitemap-root4.xml
  • Sitemap: https://www.newbasenewlife.com/sitemap-root5.xml
  • Sitemap: https://www.newbasenewlife.com/sitemap-root6.xml
  • Sitemap: https://www.newbasenewlife.co.uk/sitemap-travel1.xml
  • Sitemap: https://www.newbasenewlife.co.uk/sitemap-travel2.xml
Back to top