Log In Sign Up

westbromnews.co.uk
robots.txt

Robots Exclusion Standard data for westbromnews.co.uk

Resource Scan

Scan Details

Site Domain westbromnews.co.uk
Base Domain westbromnews.co.uk
Scan Status Ok
Last Scan2024-09-21T08:03:45+00:00
Next Scan 2024-09-28T08:03:45+00:00

Last Scan

Scanned2024-09-21T08:03:45+00:00
URL https://westbromnews.co.uk/robots.txt
Redirect https://www.westbromnews.co.uk/robots.txt
Redirect Domain www.westbromnews.co.uk
Redirect Base westbromnews.co.uk
Domain IPs 104.26.0.20, 104.26.1.20, 172.67.73.150, 2606:4700:20::681a:114, 2606:4700:20::681a:14, 2606:4700:20::ac43:4996
Redirect IPs 104.26.0.20, 104.26.1.20, 172.67.73.150, 2606:4700:20::681a:114, 2606:4700:20::681a:14, 2606:4700:20::ac43:4996
Response IP 104.26.0.20
Found Yes
Hash cd9352516c27e322fb94a0f0e22b30cd3e0661c0ef38d50f4813ea6e8285657a
SimHash 1b20da006430

Groups

*

Rule Path
Disallow /core/wp-admin/
Allow /core/wp-admin/admin-ajax.php
Disallow /?s=

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

gptbot

Rule Path
Disallow /

chatgpt-user

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap https://www.westbromnews.co.uk/sitemap-news.xml
sitemap https://www.westbromnews.co.uk/sitemap_index.xml
Back to top

Comments

  • XML Sitemap & Google News version 5.3.6 - https://status301.net/wordpress-plugins/xml-sitemap-feed/
  • Block Common Crawl
  • Block Google Bard AI
  • Block Open AI
Back to top