webfirm.com
robots.txt

Robots Exclusion Standard data for webfirm.com

Resource Scan

Scan Details

Site Domain webfirm.com
Base Domain webfirm.com
Scan Status Ok
Last Scan2025-10-25T09:06:37+00:00
Next Scan 2025-11-24T09:06:37+00:00

Last Scan

Scanned2025-10-25T09:06:37+00:00
URL https://webfirm.com/robots.txt
Redirect https://www.webfirm.com/robots.txt
Redirect Domain www.webfirm.com
Redirect Base webfirm.com
Domain IPs 54.252.214.180
Redirect IPs 54.252.214.180
Response IP 54.252.214.180
Found Yes
Hash 9f3c8b1ec7f86c982e856020f2002de5b6cf24fd3d059a5dbbbda6c8400ab6d0
SimHash 611d8815fbf7

Groups

*

Rule Path
Disallow /wp-admin
Allow /wp-content/uploads
Allow /wp-content/themes/*
Allow /wp-includes/*

googlebot-image

Rule Path
Disallow
Allow /*

mediapartners-google*

Rule Path
Disallow
Allow /*

mj12bot

Rule Path
Disallow

semrushbot-desktop

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 5

Other Records

Field Value
sitemap https://www.webfirm.com/sitemap_index.xml

Comments

  • Google Image
  • Google AdSense