Log In Sign Up

weatherbug.net
robots.txt

Robots Exclusion Standard data for weatherbug.net

Resource Scan

Scan Details

Site Domain weatherbug.net
Base Domain weatherbug.net
Scan Status Ok
Last Scan2024-05-17T14:19:11+00:00
Next Scan 2024-05-24T14:19:11+00:00

Last Scan

Scanned2024-05-17T14:19:11+00:00
URL https://www.weatherbug.net/robots.txt
Domain IPs 108.156.133.102, 108.156.133.117, 108.156.133.121, 108.156.133.92, 2600:9000:2755:3a00:17:976f:31c0:93a1, 2600:9000:2755:6000:17:976f:31c0:93a1, 2600:9000:2755:6200:17:976f:31c0:93a1, 2600:9000:2755:7c00:17:976f:31c0:93a1, 2600:9000:2755:8800:17:976f:31c0:93a1, 2600:9000:2755:a000:17:976f:31c0:93a1, 2600:9000:2755:b400:17:976f:31c0:93a1, 2600:9000:2755:ea00:17:976f:31c0:93a1
Response IP 108.156.133.121
Found Yes
Hash 568177cf09d59b1a5fd191d2235ad8c2bf8e7779a68c6a68f3917bafe053ff7c
SimHash c66c28d51c37

Groups

*

Rule Path Comment
Disallow /O.*Z Fix for robots thinking alert IDs are links
Disallow /corporate/ Fix for old corporate links
Disallow /alerts/* -
Disallow /life/* -
Disallow /maps/* -
Allow ****************************************************************************************/alerts/$ -
Allow ****************************************************************************************/alerts/weather/$ -
Allow ****************************************************************************************/alerts/spark/$ -
Allow ****************************************************************************************/alerts/spark$ -
Allow ****************************************************************************************/alerts/hurricane/$ -
Allow ****************************************************************************************/alerts/fire/$ -
Allow ****************************************************************************************/alerts/pollen/$ -
Allow ****************************************************************************************/alerts/airquality/$ -
Allow ****************************************************************************************/alerts/severestormrisk/$ -
Allow ****************************************************************************************/alerts/commuter/$ -
Allow ****************************************************************************************/life/pollen/$ -
Allow ****************************************************************************************/life/pollen$ -
Allow ****************************************************************************************/life/cold-flu/$ -
Allow ****************************************************************************************/life/outdoor-sports/$ -
Allow ****************************************************************************************/life/snow-ski/$ -
Allow ****************************************************************************************/life/travel/$ -
Allow ****************************************************************************************/air-quality/$ -
Disallow *********************************************************/alerts/weather/* -
Disallow *********************************************************/alerts/pollen/* -
Disallow *********************************************************/alerts/commuter/* -
Disallow *********************************************************/life/cold-flu/* -
Disallow *********************************************************/life/outdoor-sports/* -
Disallow *********************************************************/life/travel/* -
Disallow *********************************************************/alerts/airquality/* -
Allow *******************************/*-ak-* -
Allow *******************************/*-al-* -
Allow *******************************/*-ar-* -
Allow *******************************/*-az-* -
Allow *******************************/*-ca-* -
Allow *******************************/*-co-* -
Allow *******************************/*-ct-* -
Allow *******************************/*-de-* -
Allow *******************************/*-dc-* -
Allow *******************************/*-fl-* -
Allow *******************************/*-ga-* -
Allow *******************************/*-hi-* -
Allow *******************************/*-id-* -
Allow *******************************/*-il-* -
Allow *******************************/*-in-* -
Allow *******************************/*-ia-* -
Allow *******************************/*-ks-* -
Allow *******************************/*-ky-* -
Allow *******************************/*-la-* -
Allow *******************************/*-me-* -
Allow *******************************/*-md-* -
Allow *******************************/*-ma-* -
Allow *******************************/*-mi-* -
Allow *******************************/*-mn-* -
Allow *******************************/*-ms-* -
Allow *******************************/*-mo-* -
Allow *******************************/*-mt-* -
Allow *******************************/*-ne-* -
Allow *******************************/*-nv-* -
Allow *******************************/*-nh-* -
Allow *******************************/*-nj-* -
Allow *******************************/*-nm-* -
Allow *******************************/*-ny-* -
Allow *******************************/*-nc-* -
Allow *******************************/*-nd-* -
Allow *******************************/*-oh-* -
Allow *******************************/*-ok-* -
Allow *******************************/*-or-* -
Allow *******************************/*-pa-* -
Allow *******************************/*-ri-* -
Allow *******************************/*-sc-* -
Allow *******************************/*-sd-* -
Allow *******************************/*-tn-* -
Allow *******************************/*-tx-* -
Allow *******************************/*-ut-* -
Allow *******************************/*-vt-* -
Allow *******************************/*-va-* -
Allow *******************************/*-wa-* -
Allow *******************************/*-wv-* -
Allow *******************************/*-wi-* -
Allow *******************************/*-wy-* -
Back to top

Other Records

Field Value
sitemap https://www.weatherbug.com/sitemap.xml
Back to top

Comments

  • robots.txt for https://www.weatherbug.com/
  • Disallow: /ad/ #Container for ad files
  • Disallow: /api/ #block Api crawling
  • Limit local page exposure
  • Allow national pages
  • Disallow specific localized pages
Back to top