cdn.burst.com
robots.txt

Robots Exclusion Standard data for cdn.burst.com

Resource Scan

Scan Details

Site Domain cdn.burst.com
Base Domain burst.com
Scan Status Ok
Last Scan2024-06-17T14:02:56+00:00
Next Scan 2024-07-01T14:02:56+00:00

Last Scan

Scanned2024-06-17T14:02:56+00:00
URL https://cdn.burst.com/robots.txt
Domain IPs 108.156.133.106, 108.156.133.128, 108.156.133.19, 108.156.133.77
Response IP 108.156.133.19
Found Yes
Hash 05db2efa76e88a3de53b3ba0c12ed7d7f1452dc21c7baa7535a81bfa2207840c
SimHash 890c8b40d7f2

Groups

bitlybot

Rule Path
Disallow /

*

Rule Path
Disallow /check.php

Comments

  • Tell "bitlybot" not to come here at all
  • Allow all other robots to browse everywhere