tmz.com
robots.txt

Robots Exclusion Standard data for tmz.com

Resource Scan

Scan Details

Site Domain tmz.com
Base Domain tmz.com
Scan Status Ok
Last Scan2024-04-23T20:50:42+00:00
Next Scan 2024-04-30T20:50:42+00:00

Last Scan

Scanned2024-04-23T20:50:42+00:00
URL https://tmz.com/robots.txt
Redirect https://www.tmz.com/robots.txt
Redirect Domain www.tmz.com
Redirect Base tmz.com
Domain IPs 13.248.160.137, 76.223.34.124
Redirect IPs 108.157.254.21, 108.157.254.28, 108.157.254.47, 108.157.254.89, 2600:9000:2753:2800:b:ee9a:1d00:93a1, 2600:9000:2753:2e00:b:ee9a:1d00:93a1, 2600:9000:2753:3000:b:ee9a:1d00:93a1, 2600:9000:2753:9600:b:ee9a:1d00:93a1, 2600:9000:2753:c800:b:ee9a:1d00:93a1, 2600:9000:2753:fa00:b:ee9a:1d00:93a1, 2600:9000:2753:fc00:b:ee9a:1d00:93a1, 2600:9000:2753:fe00:b:ee9a:1d00:93a1
Response IP 108.157.254.28
Found Yes
Hash 28f4f276d9773a4dcf3d7e3db5dfed803dfe4b5dd8656db11f398bea087fe421
SimHash 65ad5650e7b3

Groups

googlebot-news

Rule Path
Disallow /photos
Disallow /videos

proximic

Rule Path
Disallow

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

*

Rule Path
Disallow /_/
Disallow */print
Disallow /search
Disallow /xid

Other Records

Field Value
sitemap https://www.tmz.com/sitemaps/article/index.xml
sitemap https://www.tmz.com/sitemaps/gallery/index.xml
sitemap https://www.tmz.com/sitemaps/page/index.xml
sitemap https://www.tmz.com/sitemaps/watch/index.xml
sitemap https://www.tmz.com/sitemaps/news.xml