tatler.com
robots.txt

Robots Exclusion Standard data for tatler.com

Resource Scan

Scan Details

Site Domain tatler.com
Base Domain tatler.com
Scan Status Ok
Last Scan2024-11-08T22:13:40+00:00
Next Scan 2024-11-15T22:13:40+00:00

Last Scan

Scanned2024-11-08T22:13:40+00:00
URL https://tatler.com/robots.txt
Redirect https://www.tatler.com/robots.txt
Redirect Domain www.tatler.com
Redirect Base tatler.com
Domain IPs 2a05:d018:f0d:d700:b950:de51:9b80:2d4f, 2a05:d018:f0d:d701:c35f:807d:3d2f:2b87, 2a05:d018:f0d:d702:675b:fce1:1739:b5cc, 34.251.30.219, 34.255.75.117, 52.31.246.218
Redirect IPs 13.35.210.111, 13.35.210.41, 13.35.210.59, 13.35.210.97, 2600:9000:2078:3600:2:f07a:3880:93a1, 2600:9000:2078:4000:2:f07a:3880:93a1, 2600:9000:2078:4e00:2:f07a:3880:93a1, 2600:9000:2078:6800:2:f07a:3880:93a1, 2600:9000:2078:9600:2:f07a:3880:93a1, 2600:9000:2078:c800:2:f07a:3880:93a1, 2600:9000:2078:e000:2:f07a:3880:93a1, 2600:9000:2078:e00:2:f07a:3880:93a1
Response IP 13.35.210.97
Found Yes
Hash 256a283f83edca343ca1e30829946a7d345ca88b259b90f00e5fd4765e96ea1f
SimHash 251899e1c6b3

Groups

*

Rule Path
Disallow /preview/
Disallow /5574*
Disallow /search
Disallow /xhr/
Disallow /*?_escaped_fragment_=%2F*%2Fimage
Disallow /*?redirectURL=
Disallow /auth/*
Disallow /account/*
Disallow /product/
Disallow /user-context?referrer
Disallow /services.min.js?_=
Disallow /com.condenast/yv8*

google-inspectiontool

Rule Path
Allow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.tatler.com/sitemap.xml
sitemap https://www.tatler.com/feed/rss
sitemap https://www.tatler.com/feed/google-latest-news/sitemap-google-news
sitemap https://www.tatler.com/categories-sitemap.xml
sitemap https://www.tatler.com/contributors-sitemap.xml
sitemap https://www.tatler.com/branded-sitemap.xml
sitemap https://www.tatler.com/sitemap-archive-1.xml
sitemap https://www.tatler.com/bundles-sitemap.xml