archdigest.com
robots.txt

Robots Exclusion Standard data for archdigest.com

Resource Scan

Scan Details

Site Domain archdigest.com
Base Domain archdigest.com
Scan Status Ok
Last Scan2024-11-12T03:29:37+00:00
Next Scan 2024-11-19T03:29:37+00:00

Last Scan

Scanned2024-11-12T03:29:37+00:00
URL https://archdigest.com/robots.txt
Redirect https://www.architecturaldigest.com/robots.txt
Redirect Domain www.architecturaldigest.com
Redirect Base architecturaldigest.com
Domain IPs 3.222.163.76, 34.200.3.79
Redirect IPs 2600:9000:271a:1a00:d:f926:50c0:93a1, 2600:9000:271a:2a00:d:f926:50c0:93a1, 2600:9000:271a:400:d:f926:50c0:93a1, 2600:9000:271a:7200:d:f926:50c0:93a1, 2600:9000:271a:9a00:d:f926:50c0:93a1, 2600:9000:271a:a000:d:f926:50c0:93a1, 2600:9000:271a:c800:d:f926:50c0:93a1, 2600:9000:271a:ec00:d:f926:50c0:93a1, 3.165.82.113, 3.165.82.13, 3.165.82.41, 3.165.82.62
Response IP 3.165.82.62
Found Yes
Hash e4fa27b1191215e67e06050a4ed569f9ec65d426ba9796e777870947e6f34f4b
SimHash 0584c143e6d3

Groups

*

Rule Path
Disallow /user/
Disallow /ontheweb/blogs/delineator
Disallow /preview/
Disallow /video/right_rail*
Disallow /*?redirectURL=
Disallow /*%7B%7Burl%7D%7D
Disallow /*%7B%7Burl%7D%7D
Disallow /auth/*
Disallow /account/*
Disallow /*%7B%7Bsection%7D%7D
Disallow /*%7B%7Bsection%7D%7D
Disallow /search
Disallow /adpro/directory/application
Disallow /adpro/directory/businesses?*=
Allow /adpro/directory/businesses?page=
Disallow /product/
Disallow /user-context?referrer
Disallow /services.min.js?_=
Disallow /com.condenast/yv8*

google-inspectiontool

Rule Path
Allow /

ccbot

Rule Path
Disallow /

google-extended

Rule Path
Disallow /

amazonbot

Rule Path
Disallow /

facebookbot

Rule Path
Disallow /

meta-externalagent

Rule Path
Disallow /

anthropic-ai

Rule Path
Disallow /

claude-web

Rule Path
Disallow /

cohere-ai

Rule Path
Disallow /

omgilibot

Rule Path
Disallow /

omgili

Rule Path
Disallow /

perplexitybot

Rule Path
Disallow /

youbot

Rule Path
Disallow /

bytespider

Rule Path
Disallow /

diffbot

Rule Path
Disallow /

applebot-extended

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.architecturaldigest.com/sitemap.xml
sitemap https://www.architecturaldigest.com/branded-sitemap.xml
sitemap https://www.architecturaldigest.com/feed/google-latest-news/sitemap-google-news
sitemap https://www.architecturaldigest.com/reviews/sitemap.xml
sitemap https://www.architecturaldigest.com/feed/rss
sitemap https://www.architecturaldigest.com/categories-sitemap.xml
sitemap https://www.architecturaldigest.com/contributors-sitemap.xml
sitemap https://www.architecturaldigest.com/business-sitemap.xml
sitemap https://www.architecturaldigest.com/bundles-sitemap.xml