awwwards.com
robots.txt

Robots Exclusion Standard data for awwwards.com

Resource Scan

Scan Details

Site Domain awwwards.com
Base Domain awwwards.com
Scan Status Ok
Last Scan2024-09-18T20:29:15+00:00
Next Scan 2024-09-25T20:29:15+00:00

Last Scan

Scanned2024-09-18T20:29:15+00:00
URL https://awwwards.com/robots.txt
Redirect https://www.awwwards.com/robots.txt
Redirect Domain www.awwwards.com
Redirect Base awwwards.com
Domain IPs 2a00:de40:10:316::100, 5.57.227.251
Redirect IPs 2a00:de40:10:316::100, 5.57.227.251
Response IP 5.57.227.251
Found Yes
Hash 1e9da203e43a543583341e63c412007d598302176e08a81fc2fd71563e4c21ff
SimHash c8555521ff93

Groups

*

Rule Path
Disallow /feed
Disallow /vote/
Disallow /tag/
Disallow /11d12designs/
Disallow /publicclass/
Disallow /design-products/
Disallow /submission/
Disallow /app.php/
Disallow /gallery/
Disallow /search-websites
Disallow /events/tag/
Disallow *favourites?page
Disallow /preview/
Disallow /search-articles
Disallow /inspiration/search
Disallow /websites/?
Disallow /directory/search/?
Disallow /*/likes$
Disallow /*/awards$
Disallow /*/follow$
Disallow /*/votes$
Disallow /*/favorites$
Disallow /elements/*
Disallow /inspiration_search/?
Disallow /*/mobile-excellence-report
Disallow /academy/new/courses/*
Disallow /academy/new/courses/*

msiecrawler

Rule Path
Disallow /

psbot

Rule Path
Disallow /

fasterfox

Rule Path
Disallow /

xenu

Rule Path
Disallow /

sitesucker

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.awwwards.com/sitemap.xml