unix.com
robots.txt

Robots Exclusion Standard data for unix.com

Resource Scan

Scan Details

Site Domain unix.com
Base Domain unix.com
Scan Status Ok
Last Scan2024-10-30T19:36:37+00:00
Next Scan 2024-11-06T19:36:37+00:00

Last Scan

Scanned2024-10-30T19:36:37+00:00
URL https://unix.com/robots.txt
Domain IPs 209.126.104.117
Response IP 209.126.104.117
Found Yes
Hash 376bff0f8b4855723644b25ba40386dd26bf59b5eca632e138f60b29e4a38bcf
SimHash 91385a778f53

Groups

amazonbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

semrushbot-sa

Rule Path
Disallow /

cliqzbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

grapeshot

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.unix.com/dbseo_sitemaps_https/dbseo_sitemap_index.xml.gz
sitemap https://www.unix.com/maps/dbmaps/linux_dbman_index.xml.gz
sitemap https://www.unix.com/maps/dbmaps/unix_dbman_index.xml.gz
sitemap https://www.unix.com/maps/maps/misc_sitemap.xml.gz