cncmasters.com
robots.txt

Robots Exclusion Standard data for cncmasters.com

Resource Scan

Scan Details

Site Domain cncmasters.com
Base Domain cncmasters.com
Scan Status Ok
Last Scan2024-06-15T18:54:57+00:00
Next Scan 2024-06-22T18:54:57+00:00

Last Scan

Scanned2024-06-15T18:54:57+00:00
URL https://cncmasters.com/robots.txt
Redirect https://www.cncmasters.com/robots.txt
Redirect Domain www.cncmasters.com
Redirect Base cncmasters.com
Domain IPs 104.26.14.18, 104.26.15.18, 172.67.71.53, 2606:4700:20::681a:e12, 2606:4700:20::681a:f12, 2606:4700:20::ac43:4735
Redirect IPs 104.26.14.18, 104.26.15.18, 172.67.71.53, 2606:4700:20::681a:e12, 2606:4700:20::681a:f12, 2606:4700:20::ac43:4735
Response IP 172.67.71.53
Found Yes
Hash 6eb4aaece68ea4840f1ecee0384fdc36ec163416607a162cff1cbe75aa8b6a98
SimHash c8080adae792

Groups

scrapy

Rule Path
Allow /

*

Rule Path
Allow /wp-content/plugins/*.css$
Allow /wp-content/plugins/*.js$
Allow /wp-includes/*.js$
Disallow /wp-admin/
Disallow /trackback/
Disallow /feed/
Disallow /comments/
Disallow /feed/comments/
Disallow /*?*
Disallow /product-quote

googlebot

Rule Path
Allow *.css$
Allow *.js$