c21media.net
robots.txt

Robots Exclusion Standard data for c21media.net

Resource Scan

Scan Details

Site Domain c21media.net
Base Domain c21media.net
Scan Status Ok
Last Scan2024-11-18T15:41:58+00:00
Next Scan 2024-11-25T15:41:58+00:00

Last Scan

Scanned2024-11-18T15:41:58+00:00
URL https://c21media.net/robots.txt
Domain IPs 104.20.216.20, 104.20.217.20, 172.67.0.166, 2606:4700:10::6814:d814, 2606:4700:10::6814:d914, 2606:4700:10::ac43:a6
Response IP 172.67.0.166
Found Yes
Hash ee500e0d428922ab32e1768de009c92ca238c4b121b698512518fa6484494cc7
SimHash 4800d112c535

Groups

*

Rule Path
Allow /wp-content/uploads/
Disallow /wp-content/plugins/
Allow /wp-content/plugins/buddypress/bp-themes/bp-c21/_inc/
Disallow /conference/content-london-2018/

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 18

yahoo! slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 19

twitterbot

Rule Path
Disallow