webn-chu.com
robots.txt

Robots Exclusion Standard data for webn-chu.com

Resource Scan

Scan Details

Site Domain webn-chu.com
Base Domain webn-chu.com
Scan Status Ok
Last Scan2024-11-10T00:13:24+00:00
Next Scan 2024-11-17T00:13:24+00:00

Last Scan

Scanned2024-11-10T00:13:24+00:00
URL https://webn-chu.com/robots.txt
Domain IPs 104.21.9.139, 172.67.160.130, 2606:4700:3032::ac43:a082, 2606:4700:3033::6815:98b
Response IP 172.67.160.130
Found Yes
Hash 10def7da12a276e706a24b22d75e36daed52381f5a210c92ca85656616c5710d
SimHash 4b27ff822aca

Groups

*

Rule Path
Disallow /humans.txt

googlebot-image

Rule Path
Disallow /

yahoo-mmcrawler

Rule Path
Disallow /

bingbot

Rule Path
Disallow /

msnbot-media

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

obot

Rule Path
Disallow /

mail.ru_bot

Rule Path
Disallow /

turnitinbot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

baiduspider+

Rule Path
Disallow /

baiduimagespider

Rule Path
Disallow /

baiduimagespider+

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /
Disallow /?s=
Disallow /tag/
Disallow /page/
Disallow /author/
Disallow /wp-admin/
Disallow /wp-content/
Disallow /wp-login.php
Disallow /category/
Disallow /*category
Disallow /category/*/*
Disallow /wp-includes/
Disallow /wp-content/cache
Disallow /wp-content/themes
Disallow /wp-content/plugins
Disallow /feed
Disallow /feed/$
Disallow */feed/
Disallow /*/feed/$
Disallow /*/feed/rss/$
Disallow /*/*/feed/$
Disallow /*/*/feed/rss/$
Disallow /*/*/*/feed/$
Disallow /*/*/*/feed/rss/$
Disallow /cgi-bin
Disallow /search/
Disallow /comments
Disallow /comments/feed
Disallow /trackback/
Disallow /*/trackback/$
Disallow /*/*/trackback/$
Disallow /*/*/*/trackback/$
Disallow /archives/
Disallow /*?
Disallow /*?*
Disallow /*.gz$
Disallow /*.swf$
Disallow /*.wmv$
Disallow /*.cgi$
Disallow /*.php$
Disallow /*.inc$
Disallow /*.jpg$
Disallow /*.gif$
Disallow */feed/
Disallow /*.xhtml$
Disallow */comments
Disallow */trackback/
Disallow /*?utm_source
Disallow /*?utm_medium
Disallow /*?utm_campaign
Disallow /*?utm_term
Disallow /*?utm_content
Disallow /*?utm_nooverride
Disallow /*%26utm_source
Disallow /*%26utm_medium
Disallow /*%26utm_campaign
Disallow /*%26utm_term
Disallow /*%26utm_content
Disallow /*%26utm_nooverride
Allow /feed/$
Allow /wp-content/uploads/
Allow /wp-admin/admin-ajax.php
Allow *****************/*.js$
Allow *****************/*.css$

bingbot-image

Rule Path
Disallow /

msnbot-image

Rule Path
Disallow /

mediapartners-google*

Rule Path
Disallow
Allow /*

ia_archiver

Rule Path
Disallow /

duggmirror

Rule Path
Disallow /

Other Records

Field Value
sitemap https://webn-chu.com/sitemap_index.xml

Comments

  • Search engine robot disallow
  • Disallow Bing image bot to search all images
  • Allow Google Adsense bot on entire site
  • Internet Archive Denial
  • digg mirror