Log In Sign Up

digiutm.com
robots.txt

Robots Exclusion Standard data for digiutm.com

Resource Scan

Scan Details

Site Domain digiutm.com
Base Domain digiutm.com
Scan Status Ok
Last Scan2024-09-25T09:43:20+00:00
Next Scan 2024-10-02T09:43:20+00:00

Last Scan

Scanned2024-09-25T09:43:20+00:00
URL https://digiutm.com/robots.txt
Domain IPs 192.0.78.158, 192.0.78.253
Response IP 192.0.78.253
Found Yes
Hash 375e06dd4922ec66ab38ae099d6356bc7134ddc4191649378560be88393f91ca
SimHash 4d60d33753b7

Groups

googlebot
googlebot-mobile
adsbot-google
googlebot-image
mediapartners
mediapartners-google
googlebot-news
googlebot-video

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

slurp

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

Other Records

Field Value
crawl-delay 10

bingbot

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

Other Records

Field Value
crawl-delay 10

yandex
yandexbot
yandexmedia
yandeximages
yandexcatalog
yandexdirect
yandexblogs
yandexnews
yandexpagechecker

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

dotbot
rogerbot

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

*

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*

*

Rule Path
Allow /
Disallow /cgi-bin/
Disallow /wp-admin/$
Disallow /author/admin/
Disallow /wp-content/cache*
Disallow */trackback/$
Disallow /comments/feed*
Disallow /wp-login.php?*
Disallow *?comments=
Disallow *?replytocom
Disallow *?s=
Disallow */feed
Disallow */page/
Disallow */tag/
Allow /*.js*
Allow /*.css*
Allow /wp-admin/admin-ajax.php
Allow /wp-admin/admin-ajax.php?action=*
Back to top

Other Records

Field Value
sitemap https://digiutm.com/sitemap.xml
sitemap https://digiutm.com/news-sitemap.xml
Back to top

Comments

  • Google Bots
  • slow down Yahoo
  • Slow Down Bing Bot
  • Yandex Bot
  • Moz Bot
Back to top