99div.com
robots.txt

Robots Exclusion Standard data for 99div.com

Resource Scan

Scan Details

Site Domain 99div.com
Base Domain 99div.com
Scan Status Ok
Last Scan2025-12-04T23:59:10+00:00
Next Scan 2025-12-11T23:59:10+00:00

Last Scan

Scanned2025-12-04T23:59:10+00:00
URL https://99div.com/robots.txt
Domain IPs 66.185.26.90
Response IP 66.185.26.90
Found Yes
Hash 26f894ce44c5d628a1ce6c3d9b05ca9a8253fa5098e1cd0c0b7db61735ed7b6a
SimHash 181f9077fed8

Groups

googlebot

Rule Path
Allow /
Allow /cgi-bin/
Allow /images/
Allow /css/

bingbot

Rule Path
Allow /
Allow /cgi-bin/
Allow /images/
Allow /css/

ccbot

Rule Path
Allow /
Allow /cgi-bin/
Allow /images/
Allow /css/

awariosmartbot

Rule Path
Disallow /

ahrefs

Rule Path
Disallow /

colly

Rule Path
Disallow /

clickagy intelligence bot

Rule Path
Disallow /

criteobot

Rule Path
Disallow /

daum

Rule Path
Disallow /

domainappender

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

grapeshotcrawler

Rule Path
Disallow /

go-http-client

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

newswhipbox

Rule Path
Disallow /

obot

Rule Path
Disallow /

pagesinventory

Rule Path
Disallow /

seekportbot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

turnitin

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

ttd-content

Rule Path
Disallow /

twitterbot

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

zoominfobot

Rule Path
Disallow /

*

Rule Path
Disallow /0/
Disallow /_private/
Disallow /_vti_bin/
Disallow /_vti_cnf/
Disallow /_vti_log/
Disallow /_vti_txt/
Disallow /ad/
Disallow /cgi/
Disallow /cgi-bin/
Disallow /checkerboard/
Disallow /ftp/
Allow /images/
Allow /css/
Disallow /scgi-bin/
Disallow /subscribe/
Disallow /suspended.page/
Disallow /w3c/
Disallow /weather/

Other Records

Field Value
crawl-delay 2

Other Records

Field Value
sitemap http://marionrecord.com/rss.cgi
sitemap http://marionrecord.com/sitemap.cgi
sitemap http://marionrecord.com/oldsitemap.cgi
sitemap http://starj.com/rss.cgi
sitemap http://starj.com/sitemap.cgi
sitemap http://starj.com/oldsitemap.cgi
sitemap http://peabodykansas.com/rss.cgi
sitemap http://peabodykansas.com/sitemap.cgi
sitemap http://peabodykansas.com/oldsitemap.cgi
sitemap http://99div.com/rss.cgi
sitemap http://99div.com/sitemap.cgi
sitemap http://99div.com/oldsitemap.cgi