196flavors.com
robots.txt

Robots Exclusion Standard data for 196flavors.com

Resource Scan

Scan Details

Site Domain 196flavors.com
Base Domain 196flavors.com
Scan Status Ok
Last Scan2024-11-15T13:14:15+00:00
Next Scan 2024-11-22T13:14:15+00:00

Last Scan

Scanned2024-11-15T13:14:15+00:00
URL https://196flavors.com/robots.txt
Domain IPs 104.21.74.233, 172.67.164.82, 2606:4700:3030::6815:4ae9, 2606:4700:3031::ac43:a452
Response IP 104.21.74.233
Found Yes
Hash b5df0f54985d9c654b65b23559db4399318209c9ed66115a3a07d2ce878d13a8
SimHash 6870d0824587

Groups

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

googlebot-image

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

googlebot-mobile

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

teoma

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

gigabot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

robozilla

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

nutch

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

ia_archiver

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

baiduspider

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

naverbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

yeti

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

yahoo-mmcrawler

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

psbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

yahoo-blogs/v3.9

Rule Path
Disallow

Other Records

Field Value
crawl-delay 120

*

Rule Path
Disallow /
Disallow /cgi-bin/
Disallow /wp-admin/
Disallow /wp-includes/
Disallow /dev/
Disallow /test/
Disallow /es//comment-subscriptions/
Disallow /fr//comment-subscriptions/
Disallow /comment-subscriptions/

Other Records

Field Value
crawl-delay 120