tweaktown.com
robots.txt

Robots Exclusion Standard data for tweaktown.com

Resource Scan

Scan Details

Site Domain tweaktown.com
Base Domain tweaktown.com
Scan Status Ok
Last Scan2024-11-03T17:17:30+00:00
Next Scan 2024-11-10T17:17:30+00:00

Last Scan

Scanned2024-11-03T17:17:30+00:00
URL https://tweaktown.com/robots.txt
Redirect https://www.tweaktown.com/robots.txt
Redirect Domain www.tweaktown.com
Redirect Base tweaktown.com
Domain IPs 104.22.36.139, 104.22.37.139, 172.67.12.176, 2606:4700:10::6816:248b, 2606:4700:10::6816:258b, 2606:4700:10::ac43:cb0
Redirect IPs 104.22.36.139, 104.22.37.139, 172.67.12.176, 2606:4700:10::6816:248b, 2606:4700:10::6816:258b, 2606:4700:10::ac43:cb0
Response IP 172.67.12.176
Found Yes
Hash b0b23c20317934bcd181daef943a50b2ad7a87e7a1c3a7bd01609158148934bf
SimHash 07e5d454c714

Groups

*

Rule Path
Disallow /admin
Disallow /cse
Disallow */preview/
Disallow /dada_mail_support_files
Disallow /comments

mediapartners-google

Rule Path
Disallow

adsbot-google-mobile

Rule Path
Disallow

mediapartners(googlebot)

Rule Path
Disallow

adsbot-google

Rule Path
Disallow

facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)

Rule Path
Disallow

facebookexternalhit/1.1

Rule Path
Disallow

twitterbot

Rule Path
Disallow

adidxbot

Rule Path
Disallow

Other Records

Field Value
sitemap https://www.tweaktown.com/sitemap_index.xml
sitemap https://www.tweaktown.com/feeds/google_news.xml
sitemap https://www.tweaktown.com/feeds/google_articles.xml
sitemap https://www.tweaktown.com/feeds/google_reviews.xml
sitemap https://www.tweaktown.com/feeds/google_guides.xml