dsmtuners.com
robots.txt

Robots Exclusion Standard data for dsmtuners.com

Resource Scan

Scan Details

Site Domain dsmtuners.com
Base Domain dsmtuners.com
Scan Status Ok
Last Scan2024-11-14T16:13:04+00:00
Next Scan 2024-11-21T16:13:04+00:00

Last Scan

Scanned2024-11-14T16:13:04+00:00
URL https://dsmtuners.com/robots.txt
Redirect https://www.dsmtuners.com/robots.txt
Redirect Domain www.dsmtuners.com
Redirect Base dsmtuners.com
Domain IPs 104.26.0.219, 104.26.1.219, 172.67.71.194, 2606:4700:20::681a:1db, 2606:4700:20::681a:db, 2606:4700:20::ac43:47c2
Redirect IPs 104.26.0.219, 104.26.1.219, 172.67.71.194, 2606:4700:20::681a:1db, 2606:4700:20::681a:db, 2606:4700:20::ac43:47c2
Response IP 104.26.0.219
Found Yes
Hash e624d0442f3fc6327e5df2800f3ec188d4eec4e4c5826111880d6a2bf3e10100
SimHash 4e62d4544733

Groups

mediapartners-google

Rule Path
Disallow

*

Rule Path
Disallow /account/
Disallow /goto/
Disallow /go-to/
Disallow /login/
Disallow /register/
Disallow /lost-password/
Disallow /members/
Disallow /online/
Disallow /misc/
Disallow /admin.php
Disallow /deferred.php
Disallow /threads/*?view=reaction_score$
Disallow */unread$
Disallow */latest$
Allow /search/$
Disallow /search/*
Allow */threads/post-*
Disallow */post-*
Allow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

boardreader
boardtracker
gigabot
twiceler
dotbot
baidu
baiduspider
baiduspider-video
baiduspider-image
naverbot
sosospider
yandex
youdaobot
yeti
magpie-crawler
magpie-crawler/1.1
magpie-crawler/1
genieo/x.x
spinn3r
spinn3r 3.1
feedwordpress/2012.1218
feedwordpress
spinn3r (spinn3r 3.1)

Rule Path
Disallow /

Other Records

Field Value
sitemap https://www.dsmtuners.com/sitemap.php