Log In Sign Up

adultdvdtalk.com
robots.txt

Robots Exclusion Standard data for adultdvdtalk.com

Resource Scan

Scan Details

Site Domain adultdvdtalk.com
Base Domain adultdvdtalk.com
Scan Status Ok
Last Scan2024-11-11T21:09:58+00:00
Next Scan 2024-12-11T21:09:58+00:00

Last Scan

Scanned2024-11-11T21:09:58+00:00
URL https://adultdvdtalk.com/robots.txt
Redirect https://www.adultdvdtalk.com/robots.txt
Redirect Domain www.adultdvdtalk.com
Redirect Base adultdvdtalk.com
Domain IPs 104.22.62.96, 104.22.63.96, 172.67.5.2, 2606:4700:10::6816:3e60, 2606:4700:10::6816:3f60, 2606:4700:10::ac43:502
Redirect IPs 104.22.62.96, 104.22.63.96, 172.67.5.2, 2606:4700:10::6816:3e60, 2606:4700:10::6816:3f60, 2606:4700:10::ac43:502
Response IP 172.67.5.2
Found Yes
Hash 2aec046799acba4d610f88d546418261da9f2d8c2f11f21ab5c6c298c0b1ad5c
SimHash 700fd444fe91

Groups

ia_archiver

Rule Path
Disallow /

ia_archiver-web.archive.org

Rule Path
Disallow /

archive.org_bot

Rule Path
Disallow /

ahrefsbot

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

baiduspider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

dataforseobot

Rule Path
Disallow /

semrushbot

Rule Path
Disallow /

googlebot

Rule Path
Disallow /tweet_this*
Disallow /*click*
Disallow /privacy
Disallow /terms
Disallow /login
Disallow /register
Disallow /advanced_search*
Disallow /*?*
Disallow /pornstar/*/trailers*
Disallow /pornstar/*/cat-*
Disallow /pornstar/*/s-*
Disallow /pornstar/*/galleries*
Disallow /pornstar/*/movies-*
Disallow /pornstar/*/sort-*
Disallow /reviewer/*/trailers*
Disallow /reviewer/*/cat-*
Disallow /reviewer/*/s-*
Disallow /reviewer/*/galleries*
Disallow /reviewer/*/movies-*
Disallow /reviewer/*/sort-*
Disallow /reviews/*/trailers*
Disallow /reviews/*/cat-*
Disallow /reviews/*/s-*
Disallow /reviews/*/galleries*
Disallow /reviews/*/movies-*
Disallow /reviews/*/sort-*
Disallow /director/*/trailers*
Disallow /director/*/cat-*
Disallow /director/*/s-*
Disallow /director/*/galleries*
Disallow /director/*/movies-*
Disallow /director/*/sort-*
Disallow /porn/*/trailers*
Disallow /porn/*/cat-*
Disallow /porn/*/s-*
Disallow /porn/*/galleries*
Disallow /porn/*/movies-*
Disallow /porn/*/sort-*
Disallow /porno/*/trailers*
Disallow /porno/*/cat-*
Disallow /porno/*/s-*
Disallow /porno/*/galleries*
Disallow /porno/*/movies-*
Disallow /porno/*/sort-*

slurp

Rule Path
Disallow /tweet_this*
Disallow /*click*
Disallow /privacy
Disallow /terms
Disallow /login
Disallow /register
Disallow /advanced_search*
Disallow /*?*
Disallow /pornstar/*/trailers*
Disallow /pornstar/*/cat-*
Disallow /pornstar/*/s-*
Disallow /pornstar/*/galleries*
Disallow /pornstar/*/movies-*
Disallow /pornstar/*/sort-*
Disallow /reviewer/*/trailers*
Disallow /reviewer/*/cat-*
Disallow /reviewer/*/s-*
Disallow /reviewer/*/galleries*
Disallow /reviewer/*/movies-*
Disallow /reviewer/*/sort-*
Disallow /reviews/*/trailers*
Disallow /reviews/*/cat-*
Disallow /reviews/*/s-*
Disallow /reviews/*/galleries*
Disallow /reviews/*/movies-*
Disallow /reviews/*/sort-*
Disallow /director/*/trailers*
Disallow /director/*/cat-*
Disallow /director/*/s-*
Disallow /director/*/galleries*
Disallow /director/*/movies-*
Disallow /director/*/sort-*
Disallow /porn/*/trailers*
Disallow /porn/*/cat-*
Disallow /porn/*/s-*
Disallow /porn/*/galleries*
Disallow /porn/*/movies-*
Disallow /porn/*/sort-*
Disallow /porno/*/trailers*
Disallow /porno/*/cat-*
Disallow /porno/*/s-*
Disallow /porno/*/galleries*
Disallow /porno/*/movies-*
Disallow /porno/*/sort-*

msnbot

Rule Path
Disallow /tweet_this*
Disallow /*click*
Disallow /privacy
Disallow /terms
Disallow /login
Disallow /register
Disallow /advanced_search*
Disallow /*?*
Disallow /pornstar/*/trailers*
Disallow /pornstar/*/cat-*
Disallow /pornstar/*/s-*
Disallow /pornstar/*/galleries*
Disallow /pornstar/*/movies-*
Disallow /pornstar/*/sort-*
Disallow /reviewer/*/trailers*
Disallow /reviewer/*/cat-*
Disallow /reviewer/*/s-*
Disallow /reviewer/*/galleries*
Disallow /reviewer/*/movies-*
Disallow /reviewer/*/sort-*
Disallow /reviews/*/trailers*
Disallow /reviews/*/cat-*
Disallow /reviews/*/s-*
Disallow /reviews/*/galleries*
Disallow /reviews/*/movies-*
Disallow /reviews/*/sort-*
Disallow /director/*/trailers*
Disallow /director/*/cat-*
Disallow /director/*/s-*
Disallow /director/*/galleries*
Disallow /director/*/movies-*
Disallow /director/*/sort-*
Disallow /porn/*/trailers*
Disallow /porn/*/cat-*
Disallow /porn/*/s-*
Disallow /porn/*/galleries*
Disallow /porn/*/movies-*
Disallow /porn/*/sort-*
Disallow /porno/*/trailers*
Disallow /porno/*/cat-*
Disallow /porno/*/s-*
Disallow /porno/*/galleries*
Disallow /porno/*/movies-*
Disallow /porno/*/sort-*

bingbot

Rule Path
Disallow /tweet_this*
Disallow /*click*
Disallow /privacy
Disallow /terms
Disallow /login
Disallow /register
Disallow /advanced_search*
Disallow /*?*

*

Rule Path
Disallow /tweet_this*
Disallow /*click*
Disallow /privacy
Disallow /terms
Disallow /login
Disallow /register
Disallow /advanced_search*
Disallow /*?*
Disallow /pornstar/*/trailers*
Disallow /pornstar/*/cat-*
Disallow /pornstar/*/s-*
Disallow /pornstar/*/galleries*
Disallow /pornstar/*/movies-*
Disallow /pornstar/*/sort-*
Disallow /reviewer/*/trailers*
Disallow /reviewer/*/cat-*
Disallow /reviewer/*/s-*
Disallow /reviewer/*/galleries*
Disallow /reviewer/*/movies-*
Disallow /reviewer/*/sort-*
Disallow /reviews/*/trailers*
Disallow /reviews/*/cat-*
Disallow /reviews/*/s-*
Disallow /reviews/*/galleries*
Disallow /reviews/*/movies-*
Disallow /reviews/*/sort-*
Disallow /director/*/trailers*
Disallow /director/*/cat-*
Disallow /director/*/s-*
Disallow /director/*/galleries*
Disallow /director/*/movies-*
Disallow /director/*/sort-*
Disallow /porn/*/trailers*
Disallow /porn/*/cat-*
Disallow /porn/*/s-*
Disallow /porn/*/galleries*
Disallow /porn/*/movies-*
Disallow /porn/*/sort-*
Disallow /porno/*/trailers*
Disallow /porno/*/cat-*
Disallow /porno/*/s-*
Disallow /porno/*/galleries*
Disallow /porno/*/movies-*
Disallow /porno/*/sort-*

fasterfox

Rule Path
Disallow /

webaroobot

Rule Path
Disallow /

wget

Rule Path
Disallow /
Back to top