thisvid.com
robots.txt

Robots Exclusion Standard data for thisvid.com

Resource Scan

Scan Details

Site Domain thisvid.com
Base Domain thisvid.com
Scan Status Ok
Last Scan2024-09-29T21:54:44+00:00
Next Scan 2024-10-29T21:54:44+00:00

Last Scan

Scanned2024-09-29T21:54:44+00:00
URL https://thisvid.com/robots.txt
Domain IPs 88.208.57.223
Response IP 88.208.57.223
Found Yes
Hash a55e298c81d34354714c86f74d65cded14cd6f48493266202d77eea710721a67
SimHash 0c1cc446cb55

Groups

*

Rule Path
Disallow /captcha/*
Disallow /cs/
Disallow /feedback/
Disallow /members/
Disallow /support.php
Disallow /categories-selector/
Disallow /content_load.php
Disallow /contents/avatars/*
Disallow /disclaimers
Disallow /dmca
Disallow /link/*
Disallow /logout/
Disallow /email/
Disallow /login/
Disallow /login-required/
Disallow /signup/
Disallow /terms
Disallow /overload.html
Disallow /redirect_cs.php
Disallow /related_videos_html/*
Disallow /reset-password/
Disallow /resend-confirmation/
Disallow /2257
Disallow /404.php
Disallow /my_album_edit/*
Disallow /my_friends
Disallow /my_conversations/
Disallow /player/embed_null.php
Disallow /player/float.php
Disallow /player/html.php
Disallow /player/timelines.php
Disallow /*?a=
Disallow /*?act=
Disallow /*?action=*$
Disallow /*?as=
Disallow /*?cb=
Disallow /*?charset=
Disallow /*?cv=
Disallow /*?dn=
Disallow /*?do=
Disallow /*?dcwpf0dc4=*$
Disallow /*?encode=
Disallow /*?epl=
Disallow /*?fp
Disallow /*?framerequest
Disallow /*?h=
Disallow /*?hv
Disallow /*?i=
Disallow /*?i5dl
Disallow /*?loc=
Disallow /*mode%3Dasync*$
Disallow /*function%3Dget_block*$
Disallow /*?nats
Disallow /*?p_
Disallow /*?P_AVPASS=
Disallow /*?page
Disallow /*?pid
Disallow /*?pv=
Disallow /*?r=
Disallow /*?ref
Disallow /*?refsite
Disallow /*?rnd=*$
Disallow /*?s=
Disallow /*?sa=
Disallow /*?safesearch=*$
Disallow /*?tcc=*$
Disallow /*?toper
Disallow /*?ttc=*$
Disallow /*?user
Disallow /*?User-Agent=
Disallow /*?view=
Disallow /*?visit
Disallow /*?_t
Disallow /*?wvdpforce
Disallow /*?zoneid
Disallow /*wp-login
Disallow /*piwik
Disallow /*x%3D
Disallow /*y%3D

yandex

Rule Path
Disallow /tags/*/popular-males/
Disallow /tags/*/popular-females/

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

semrushbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 10

ia_archiver
ia_archiver-web.archive.org
mj12bot
ahrefsbot
seekport crawler

Rule Path
Disallow /