tophd.xxx
robots.txt

Robots Exclusion Standard data for tophd.xxx

Resource Scan

Scan Details

Site Domain tophd.xxx
Base Domain tophd.xxx
Scan Status Ok
Last Scan2025-06-20T09:15:42+00:00
Next Scan 2025-07-20T09:15:42+00:00

Last Scan

Scanned2025-06-20T09:15:42+00:00
URL https://tophd.xxx/robots.txt
Domain IPs 104.21.49.31, 172.67.140.131, 2606:4700:3035::6815:311f, 2606:4700:3037::ac43:8c83
Response IP 172.67.140.131
Found Yes
Hash 6a50593f0e5513ad425eece1324a3ce606e7d02aab60b880afe9628a844958bf
SimHash 231fc27063a0

Groups

*

Rule Path
Disallow /*/video/*
Disallow /*/terms
Disallow /*/privacy
Disallow /*/faq
Disallow /*/dmca
Disallow /*/get-listed
Disallow /*/contact
Disallow /*/liked
Disallow /*/watch-later
Disallow /*/clicked
Disallow /*/*long%3Dlong*
Disallow /*/*order%3Ddate*
Disallow /*/pornstar/*/long*
Disallow /*/pornstar/*/order/date*
Disallow /*/category/*/long*
Disallow /*/category/*/order/date*
Disallow /*/search/*/long*
Disallow /*/search/*/order/date*
Disallow /*/js
Disallow /*/css
Disallow /*/page/
Disallow /api/*
Disallow /accessblocked

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

yandexbot

Rule Path
Allow /en
Allow /ru
Allow /tr
Disallow /

Other Records

Field Value
crawl-delay 60

semrushbot

Rule Path
Disallow /

dotbot

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /