cl.top10place.com
robots.txt

Robots Exclusion Standard data for cl.top10place.com

Resource Scan

Scan Details

Site Domain cl.top10place.com
Base Domain top10place.com
Scan Status Failed
Failure StageFetching resource.
Failure ReasonServer returned a client error.
Last Scan2025-09-05T01:35:41+00:00
Next Scan 2025-12-04T01:35:41+00:00

Last Successful Scan

Scanned2024-07-20T01:26:53+00:00
URL https://cl.top10place.com/robots.txt
Domain IPs 104.26.14.216, 104.26.15.216, 172.67.75.71, 2606:4700:20::681a:ed8, 2606:4700:20::681a:fd8, 2606:4700:20::ac43:4b47
Response IP 104.26.14.216
Found Yes
Hash eb827fb0c9555c5dec0e41e100411f5a7c9d01143d52f212003dfbb1fd62cae7
SimHash 6e1dd8c346b3

Groups

*

Rule Path
Disallow /cache/
Disallow /members/
Disallow /cron.php
Disallow /ajax.php
Disallow /maintenance.php
Disallow /error.html
Disallow /*/send-message.html
Disallow /*/send-message-friend.html
Disallow /*/add-review.html
Disallow /*/suggestion.html
Disallow /*/claim.html

mediapartners-google

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googlebot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

adsbot-google

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googlebot-image

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

googlebot-mobile

Rule Path
Disallow

Other Records

Field Value
crawl-delay 1

googleother

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

msnbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 20

bingbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 5

slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 360

yahoo! slurp

Rule Path
Disallow

Other Records

Field Value
crawl-delay 360

yandexbot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 8

yandex

Rule Path
Disallow

Other Records

Field Value
crawl-delay 8

wasalive-bot

Rule Path
Disallow

Other Records

Field Value
crawl-delay 180

kulturarw3

Rule Path
Disallow

Other Records

Field Value
crawl-delay 180

facebookexternalhit

Rule Path
Disallow

Other Records

Field Value
crawl-delay 8

ahrefsbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

mj12bot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

blexbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

proximic

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

seokicks-robot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

admantx

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

dotbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

wiederfreibot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

cliqzbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

grapeshotcrawler

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

seznambot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 360

coccocbot

Rule Path
Disallow /

Other Records

Field Value
crawl-delay 10