m.roleplayer.me
robots.txt

Robots Exclusion Standard data for m.roleplayer.me

Resource Scan

Scan Details

Site Domain m.roleplayer.me
Base Domain roleplayer.me
Scan Status Ok
Last Scan2024-07-06T07:32:10+00:00
Next Scan 2024-07-13T07:32:10+00:00

Last Scan

Scanned2024-07-06T07:32:10+00:00
URL https://m.roleplayer.me/robots.txt
Domain IPs 104.26.10.25, 104.26.11.25, 172.67.73.228, 2606:4700:20::681a:a19, 2606:4700:20::681a:b19, 2606:4700:20::ac43:49e4
Response IP 104.26.11.25
Found Yes
Hash 46127dd7f2094fb85351b04272b25b8828f34334e1319ef9c8fe080b62809040
SimHash 4a5fe66696ba

Groups

mediapartners-google

Rule Path
Disallow

googlebot-image

Rule Path
Disallow /

genieo

Rule Path
Disallow /

jikespider

Rule Path
Disallow /

yandex

Rule Path
Disallow /

yandexbot

Rule Path
Disallow /

yandeximages

Rule Path
Disallow /

baiduspider baiduspider-video baiduspider-image baiduspider-video baiduspider-news baiduspider-favo baiduspider-cpro baiduspider-ads

Rule Path
Disallow /

naverbot

Rule Path
Disallow /

yeti

Rule Path
Disallow /

spinn3r

Rule Path
Disallow /

sogou

Rule Path
Disallow /

sogou spider

Rule Path
Disallow /

ia_archiver

Rule Path
Disallow

ahrefsbot

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

discobot

Rule Path
Disallow /

uptimebot

Rule Path
Disallow /

seznambot

Rule Path
Disallow /

ezooms

Rule Path
Disallow /

acoon

Rule Path
Disallow /

mlbot

Rule Path
Disallow /

sosospider

Rule Path
Disallow /

*
*

Rule Path
Disallow /add_bookmark.php
Disallow /add_comment.php
Disallow /add_group.php
Disallow /add_testimonial.php
Disallow /advertise.php
Disallow /albums.php
Disallow /approve_friend.php
Disallow /approve_test.php
Disallow /blocked.php
Disallow /block_user.php
Disallow /blognotifications.php
Disallow /bulletinnotifications.php
Disallow /commentnotifications.php
Disallow /delete_test.php
Disallow /edit_comments.php
Disallow /flag.php
Disallow /forgot_password.php
Disallow /forward_friend.php
Disallow /friendnotifications.php
Disallow /gallery.php
Disallow /gallery/
Disallow /groupnotifications.php
Disallow /group_images/
Disallow /home.php
Disallow /images/
Disallow /invite_friend.php
Disallow /join_me.php
Disallow /login.php
Disallow /m/
Disallow /m/images/
Disallow /m/old_dirs/
Disallow /m/old_files/
Disallow /m/photos/
Disallow /m/rss_blog.php
Disallow /m/songs/
Disallow /m/xspf/
Disallow /make_default.php
Disallow /mobile.php
Disallow /new_friends.php
Disallow /notice.php
Disallow /old_dirs/
Disallow /old_files/
Disallow /online_friends.php
Disallow /photos/
Disallow /picturenotifications.php
Disallow /profile_setting.php
Disallow /rank_band.php
Disallow /reply.php
Disallow /rss_blog.php
Disallow /search_profiles.php
Disallow /send_message.php
Disallow /send_message1.php
Disallow /shared_layouts.php
Disallow /songs/
Disallow /statusnotifications.php
Disallow /status_stream.php
Disallow /subscribe_blog.php
Disallow /user_images/
Disallow /view_blogs_category.php
Disallow /view_blog_layout.php
Disallow /view_comments.php
Disallow /view_friends.php
Disallow /view_group.php
Disallow /view_layout.php
Disallow /view_message.php
Disallow /view_messages.php
Disallow /view_photo.php
Disallow /view_post.php
Disallow /view_testimonials.php
Disallow /xspf/

Other Records

Field Value
crawl-delay 10

Comments

  • http://riskyinternet.com/what-is/web-robot/jikespider-6996/
  • http://riskyinternet.com/what-is/web-robot/yandex-media-6835/
  • http://riskyinternet.com/what-is/web-robot/yandex-std-crawler-6821/
  • http://riskyinternet.com/what-is/web-robot/yandex-images-6828/
  • http://riskyinternet.com/what-is/web-robot/baiduspider-5925/
  • http://riskyinternet.com/what-is/web-robot/naverbot-crawler-6443/
  • http://help.naver.com/robots
  • http://riskyinternet.com/what-is/web-robot/spinn3rspinner-aka-tailrank-inc-7073/
  • http://spinn3r.com/robot
  • http://riskyinternet.com/what-is/web-robot/new-sogou-spider-6562/
  • http://riskyinternet.com/what-is/web-robot/internet-archiver-lift-all-internet-content-over-and-over-3517/
  • http://archive.org/abot/exclude.php
  • http://www.alexa.com/help/webmasters
  • http://riskyinternet.com/what-is/web-robot/ahrefscom-bot-6002/
  • https://ahrefs.com/robot/
  • http://riskyinternet.com/what-is/web-robot/mj12-bot-4994/
  • http://www.majestic12.co.uk/projects/dsearch/mj12bot.php
  • http://riskyinternet.com/what-is/web-robot/discoveryengine-robot-6142/
  • http://discoveryengine.com/discobot.html
  • http://uptimerobot.com
  • http://fulltext.sblog.cz/category/robot
  • http://riskyinternet.com/what-is/web-robot/dotnetdotcom-3573/
  • http://www.acoon.de/robot.asp
  • http://riskyinternet.com/what-is/web-robot/mlbot-3559/
  • http://www.metadatalabs.com/mlbot