surfing-waves.com
robots.txt

Robots Exclusion Standard data for surfing-waves.com

Resource Scan

Scan Details

Site Domain surfing-waves.com
Base Domain surfing-waves.com
Scan Status Ok
Last Scan2024-11-13T12:06:46+00:00
Next Scan 2024-11-20T12:06:46+00:00

Last Scan

Scanned2024-11-13T12:06:46+00:00
URL https://surfing-waves.com/robots.txt
Domain IPs 142.93.205.93
Response IP 142.93.205.93
Found Yes
Hash 9c0485ac6ad606e1029ad5104fc119997c7dcecfba551c49f6f6434131e750a1
SimHash ae58b8c2c9a2

Groups

ahrefsbot

Rule Path
Disallow /

alphaseobot

Rule Path
Disallow /

boardreader

Rule Path
Disallow /

ccbot

Rule Path
Disallow /

ltx71

Rule Path
Disallow /

psbot

Rule Path
Disallow /

shopwiki

Rule Path
Disallow /

twiceler

Rule Path
Disallow /

nutch

Rule Path
Disallow /

linksmanager_bot

Rule Path
Disallow /

linksmanager

Rule Path
Disallow /

magpie-crawler

Rule Path
Disallow /

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 20

spbot

Rule Path
Disallow /

snapbot/1.0

Rule Path
Disallow /

speedy

Rule Path
Disallow /

yandex

Rule Path
Disallow /

mj12bot

Rule Path
Disallow /

blexbot

Rule Path
Disallow /

trendictionbot

Rule Path
Disallow /

user-agent: xovibot

Rule Path
Disallow /

*

Rule Path
Disallow /cgi-bin/
Disallow /Dept/noparent/
Disallow /shop/pic.php
Disallow /search.htm
Disallow /forum/*%26start%3D0
Disallow /forum/*%26view%3Dprevious
Disallow /forum/*%26view%3Dnext
Disallow /forum/faq.php
Disallow /forum/memberlist.php
Disallow /forum/posting.php
Disallow /forum/profile.php
Disallow /forum/search.php
Disallow /forum/ucp.php
Disallow /forum/viewonline.php
Disallow /min/
Disallow /watch/
Disallow /ajax.get.php

googlebot

Rule Path
Disallow /cgi-bin/
Disallow /Dept/noparent/
Disallow /shop/pic.php
Disallow /search.htm
Disallow /forum/*%26start%3D0
Disallow /forum/*%26view%3Dprevious
Disallow /forum/*%26view%3Dnext
Disallow /forum/faq.php
Disallow /forum/memberlist.php
Disallow /forum/posting.php
Disallow /forum/profile.php
Disallow /forum/search.php
Disallow /forum/ucp.php
Disallow /forum/viewonline.php
Disallow /min/
Disallow /3887137/
Disallow /forum/viewtopic.php%3F*
Disallow /forum/viewforum.php%3F*
Disallow /watch/

adsbot-google

Rule Path
Disallow /cgi-bin/
Disallow /Dept/noparent/
Disallow /shop/pic.php
Disallow /search.htm
Disallow /forum/*%26start%3D0
Disallow /forum/*%26view%3Dprevious
Disallow /forum/*%26view%3Dnext
Disallow /forum/faq.php
Disallow /forum/memberlist.php
Disallow /forum/posting.php
Disallow /forum/profile.php
Disallow /forum/search.php
Disallow /forum/ucp.php
Disallow /forum/viewonline.php
Disallow /min/
Disallow /3887137/
Disallow /forum/viewtopic.php%3F*
Disallow /forum/viewforum.php%3F*
Disallow /watch/

Comments

  • Disallow: /Item/

Warnings

  • 2 invalid lines.