hardmob.com.br
robots.txt

Robots Exclusion Standard data for hardmob.com.br

Resource Scan

Scan Details

Site Domain hardmob.com.br
Base Domain hardmob.com.br
Scan Status Ok
Last Scan2024-11-16T22:00:25+00:00
Next Scan 2024-11-23T22:00:25+00:00

Last Scan

Scanned2024-11-16T22:00:25+00:00
URL https://hardmob.com.br/robots.txt
Redirect https://www.hardmob.com.br/robots.txt
Redirect Domain www.hardmob.com.br
Redirect Base hardmob.com.br
Domain IPs 104.26.8.6, 104.26.9.6, 172.67.69.54, 2606:4700:20::681a:806, 2606:4700:20::681a:906, 2606:4700:20::ac43:4536
Redirect IPs 104.26.8.6, 104.26.9.6, 172.67.69.54, 2606:4700:20::681a:806, 2606:4700:20::681a:906, 2606:4700:20::ac43:4536
Response IP 104.26.9.6
Found Yes
Hash 1c37ba7749b99aa3ddb0c9d27a03dd4a29da58a15725d02a810436199c9a6c18
SimHash 2eba9a90c7c7

Groups

mediapartners-google

Rule Path
Disallow

slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

yahoo! slurp

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 600

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 60

ia_archiver

Rule Path
Allow /

*

Rule Path
Disallow /admincp/
Disallow /ajax.php
Disallow /ajaxlivesearch.php
Disallow /ajaxthreads.php
Disallow /arcade/
Disallow /apc.php
Disallow /chat/
Disallow /chathm/
Disallow /avatares.php
Disallow /batepapo.php
Disallow /calendar.php
Disallow /cannedreplies.php
Disallow /customavatars/
Disallow /customgroupicons/
Disallow /customprofilepics/
Disallow /dbtech/vbshout/
Disallow /dbtech/
Disallow /editpost.php
Disallow /external.php
Disallow /faq.php
Disallow /helpfulanswers.php
Disallow /home_servidores_codbo.php
Disallow /ideal_chat_update_stream.php
Disallow /ideal_chat_ajax.php
Disallow /includes/
Disallow /install/
Disallow /iwt/
Disallow /login.php
Disallow /member.php
Disallow /memberlist.php
Disallow /members/
Disallow /media.php
Disallow /media_ajax.php
Disallow /media/
Disallow /micronewsletter/
Disallow /misc/radio/
Disallow /misc.php
Disallow /mobiquo/
Disallow /modcp/
Disallow /moderator.php
Disallow /newreply.php
Disallow /newthread.php
Disallow /online.php
Disallow /photoeditor.php
Disallow /poll.php
Disallow /postings.php
Disallow /post_thanks.php
Disallow /post_groans.php
Disallow /printthread.php
Disallow /private.php
Disallow /profile.php
Disallow /radio/
Disallow /register.php
Disallow /report.php
Disallow /reputation.php
Disallow /xcache2adm/
Disallow /sas/
Disallow /search.php
Disallow /sendmessage.php
Disallow /showgroups.php
Disallow /sphinxapi.php
Disallow /subscription.php
Disallow /tags/
Disallow /teamspeak3-server_linux-amd64/
Disallow /thanks.php
Disallow /threadrate.php
Disallow /tsstatus/
Disallow /usercp.php
Disallow /usernote.php
Disallow /vbnow.php
Disallow /vbnowrss.php
Disallow /vbseocp.php
Disallow /vbseo_checkplugins.php
Disallow /vbshout.php
Disallow /vbstatus.php
Disallow /vbstatus/
Disallow /vbsoccer.php
Disallow /Warn.php
Disallow /sitemap/
Disallow /archive/
Disallow /xptoadmincp/
Disallow /xptomodcp/

Other Records

Field Value
sitemap http://www.hardmob.com.br/sitemap_index.xml.gz

Comments

  • Tame yahoo... it tends to eat a ton of resources without a delay 60
  • Tame yahoo... it tends toeat a ton of resources without a delay
  • Tame bing... it tends to eat a ton of resources without a delay
  • Allow Archive.org to save snapshots of everything
  • All Search Bots
  • Disallow: /*-print/
  • Disallow: /*/?nocache=1
  • Mediapartners-Google
  • User-agent: Mediapartners-Google*
  • Disallow: