Log In Sign Up

gtopala.com
robots.txt

Robots Exclusion Standard data for gtopala.com

Resource Scan

Scan Details

Site Domain gtopala.com
Base Domain gtopala.com
Scan Status Ok
Last Scan4/17/2025, 10:09:52 PM
Next Scan 4/24/2025, 10:09:52 PM

Last Scan

Scanned4/17/2025, 10:09:52 PM
URL https://gtopala.com/robots.txt
Redirect https://www.gtopala.com/robots.txt
Redirect Domain www.gtopala.com
Redirect Base gtopala.com
Domain IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Redirect IPs 104.21.112.1, 104.21.16.1, 104.21.32.1, 104.21.48.1, 104.21.64.1, 104.21.80.1, 104.21.96.1, 2606:4700:3030::6815:1001, 2606:4700:3030::6815:2001, 2606:4700:3030::6815:3001, 2606:4700:3030::6815:4001, 2606:4700:3030::6815:5001, 2606:4700:3030::6815:6001, 2606:4700:3030::6815:7001
Response IP 104.21.80.1
Found Yes
Hash 174da17cc93622afdb6850b08f7910ac238c33f15b95f87ab5fb4e481cb8d711
SimHash 971a5509e6b7

Groups

checkmarknetwork/1.0 (+http://www.checkmarknetwork.com/spider.html)

Rule Path
Disallow /

ubicrawler

Rule Path
Disallow /

doc

Rule Path
Disallow /

zao

Rule Path
Disallow /

sitecheck.internetseer.com

Rule Path
Disallow /

zealbot

Rule Path
Disallow /

msiecrawler

Rule Path
Disallow /

sitesnagger

Rule Path
Disallow /

webstripper

Rule Path
Disallow /

webcopier

Rule Path
Disallow /

fetch

Rule Path
Disallow /

offline explorer

Rule Path
Disallow /

teleport

Rule Path
Disallow /

teleportpro

Rule Path
Disallow /

webzip

Rule Path
Disallow /

linko

Rule Path
Disallow /

httrack

Rule Path
Disallow /

microsoft.url.control

Rule Path
Disallow /

xenu

Rule Path
Disallow /

larbin

Rule Path
Disallow /

libwww

Rule Path
Disallow /

zyborg

Rule Path
Disallow /

download ninja

Rule Path
Disallow /

fast

Rule Path
Disallow /

wget

Rule Path
Disallow /

grub-client

Rule Path
Disallow /

k2spider

Rule Path
Disallow /

npbot

Rule Path
Disallow /

webreaper

Rule Path
Disallow /

*

Rule Path
Disallow /report-error/
Disallow /test/
Disallow /templates/
Disallow /ssl/
Disallow /1/
Disallow /health/
Disallow /health/index.html
Disallow /includes/
Disallow /scripts/
Disallow /scripts/1/
Disallow /scripts/2/
Disallow /scripts/crash_reports/
Disallow /scripts/static/
Disallow /scripts/update.php
Disallow /scripts/crashfix.php
Disallow /siw/changelog_new.php
Disallow /banners/
Disallow /matomo/
Disallow /labels.rdf
Disallow /forum/stats.php
Disallow /forum/stats.php*
Disallow /forum/ratethread.php
Disallow /forum/ratethread.php*
Disallow /forum/calendar-*
Disallow /forum/Calendar-*
Disallow /forum/Calendar-Default-Calendar*
Disallow /forum/misc.php
Disallow /forum/misc.php*
Disallow /forum/private.php
Disallow /forum/private.php*
Disallow /forum/newreply.php
Disallow /forum/newreply.php*
Disallow /forum/attachment.php
Disallow /forum/attachment.php*
Disallow /forum/User-*
Disallow /forum/user-*
Disallow /forum/reputation.php
Disallow /forum/reputation.php*
Disallow /forum/sitemap-users.xml
Disallow /forum/search.php
Disallow /forum/search.php*
Disallow /forum/usercp.php
Disallow /forum/usercp.php*
Disallow /forum/memberlist.php
Disallow /forum/memberlist.php*
Disallow /forum/printthread.php
Disallow /forum/printthread.php*
Disallow /forum/online.php
Disallow /forum/online.php*
Disallow /forum/archive/
Disallow /forum/admin/
Disallow /forum/images/bootbb/
Disallow /forum/showthread.php
Disallow /forum/showthread.php*
Disallow /forum/showteam.php
Disallow /forum/showteam.php*
Disallow /401.php
Disallow /404.php
Disallow /50x.html
Disallow /siw-report/siw-xml-report.xml
Disallow /siw-report/siw-json-report.json
Disallow /siw-report/siw-html-report.html
Disallow /siw-report/siw-csv-report.csv
Disallow /siw-report/siw-txt-report.txt
Disallow /siw-report/siw-short-report.html
Disallow /siw-report/siw-mysql-report.sql
Disallow /siw-report/siw-access-report.mdb
Disallow /DriverAgent
Disallow /BuySiwProLifetime
Disallow /driverfix
Disallow /BuySiwHome
Disallow /BuySiwPro
Disallow /BuySiwTech1
Disallow /BuySiwEnterprise
Disallow /BuySiwTech
Back to top

Other Records

Field Value
sitemap https://www.gtopala.com/sitemap.xml
sitemap https://www.gtopala.com/forum/sitemap-index.xml
Back to top

Comments

  • robots.txt file for https://www.gtopala.com
  • mail admin@gtopala.com for constructive criticism
  • Gabriel Topala
  • Crawlers that are kind enough to obey, but which we'd rather not have
  • unless they're feeding search engines.
  • Some bots are known to be trouble, particularly those designed to copy
  • entire sites. Please obey robots.txt.
  • Misbehaving: requests much too fast:
  • Sorry, wget in its recursive mode is a frequent problem.
  • Please read the man page and use it properly; there is a
  • --wait option you can use to set the delay between hits,
  • for instance.
  • The 'grub' distributed client has been *very* poorly behaved.
  • Doesn't follow robots.txt anyway, but...
  • Hits many times per second, not acceptable
  • http://www.nameprotect.com/botinfo.html
  • A capture bot, downloads gazillions of pages with no public benefit
  • http://www.webreaper.net/
  • User-agent: Yandex
  • Disallow: /test/
Back to top