Log In Sign Up

namepros.com
robots.txt

Robots Exclusion Standard data for namepros.com

Resource Scan

Scan Details

Site Domain namepros.com
Base Domain namepros.com
Scan Status Ok
Last Scan2024-11-10T14:38:56+00:00
Next Scan 2024-11-24T14:38:56+00:00

Last Scan

Scanned2024-11-10T14:38:56+00:00
URL https://namepros.com/robots.txt
Redirect https://www.namepros.com/robots.txt
Redirect Domain www.namepros.com
Redirect Base namepros.com
Domain IPs 104.26.10.237, 104.26.11.237, 172.67.72.183, 2606:4700:20::681a:aed, 2606:4700:20::681a:bed, 2606:4700:20::ac43:48b7
Redirect IPs 104.26.10.237, 104.26.11.237, 172.67.72.183, 2606:4700:20::681a:aed, 2606:4700:20::681a:bed, 2606:4700:20::ac43:48b7
Response IP 172.67.72.183
Found Yes
Hash 5e32f65cc0944236384969e2c92db73e2560950ea37cdf4f2810bf9791626162
SimHash 2c802550db05

Groups

*

Rule Path
Disallow /account/
Disallow /blog/*/metrics$
Disallow /blog/*/reply$
Disallow /blog/*/reply?
Disallow /blog/*/unread$
Disallow /blog/*/watch$
Disallow /blog/create-thread
Disallow /blog/post-thread
Disallow /chat/
Disallow /conversations/
Disallow /credits/
Disallow /forums/*/create-thread
Disallow /forums/*/post-thread
Disallow /forums/*/watch$
Disallow /parking/*
Allow /parking/$
Disallow /points/*/add$
Disallow /points/*/remove$
Disallow /pq/points/*/*/add$
Disallow /pq/points/*/*/remove$
Disallow /pq/quick-replies/*/*/mode?
Disallow /pq/quick-replies/*/*/reply?
Disallow /quick-reply/*/reply?
Disallow /quick-reply/*/toggle-mode?
Disallow /tags/*/watch$
Disallow /threads/*/metrics$
Disallow /threads/*/reply$
Disallow /threads/*/reply?
Disallow /threads/*/unread$
Disallow /threads/*/watch$
Disallow /login/login
Disallow /register/register
Disallow /members/list/
Disallow /members/list$
Disallow /members/list?
Disallow /members/*/followers
Disallow /members/*/following
Disallow /members/*/post-areas
Disallow /members/*/recent-activity
Disallow /members/*/recent-content
Disallow /members/*/trophies
Disallow /search/1*/
Disallow /search/2*/
Disallow /search/3*/
Disallow /search/4*/
Disallow /search/5*/
Disallow /search/6*/
Disallow /search/7*/
Disallow /search/8*/
Disallow /search/9*/
Disallow /search/member$
Disallow /search/member?
Disallow /bclk/
Disallow /bevt/
Disallow /admin.php
Disallow /internal/

adsbot-google
mediapartners-google

Rule Path
Disallow /bclk/
Disallow /bevt/

ahrefsbot
linguee

Rule Path
Disallow /account/
Disallow /conversations/
Disallow /credits/
Disallow /posts/
Disallow /find-new/
Disallow /search/
Disallow /tradereviews/
Disallow /members/
Disallow /misc/
Disallow /goto/
Disallow /data/
Disallow /thanks/
Disallow /bclk/
Disallow /bevt/
Back to top

Comments

  • vim: ft=robots
  • Mediapartners-Google and AdsBot-Google ignore wildcard groups.
  • Inaccessible by crawlers because of authentication requirements
  • Require login
  • Require POST
  • Require login often enough to cause problems
  • Transient pages
  • Only intended for real, authentic, genuine homo sapiens
  • Internal stuff.
  • If you're a hacker, go straight to /internal/ because it's obviously the most vulnerable.
  • That's where we hide our ion cannon. (We have cake!)
  • Bots that ignore *
  • Less-than-intelligent bots that can't properly parse constructs such as rel="nofollow", X-Robots-Tag, or base tags.
  • They also tend not to deduplicate URLs (so if a link to /misc/style appears on every page, they'll crawl it once for each page).
  • If your crawler appears here, it's probably snowballing, resulting in a ridiculous number of requests.
  • THE CAKE IS A LIE
  • THE CAKE IS A LIE
  • THE CAKE IS^D
Back to top