Log In Sign Up

techmap.me
robots.txt

Robots Exclusion Standard data for techmap.me

Resource Scan

Scan Details

Site Domain techmap.me
Base Domain techmap.me
Scan Status Ok
Last Scan2025-03-07T17:41:59+00:00
Next Scan 2025-04-06T17:41:59+00:00

Last Scan

Scanned2025-03-07T17:41:59+00:00
URL https://techmap.me/robots.txt
Domain IPs 104.21.18.139, 172.67.182.78, 2606:4700:3031::ac43:b64e, 2606:4700:3037::6815:128b
Response IP 172.67.182.78
Found Yes
Hash 83cdfd54b304fc0a18f0591f04955456f9407caba20a6fac9719a53a470e67a9
SimHash ef05fdf7815b

Groups

*

Rule Path
Disallow /Javascript/
Disallow /themes/
Disallow /stylesheets/
Disallow /Content/
Disallow /Images/icons/
Disallow /img/icons/
Disallow /img/bodybg/
Disallow /img/camera/
Disallow /img/dummies/
Disallow /img/lp/
Disallow /img/prettyPhoto/
Disallow /img/slides/
Disallow /Company/Delete/
Disallow /Company/Create/
Disallow /Company/Edit/
Disallow /Jobs/Delete/
Disallow /Jobs/Create/
Disallow /Jobs/Edit/
Disallow /Jobs/Payment/
Disallow /Jobs/Search/
Disallow /Thankyou
Disallow /error
Disallow /NotFound
Disallow /ServerError
Disallow /error
Disallow /error
Disallow /error
Disallow /error
Disallow /Privacy
Disallow /Terms
Allow /

mediapartners-google

Rule Path
Disallow /

slurp

Rule Path
Disallow /

yahoo pipes 1.0

Rule Path
Disallow /

googlebot-image

Rule Path
Disallow /*/ivc/*
Disallow /users/flair/

baiduspider

Rule Path
Disallow /
Back to top

Other Records

Field Value
sitemap /sitemap.xml
Back to top

Comments

  • beware, the sections below WILL NOT INHERIT from the above!
  • http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=40360
  • disallow adsense bot, as we no longer do adsense.
  • Yahoo bot is evil.
  • Yahoo Pipes is for feeds not web pages.
  • This isn't really an image
  • Baidu you have been a bad boy
  • this technically isn't valid, since for some godforsaken reason
  • sitemap paths must be ABSOLUTE and not relative.
Back to top