Log In Sign Up

techraptor.net
robots.txt

Robots Exclusion Standard data for techraptor.net

Resource Scan

Scan Details

Site Domain techraptor.net
Base Domain techraptor.net
Scan Status Ok
Last Scan2024-11-02T12:06:04+00:00
Next Scan 2024-11-09T12:06:04+00:00

Last Scan

Scanned2024-11-02T12:06:04+00:00
URL https://techraptor.net/robots.txt
Domain IPs 104.26.10.167, 104.26.11.167, 172.67.75.118, 2606:4700:20::681a:aa7, 2606:4700:20::681a:ba7, 2606:4700:20::ac43:4b76
Response IP 172.67.75.118
Found Yes
Hash fc3bc4ce46262d02d9e86493f2d3d2a1a4aacd360c47469a862f75da0dd8a6c8
SimHash 31d2ad114ec0

Groups

*

Rule Path
Allow /core/*.css$
Allow /core/*.css?
Allow /core/*.js$
Allow /core/*.js?
Allow /core/*.gif
Allow /core/*.jpg
Allow /core/*.jpeg
Allow /core/*.png
Allow /core/*.svg
Allow /profiles/*.css$
Allow /profiles/*.css?
Allow /profiles/*.js$
Allow /profiles/*.js?
Allow /profiles/*.gif
Allow /profiles/*.jpg
Allow /profiles/*.jpeg
Allow /profiles/*.png
Allow /profiles/*.svg
Disallow /core/
Disallow /profiles/
Disallow /README.txt
Disallow /web.config
Disallow /admin/
Disallow /comment/reply/
Disallow /filter/tips
Disallow /node/add/
Disallow /user/register
Disallow /user/password
Disallow /user/login
Disallow /user/logout
Disallow /index.php/admin/
Disallow /index.php/comment/reply/
Disallow /index.php/filter/tips
Disallow /index.php/node/add/
Disallow /index.php/search/
Disallow /index.php/user/password
Disallow /index.php/user/register
Disallow /index.php/user/login
Disallow /index.php/user/logout
Back to top

Other Records

Field Value
sitemap https://techraptor.net/sitemap.xml
sitemap https://techraptor.net/googlenews.xml
Back to top

Comments

  • _____ _ __ _ _
  • /__ \___ ___| |__ /__\ __ _ _ __ | |_ ___ _ __ (_)___
  • / /\/ _ \/ __| '_ \ / \/// _` | '_ \| __/ _ \| '__| | / __|
  • / / | __/ (__| | | / _ \ (_| | |_) | || (_) | | | \__ \
  • \/ \___|\___|_| |_\/ \_/\__,_| .__/ \__\___/|_| |_|___/
  • |_|
  • __ __ ___
  • /\_/\___ _ _ _ __ / _\ ___ _ _ _ __ ___ ___ / _| ___ _ __ / _ \__ _ _ __ ___ ___ ___
  • \_ _/ _ \| | | | '__| \ \ / _ \| | | | '__/ __/ _ \ | |_ / _ \| '__| / /_\/ _` | '_ ` _ \ / _ \/ __|
  • / \ (_) | |_| | | _\ \ (_) | |_| | | | (_| __/ | _| (_) | | / /_\\ (_| | | | | | | __/\__ \
  • \_/\___/ \__,_|_| \__/\___/ \__,_|_| \___\___| |_| \___/|_| \____/\__,_|_| |_| |_|\___||___/
  • Please Respect the Rules Below
  • We're fine with crawlers of many kinds, but not using our content for financial gain. Looking at you AI Crawlers.
  • CSS, JS, Images
  • Directories
  • Files
  • Paths (clean URLs)
  • Paths (no clean URLs)
  • Sitemaps
Back to top