Log In Sign Up

usapress.net
robots.txt

Robots Exclusion Standard data for usapress.net

Resource Scan

Scan Details

Site Domain usapress.net
Base Domain usapress.net
Scan Status Ok
Last Scan2024-11-11T09:53:32+00:00
Next Scan 2024-11-18T09:53:32+00:00

Last Scan

Scanned2024-11-11T09:53:32+00:00
URL https://usapress.net/robots.txt
Domain IPs 45.130.41.145
Response IP 45.130.41.145
Found Yes
Hash 526dd147f941a9ac89c85ac9a12de29741bf66b0c8c363d40a859fec6778ba43
SimHash 635e1d5d43f0

Groups

*

Rule Path
Allow /component/search
Disallow /administrator/
Disallow /bin/
Disallow /cache/
Disallow /cli/
Disallow /components/
Disallow /includes/
Disallow /installation/
Disallow /language/
Disallow /layouts/
Disallow /libraries/
Disallow /logs/
Disallow /media/
Disallow /modules/
Disallow /plugins/
Disallow /templates/
Disallow /*pik.ru/
Disallow /*17-obzory/
Disallow /*18-stati/
Disallow /*url%3D/
Disallow /tmp/
Disallow /*utm_compaign
Disallow /*utm_source
Allow /politika/749-aleks-dzhons-tramp-svoim-resheniem-sputal-vse-karty-globalistov-po-sirii?utm_source=politobzor.net
Allow /politika/941-ron-pol-amerikantsy-poteryali-doverie-k-bajdenu-iz-za-ego-lzhivykh-obeshchanij?utm_source=politobzor.net
Allow /byvshij-sssr/000290-pochemu-tsvetnaya-revolyutsiya-v-armenii-poterpela-neudachu-issledovanie?utm_source=politobzor.net
Allow /drugie-strany/650-potjomkinskaya-smert-kak-v-germanii-otreagirovali-na-delo-babchenko?utm_source=politobzor.net
Allow /drugie-strany/730-bloger-aleks-shturman-kanada-eto-ne-strana-a-nastoyashchij-kontslager?utm_source=politobzor.net
Disallow /component/search/*limitstart*
Disallow /component/search/*Itemid*
Disallow /component/search/*format*
Disallow /component/search/*url*
Disallow /component/search/*tp*
Disallow /byvshij-sssr/810-zachem-amerikanskie-smi-poyut-difiramby-putinu
Disallow /byvshij-sssr/811-zaprogrammirovannoe-vymiranie-rossii-tendentsiyu-uzhe-ne-perelomit
Disallow /byvshij-sssr/813-minselkhoz-beret-pod-kontrol-sbor-gribov-i-yagod-kakim-budet-otvet-naroda
Disallow /byvshij-sssr/815-izderzhki-nalogovogo-manevra-pochemu-benzin-v-kazakhstane-deshevle-chem-v-rf

Other Records

Field Value
crawl-delay 10

googlebot

Rule Path
Allow /templates/*.css
Allow /templates/*.js
Allow /components/*.css
Allow /components/*.js
Allow /media/*.js
Allow /media/*.css
Allow /plugins/*.css
Allow /plugins/*.js

mj12bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

bingbot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

mail.ru_bot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 500

mediapartners-google

Rule Path
Disallow /proisshestviya/000572-ostatsya-v-zhivykh-po-nyujorkski-russkij-bloger-o-nravakh-temnokozhego-getto
Disallow /proisshestviya/572-ostatsya-v-zhivykh-po-nyujorkski-russkij-bloger-o-nravakh-temnokozhego-getto
Back to top

Other Records

Field Value
sitemap http://www.usapress.net/index.php?option=com_xmap&view=xml&tmpl=component&id=1
Back to top

Comments

  • If the Joomla site is installed within a folder such as at
  • e.g. www.example.com/joomla/ the robots.txt file MUST be
  • moved to the site root at e.g. www.example.com/robots.txt
  • AND the joomla folder name MUST be prefixed to the disallowed
  • path, e.g. the Disallow rule for the /administrator/ folder
  • MUST be changed to read Disallow: /joomla/administrator/
  • For more information about the robots.txt standard, see:
  • http://www.robotstxt.org/orig.html
  • For syntax checking, see:
  • http://tool.motoricerca.info/robots-checker.phtml
Back to top

Warnings

  • `host` is not a known field.
Back to top