Log In Sign Up

authenticpharm.com
robots.txt

Robots Exclusion Standard data for authenticpharm.com

Resource Scan

Scan Details

Site Domain authenticpharm.com
Base Domain authenticpharm.com
Scan Status Ok
Last Scan2024-05-30T17:31:48+00:00
Next Scan 2024-06-29T17:31:48+00:00

Last Scan

Scanned2024-05-30T17:31:48+00:00
URL https://authenticpharm.com/robots.txt
Domain IPs 206.190.150.84, 2606:2e00:0:a7::4
Response IP 206.190.150.84
Found Yes
Hash 341e681945429ffc2eaf2dcc76274902ec8113668f0db8b8d6501f08e607f626
SimHash 807cf8c1cd91

Groups

*

Rule Path
Disallow /cgi-bin/

adbeat_bot
ahrefsbot
aitcsrobot
alexibot
blexbot
cliqzbot
dotbot
exabot
expo9
huaweisymantecspider
influencebot
ltx71 - (http://ltx71.com/)
ltx71+-+(http://ltx71.com/)
maxpointcrawler
mj12bot
offline explorer
rogerbot
semrushbot
semrushbot-sa
sitesnagger
surveybot
teleportpro
webcopier
webreaper
webstripper
webzip
xaldon_webspider
xenu’s
xenu’s link sleuth 1.1c
wordpress/4.3.1;+http://www.vagabondingheights.com
bitlybot/2.0
nerdybot
woopingbot/1.1
wordpress/4.3.1;+http://www.londonhotelsinformation.com
tineye-bot/0.51+(see+http://www.tineye.com/crawler.html)
yisouspider
securityresearch.bot

Rule Path
Disallow /

applebot
coccoc
ia_archiver
istellabot
grapeshot

No rules defined. All paths allowed.

Other Records

Field Value
crawl-delay 15
Back to top

Other Records

Field Value
sitemap https://www.authenticpharm.com/sitemap.xml
Back to top

Comments

  • Block bad bots
  • Slow some bots down
Back to top