boundless.com
robots.txt
Robots Exclusion Standard data for boundless.com
Resource Scan
Scan Details
| Site Domain | boundless.com |
| Base Domain | boundless.com |
| Scan Status | Ok |
| Last Scan | 2024-09-13T20:38:15+00:00 |
| Next Scan | 2024-10-13T20:38:15+00:00 |
Last Scan
| Scanned | 2024-09-13T20:38:15+00:00 |
| URL | https://boundless.com/robots.txt |
| Domain IPs | 172.66.41.33, 172.66.42.223, 2606:4700:3108::ac42:2921, 2606:4700:3108::ac42:2adf |
| Response IP | 172.66.42.223 |
| Found | Yes |
| Hash | b59272990b5f39a05c29db48065f2ced6e9862413db014918c0094d9a54f0774 |
| SimHash | 524379908ee6 |
Groups
ccbot
claudebot
claude-web
anthropic-ai
bytespider
friendlycrawler
icc-crawler
imagesiftbot
img2dataset
meta-externalagent
omgili
omgilibot
scrapy
timpibot
velenpublicwebcrawler
youbot
semrushbot
semrush
seositecheckup
siphon
sitesucker
vidiblescraper
webbandit
xenu
cognitiveseo
dataforseo.com
magpie-crawler
seobility
seostar
sitechecker.pro
serpstatbot
spyfu
webpros.com
webprosbot
dataforseobot
siteauditbot-mobile
dotbot
majestic
majestic-seo
majestic12
netlyzer
netspider
ninja
phpcrawl
pageanalyzer
pandalytics
pagegrabber
petalbot
mj12bot
deepcrawl
rogerbot
backlink-ceck
pycurl
seokicks
seokicks-robot
seolyticscrawler
seoprofiler
seostats
backlinkcrawler
backlinksextendedbot
builtwith
buzzsumo
crazywebcrawler
domaincrawler
domainstatsbot
erocrawler
extractor
extractorpro
headmasterseo
htmlparser
linkscan
linkwalker
linkbot
| Rule | Path |
|---|---|
| Disallow | / |
*
| Rule | Path |
|---|---|
| Disallow | /wp-admin/ |
| Disallow | /applications/ |
| Disallow | /vendor/ |
*
| Rule | Path |
|---|---|
| Allow | /wp-admin/admin-ajax.php |
Other Records
| Field | Value |
|---|---|
| sitemap | https://www.boundless.com/sitemap_index.xml |