aws.amazon.com
robots.txt

Robots Exclusion Standard data for aws.amazon.com

Resource Scan

Scan Details

Site Domain aws.amazon.com
Base Domain amazon.com
Scan Status Ok
Last Scan2024-11-06T04:47:04+00:00
Next Scan 2024-11-20T04:47:04+00:00

Last Scan

Scanned2024-11-06T04:47:04+00:00
URL https://aws.amazon.com/robots.txt
Domain IPs 108.156.133.125, 108.156.133.17, 108.156.133.43, 108.156.133.99, 2600:9000:2755:1400:1c:a813:8500:93a1, 2600:9000:2755:3e00:1c:a813:8500:93a1, 2600:9000:2755:6c00:1c:a813:8500:93a1, 2600:9000:2755:7200:1c:a813:8500:93a1, 2600:9000:2755:9800:1c:a813:8500:93a1, 2600:9000:2755:ba00:1c:a813:8500:93a1, 2600:9000:2755:d800:1c:a813:8500:93a1, 2600:9000:2755:ec00:1c:a813:8500:93a1
Response IP 108.156.133.43
Found Yes
Hash 3cace6240e47f80da14543debddfcf3ec2e49ac89678db68834497ef316063c6
SimHash ab83d1bc859d

Groups

adsbot-google
adsbot-google-mobile

Rule Path
Allow /pm/
Disallow /alexaforbusiness/
Disallow /blogs/
Disallow /ar/blogs/
Disallow /cn/blogs/
Disallow /de/blogs/
Disallow /es/blogs/
Disallow /fr/blogs/
Disallow /id/blogs/
Disallow /it/blogs/
Disallow /jp/blogs/
Disallow /ko/blogs/
Disallow /pt/blogs/
Disallow /ru/blogs/
Disallow /th/blogs/
Disallow /tr/blogs/
Disallow /tw/blogs/
Disallow /vi/blogs/
Disallow /solutions/case-studies/
Disallow /ar/solutions/case-studies/
Disallow /cn/solutions/case-studies/
Disallow /de/solutions/case-studies/
Disallow /es/solutions/case-studies/
Disallow /fr/solutions/case-studies/
Disallow /id/solutions/case-studies/
Disallow /it/solutions/case-studies/
Disallow /jp/solutions/case-studies/
Disallow /ko/solutions/case-studies/
Disallow /pt/solutions/case-studies/
Disallow /ru/solutions/case-studies/
Disallow /th/solutions/case-studies/
Disallow /tr/solutions/case-studies/
Disallow /tw/solutions/case-studies/
Allow /solutions/case-studies/sustainability-emea/
Allow /ar/solutions/case-studies/sustainability-emea/
Allow /cn/solutions/case-studies/sustainability-emea/
Allow /de/solutions/case-studies/sustainability-emea/
Allow /es/solutions/case-studies/sustainability-emea/
Allow /fr/solutions/case-studies/sustainability-emea/
Allow /id/solutions/case-studies/sustainability-emea/
Allow /it/solutions/case-studies/sustainability-emea/
Allow /jp/solutions/case-studies/sustainability-emea/
Allow /ko/solutions/case-studies/sustainability-emea/
Allow /pt/solutions/case-studies/sustainability-emea/
Allow /ru/solutions/case-studies/sustainability-emea/
Allow /th/solutions/case-studies/sustainability-emea/
Allow /tr/solutions/case-studies/sustainability-emea/
Allow /tw/solutions/case-studies/sustainability-emea/
Allow /vi/solutions/case-studies/sustainability-emea/
Allow /solutions/case-studies/innovators/innovators-eu/
Allow /ar/solutions/case-studies/innovators/innovators-eu/
Allow /cn/solutions/case-studies/innovators/innovators-eu/
Allow /de/solutions/case-studies/innovators/innovators-eu/
Allow /es/solutions/case-studies/innovators/innovators-eu/
Allow /fr/solutions/case-studies/innovators/innovators-eu/
Allow /id/solutions/case-studies/innovators/innovators-eu/
Allow /it/solutions/case-studies/innovators/innovators-eu/
Allow /jp/solutions/case-studies/innovators/innovators-eu/
Allow /ko/solutions/case-studies/innovators/innovators-eu/
Allow /pt/solutions/case-studies/innovators/innovators-eu/
Allow /ru/solutions/case-studies/innovators/innovators-eu/
Allow /th/solutions/case-studies/innovators/innovators-eu/
Allow /tr/solutions/case-studies/innovators/innovators-eu/
Allow /tw/solutions/case-studies/innovators/innovators-eu/
Allow /vi/solutions/case-studies/innovators/innovators-eu/

*

Rule Path
Disallow */confirmation/
Disallow */registration-confirmation/
Disallow /about-aws/launch-a-website-in-minutes-tutorial/
Disallow /about-aws/media-coverage/2011/08/17/rhat-support-for-sap-solutions-running-on-rhel-on-amazon-ec2/
Disallow /about-aws/whats-new/2007/11/07/red-hat-enterprise-linux-available-on-amazon-ec2/
Disallow /about-aws/whats-new/2008/06/17/jboss-on-amazon-ec2/
Disallow /about-aws/whats-new/2008/10/01/amazon-ec2-running-windows-server-coming-soon/
Disallow /about-aws/whats-new/2008/11/03/force.com-toolkit/
Disallow /about-aws/whats-new/2008/11/18/force.com-toolkit-2/
Disallow /about-aws/whats-new/2009/01/05/Amazon-CTO-Chief-of-Year/
Disallow /about-aws/whats-new/2009/01/09/Amazon-CTO-Chief-of-Year-2/
Disallow /about-aws/whats-new/2009/07/24/aws-survey/
Disallow /activate/accelerators
Disallow /activate/accelerators/*
Disallow /activate/accelerators/confirmation/
Disallow /activate/china-portfolio-signup/
Disallow /activate/cloudkata/
Disallow /activate/event/
Disallow /activate/event/*
Disallow /activate/events/*
Disallow /activate/events/hact/
Disallow /activate/hackathons
Disallow /activate/hackathons/*
Disallow /activate/harvard-innovation-lab-students-2013/
Disallow /activate/harvard-innovation-lab-vip-teams-2013/
Disallow /activate/hsw/confirmation/
Disallow /activate/loft/
Disallow /activate/self-starters/confirmation/
Disallow /activate/wdm/*
Disallow /alexa/*
Disallow /alexa-top-sites/*
Disallow /alexa-web-search/*
Disallow /apac/aws-credits-confirmation/
Disallow /apac/aws-credits-request-confirmation/
Disallow /articles/item/
Disallow /awis/*
Disallow /aws-summit-2014/enterprise-london-db/
Disallow /aws-summit-2014/london-db/
Disallow /blogs/7faebe5debfdd358a46f2452a54eec94fce49269/
Disallow /campaigns/infor-hub-xt5r5iwdrgxbnuz/
Disallow /careers/aus-nz-event/
Disallow /careers/aus-nz-event/
Disallow /careers/aws-ar-event/
Disallow /careers/aws-aus-event/
Disallow /careers/aws-australia-event/
Disallow /careers/aws-bigdata-hiring/
Disallow /careers/aws-brisbane-event/
Disallow /careers/aws-db-event/
Disallow /careers/aws-ks-event/
Disallow /careers/aws-london-event/
Disallow /careers/aws-mexicocity-event/
Disallow /careers/aws-miami-event/
Disallow /careers/aws-ny-event/
Disallow /careers/aws-pittsburgh-event/
Disallow /careers/aws-portland-event/
Disallow /careers/aws-sea-event/
Disallow /careers/aws-singapore-event/
Disallow /careers/aws-sydney-event/
Disallow /careers/chattanooga-mobile-event/
Disallow /careers/database-hiring/
Disallow /careers/devtools-australia-event/
Disallow /careers/devtools-australia-event/
Disallow /careers/ebs-cincinnati-event/
Disallow /careers/ec2-australia-event/
Disallow /careers/ec2-chicago-event/
Disallow /careers/ec2-hiring-events/
Disallow /careers/ec2-ottawa-event/
Disallow /careers/ec2-vancouver-event/
Disallow /careers/indianapolis-event/
Disallow /careers/mexico-edge-hiring-2014/
Disallow /careers/milwaukee-edge-hiring-2014/
Disallow /careers/montreal-edge-hiring/
Disallow /careers/Phoenix-Hiring-Event/
Disallow /careers/pittsburgh-platform-hiring-2014/
Disallow /careers/raleigh-mobile-event/
Disallow /careers/raleigh-storage-hiring-2014/
Disallow /careers/santiago-event/
Disallow /careers/Sydney-Hiring-Event/
Disallow /careers/texas-event/
Disallow /careers/toronto-event/
Disallow /careers/toronto-storage-hiring-2014/
Disallow /Cloud_Start_01.LandingPage.html
Disallow /Cloud_Start_01-Landing-Page-EDO.html
Disallow /Cloud_Start_02.ConfirmationPage.html
Disallow /code/*
Disallow /compliance/contact-us-confirmation/
Disallow /compliance/hipaa-eligible-services-reference/
Disallow /compliance/how-to-open-aws-compliance-reports/
Disallow /compliance/public-sector-contact-confirmation/
Disallow /compliance/security-assurance-faq
Disallow /compliance/security-assurance-faq-1
Disallow /compliance/security-assurance-faq-10
Disallow /compliance/security-assurance-faq-2
Disallow /compliance/security-assurance-faq-3
Disallow /compliance/security-assurance-faq-4
Disallow /compliance/security-assurance-faq-5
Disallow /compliance/security-assurance-faq-6
Disallow /compliance/security-assurance-faq-7
Disallow /compliance/security-assurance-faq-8
Disallow /compliance/security-assurance-faq-9
Disallow /compliance/security-assurance-faq-complete
Disallow /compliance/security-assurance-US-Public-Sector-checklist
Disallow /compliance/security-assurance-US-Public-Sector-checklist-1
Disallow /compliance/security-assurance-US-Public-Sector-checklist-10
Disallow /compliance/security-assurance-US-Public-Sector-checklist-2
Disallow /compliance/security-assurance-US-Public-Sector-checklist-3
Disallow /compliance/security-assurance-US-Public-Sector-checklist-4
Disallow /compliance/security-assurance-US-Public-Sector-checklist-5
Disallow /compliance/security-assurance-US-Public-Sector-checklist-6
Disallow /compliance/security-assurance-US-Public-Sector-checklist-7
Disallow /compliance/security-assurance-US-Public-Sector-checklist-8
Disallow /compliance/security-assurance-US-Public-Sector-checklist-9
Disallow /compliance/security-assurance-US-Public-Sector-checklist-complete
Disallow /compliance/third-party-access/
Disallow /contact-us/ami-submission-conf/
Disallow /contact-us/article-submission-conf/
Disallow /contact-us/case-contact-confirmation/
Disallow /contact-us/customer-app-submission-conf/
Disallow /contact-us/developertool-submission-conf/
Disallow /contact-us/ec2-email-limit-request-confirmation/
Disallow /contact-us/elasticache-node-limit-request-confirmation/
Disallow /contact-us/marketing-unsubscribe-confirmation/
Disallow /contact-us/publicdataset-conf/
Disallow /contact-us/sales-contact-confirmation/
Disallow /contact-us/samplecode-submission-conf/
Disallow /csds/*
Disallow /customerapps/*
Disallow /datasets/
Disallow /developertools/2555753788650372
Disallow /documentation/alexa-top-sites/
Disallow /documentation/awis/
Disallow /email-preferences
Disallow /email-preferences/
Disallow /en/
Disallow /events/applesummit
Disallow /events/cio-advisory-council/
Disallow /events/finserv-summit-dublin
Disallow /events/finserv-summit-dublin/confirmation
Disallow /events/finserv-summit-singapore
Disallow /events/finserv-summit-singapore/confirmation
Disallow /events/fsi-cloud-symposium-nyc.html
Disallow /federal/events/earthobservationday2015/
Disallow /federal/events/earthobservationday2015/ConfirmationPage/
Disallow /forms/activate-support
Disallow /free/cloud-computing-free-tier/
Disallow /free/databases-free-tier/
Disallow /get-cloud/
Disallow /getting-started/functional-groups/
Disallow /getting-started/functional-groups/*
Disallow /government-education/nonprofits/aws-serve/
Disallow /government-education/nonprofits/aws-serve-member/
Disallow /government-education/nonprofits/credit-program/
Disallow /government-education/nonprofits/credit-program/confirmation
Disallow /government-education/nonprofits/credit-program/terms
Disallow /government-education/nonprofits/grant-program/
Disallow /government-education/nonprofits/grant-program/confirmation/
Disallow /government-education/nonprofits/grant-program/terms/
Disallow /government-education/nonprofits/leadership-forum-poc
Disallow /government-education/nonprofits/leadership-forum-poc/confirmation
Disallow /government-education/public-sector-cloud-start/public-sector-cloud-start/
Disallow /government-education/public-sector-cloud-start/public-sector-cloud-start1/
Disallow /hackathons/reinvent-2013/
Disallow /legal/3874403/
Disallow /lms/contact-us/
Disallow /loft/
Disallow /marketplace/reviews/reviews-list/*/write-comment/
Disallow /marketplace/reviews/write-review/
Disallow /marketplace/search*
Disallow /microsoft/bizspark/
Disallow /microsoft/bizspark/signon/thankyou/
Disallow /nav/
Disallow /nav/side-nav/
Disallow /nav/side-nav/products-services/
Disallow /nav/tabs/
Disallow /nav/tabs/account/
Disallow /nav/tabs/aws/
Disallow /nav/tabs/community/
Disallow /nav/tabs/developers/
Disallow /nav/tabs/products/
Disallow /nav/tabs/support/
Disallow /optin/
Disallow /optout/
Disallow /partners/contact-us/partners-confirmation/
Disallow /partners/overview/partner-newsletter/confirmation/
Disallow /partners/overview/partner-terms-and-conditions/
Disallow /partners/webinars/request/
Disallow /ping
Disallow /pm/
Disallow /ar/pm/
Disallow /cn/pm/
Disallow /de/pm/
Disallow /es/pm/
Disallow /fr/pm/
Disallow /id/pm/
Disallow /it/pm/
Disallow /jp/pm/
Disallow /ko/pm/
Disallow /pt/pm/
Disallow /ru/pm/
Disallow /th/pm/
Disallow /tr/pm/
Disallow /tw/pm/
Disallow /vi/pm/
Allow /pm/amplify/
Allow /pm/lambda/
Allow /pm/sagemaker/
Allow /pm/textract/
Allow /pm/transcribe/
Disallow /premiumsupport/iam/
Disallow /premiumsupport/knowledge-center/feedback-no/
Disallow /premiumsupport/knowledge-center/feedback-yes/
Disallow /pricing/awis/
Disallow /public-data-sets/applicants/
Disallow /public-data-sets/terms/
Disallow /publicdatasets-confirmation/
Disallow /quickstart/promo
Disallow /quickstart/promo/confirmation
Disallow /reinvent/video-case-studies
Disallow /releasenotes/item/
Disallow /research-credits/data-science-bowl
Disallow /ru/blogs/497c2c2d305b7dfa351d17bd3b4398dc37b0e0a4/
Disallow /s/*
Disallow /s/dm/house/editorial/startups/ha_ed_activate_loft_launch_sf_1/
Disallow /sap/saphana/pilot/
Disallow /sap/saphana/pilot/offer/
Disallow /sap/saphana/pilot/offer/confirmation
Disallow /search
Disallow /search/
Disallow /security/aws-pgp-public-key
Disallow /security/ec2-pkgs-public-key
Disallow /security/introducing-s2n-new-open-source-TLS-implementation
Disallow /s-moved/dm/house/editorial/startups/ha_ed_activate_loft_launch_sf_1/
Disallow /solutions/case-studies/imdb/
Disallow /solutions/featured-partners/
Disallow /solutions/global-solution-providers/microsoft/bizspark/
Disallow /solutions/global-solution-providers/microsoft/bizspark/faq/
Disallow /solutions/global-solution-providers/microsoft/bizspark/signon/
Disallow /solutions/global-solution-providers/microsoft/bizspark/signon/
Disallow /solutions/global-solution-providers/microsoft/bizspark/signon/
Disallow /solutions/global-solution-providers/microsoft/bizspark/signon/thankyou/
Disallow /solutions/global-solution-providers/microsoft/bizspark/signon/thankyou/
Disallow /solutions/solution-providers/program/reseller/adguidelines/
Disallow /solutions/solution-providers/program/reseller/adguidelines/
Disallow /solutions/solution-providers/program/reseller/authorized/
Disallow /solutions/solution-providers/program/reseller/authorized/
Disallow /solutions/solution-providers/program/reseller/customerterms/
Disallow /solutions/solution-providers/program/reseller/customerterms/
Disallow /specialhandling-payments/
Disallow /start-ups/challenge/
Disallow /start-ups/loft/hack/
Disallow /start-ups/loft/hacknight/
Disallow /start-ups/loft/sf2014/
Disallow /start-ups/loft/sfff100l3ft
Disallow /start-ups/loft/survey/
Disallow /start-ups/loft/welcome
Disallow /unable-to-process-request/
Disallow /use-cases/global-solution-providers/microsoft/bizspark/
Disallow /use-cases/global-solution-providers/microsoft/bizspark/getting-started/
Disallow /use-cases/global-solution-providers/microsoft/bizspark/signon/
Disallow /use-cases/global-solution-providers/microsoft/bizspark/signon/thankyou/
Disallow /use-cases/global-solution-providers/microsoft/bizspark/thankyou/
Disallow /vmware/about
Disallow /windows/resources/whitepapers/
Disallow /windows/server2003/access/
Disallow /z_dx-marketing-team-prototypes/*

Other Records

Field Value
sitemap https://aws.amazon.com/sitemaps/index/
sitemap https://d36a099zv34xie.cloudfront.net/psf-sitemap.xml
sitemap https://aws.amazon.com/marketplace/sitemap