All the AWS security news, research, and technical changes in your inbox, every Monday.
Sunshine Through Stormy Clouds: Bringing AWS Security to Mergers and Acquisitions
Your company just acquired another company – congratulations! Acquisitions present excellent opportunities for growth, whether through expanding business opportunities, or tackling challenging IT projects as a part of the M&A process. Given that it is the 2020s, the new company likely has assets in AWS. How do you assimilate the acquired company’s stack into your environment while balancing business objectives, keeping the lights on, and oh, yeah – security?
The Acquiree may have no similarities other than running workloads in AWS. Their source control, CI/CD, deployment patterns, and IaC language(s) can and likely will differ from yours. Just as every company’s approach to AWS is nonidentical, so too is each acquisition situation. The more common differences:
- Strongly opinionated approaches to compute, data, and network strategy.
- High use of a totally different IaC platform.
- Heavy click ops and low use of IaC to manage infrastructure.
- The Acquiree may not have anyone focused entirely on cyber security matters.
- Substantial differences in workplace culture.
Let’s walk through the six-step process to get you through the next 90 days:
Step 1 - Get your house in order! (Day 0)
Key elements of a Landing Zone
Importance of Organizational Unit structure
Step 2 - Nice to meet you, welcome to the team…now surrender your root access! (Day 1-10)
Step 3 - Let the Games Begin…Onboarding Time! (Day 10-20)
Considerations for Acquisitions with AWS Organizations
Get Down to Business: Onboarding AWS Accounts
Step 4 - Your Shoes Are Untied…Identifying Riskiest Risks (Day 20-35)
Common security issues and themes observed:
Step 5 - Meetings That Should Not Be An E-Mail…Sustaining Momentum (Day 45-90)
Step 6 - The New Normal (Day 90+)
Step 1 - Get your house in order! (Day 0)
Key elements of a Landing Zone
To smoothly absorb AWS accounts as part of M&A, you need to have a great Landing Zone process. A Landing Zone is a general cloud service provider term also used by Azure and GCP to describe a multi-account environment that bolts on all fundamental operational best practices upon account creation to enable builders to build day one. AWS offers two options to landing zones: a subcomponent of AWS Control Tower creatively named AWS Landing Zone, and a custom landing zone solution customers can build. There are heavy trade offs, but I recommend the custom solution to avoid painting yourself into a corner. Though a custom approach will need to be maintained overtime during your cloud journey, it will always remain more flexible. If you are starting from scratch, Orgformation or Org-kickstart can provide a jumpstart.
Your AWS Organization’s Landing Zone and account structure must be well-established, highly automated, and aligned with both business objectives and development methodologies. There are several excellent guides that cover these, below are elements reflective of a solid Landing Zone:
- Account structure and defining when a new account should be created.
- Enabling, monitoring, and protecting all necessary logging to support incident response.
- AWS native and third-party security tools and services.
- Human and non-human access patterns (AWS SSO, third-party IDPs, etc.).
- Infrastructure and plumbing needed to enable IaC.
- Security and governance around IaC including templates for compute, identity, network, and data resources.
- Tagging strategy for accountability, billing, and security uses.
- Standardized VPC and network designs.
Other actions that can can be taken:
- Set billing contacts and automate enterprise support plans if your organization has it .
- Centralize the billions of AWS emails .
- Set account level security controls such as block public access settings for S3, EBS Snapshots, SSM Documents, etc.
- Enforce operating regions.
- Paved paths for producing secure AMIs and container images for compute workloads.
- Deleting default VPCs in all regions.
Rami McCarthy’s 2023 fwd:cloudsec talk: Beyond The AWS Security Maturity Roadmap can help provide greater detail on the above elements.
Importance of Organizational Unit structure
OU structure supports applying organization-level policies such as Resource Access Manager (RAM) , SCPs , AWS Backup Policies , and others. For M&A process, OU structure must also balance applying visibility and guardrails while providing acquired accounts time to assimilate.
Multiple Organizations can be seen as an antipattern, but Amazon does offer acquisitions as one exception . However, I feel this will only increase complexity without yielding substantial operational and security gains. When it is not perfectly executed, the negative outcomes can be disastrous. I would only advocate for an additional AWS Organization longer term to test changes ahead of introducing into your production Organization.
I recommend having an OU dedicated to accounts inherited through acquisitions. Amazon uses the term “Transitional” . I will refer to this as an “Acquisition” OU. Regardless of the name, this OU should have a subset of your total SCPs attached to balance security and functionality. You do not want to start the relationship by blocking instances from scaling up because you have an SCP that blocks IMDSv1 on day one.
Apply the following SCPs for minimum protection:
- Deny the member account the ability to leave the organization. A compromised identity may attempt this in order to defeat preventative controls such as SCPs.
-
Deny the ability to delete, disable, or modify AWS native security controls
you may use such as:
- CloudTrail
- Config
- GuardDuty
- Etc.
- Deny the ability to update account contact or billing information. Adversaries may attempt to change contacts to defeat root MFA challenges or disrupt AWS from attempting to reach out for security issues.
- Deny the ability to delete or modify IAM principals used for deployments, observability, or security.
During the initial periods post M&A, I recommend against applying these types of SCPs as they may break existing active workloads. The initial objectives should be to look, listen, and observe; from there, later you can proactively enforce controls such as :
- Block the use of IMDSv1 on EC2 instances.
- Enforce AWS services and regions.
- Deny unencrypted RDS instances/clusters from being created.
- Tagging enforcement.
- Attachment of AWS managed policies with high level permissions such as AdministratorAccess, PowerUserAccess, etc.
- IAM users with login profiles.
Simplify your life, split the above two categories into separate policies to make it easier to attach to the desired OUs. Like all things public cloud, the answer “it depends” very much applies to OU structure on the way your company operates. Below is a simplified OU structure based on function. You may certainly have a nested OU structure to include business units and other functional demarcation points that makes sense to your company:
Step 2 - Nice to meet you, welcome to the team…now surrender your root access! (Day 1-10)
Setting expectations and relationship building is a critical path to future success. The acquired company may be smaller and have limited budget towards enterprise security tools. It is a chance for them to leapfrog ahead in security visibility that you already have maturity on. Build this relationship and trust by demonstrating and granting access to your tooling and processes. To get there, you will need to:
- Identify if they have an AWS Organization, number of accounts, and how accounts are split up. If they have their own AWS Organization, it does impact how you onboard accounts into your AWS Organization (notes on this later).
- Understand workloads and services used along with their business impact. Identify regions in use. This can be invaluable to incident detection and response later.
-
Identify mission critical data, take backups and snapshots, and store them
in a safe and trusted location.
- This can be later automated into your existing process, but since you do not know the state of security yet, it is best to spend some time ensuring you have a copy in the event of a data loss event before full integration.
- Identify AWS native or third-party security services in use if any.
- Identify acceptable access patterns for humans and non-humans.
- Perform and document a light threat model. This can be later used on initial risk assessments.
During this process, the acquired organization must share root credentials for all AWS accounts. Having control over each account is absolutely vital . Changing root credentials is also highly recommended at this stage. There may be employee turnover during an acquisition introducing risk of misuse of root credentials. Having root access enables the ability to respond to just about any security incident. Use it as an opportunity to build two-way trust with the new team as you demonstrate the ability to caretake over an AWS organization.
Step 3 - Let the Games Begin…Onboarding Time! (Day 10-20)
Considerations for Acquisitions with AWS Organizations
What if the Acquiree has several AWS accounts within an AWS Organization? AWS offers some guidance :
- A member account can only be part of one AWS Organization.
- To be invited into your AWS Organization, a member account must leave its existing Organization.
- To become a standalone AWS account, the account must have a valid billing method tied to it. Amazon does this to protect its revenue which can result in pain.
At this stage, I recommend proactively involving AWS Support to help facilitate the transition and minimize the time spent in limbo.
If you run into the situation where the Acquiree’s Organization has suspended accounts, keep it simple: reactivate the suspended account(s) within the 90 day post-closure period, have that account leave the Organization, and then close the account. Life is too short to be frustrated by AWS accounts that should already be closed.
AWS advises against running workloads in the Organizations management account . I can’t agree more on this; however, the acquired company may have not followed this guidance . Thankfully, you have a great opportunity to right this wrong when it becomes a member account in your organization!
If you need to migrate the Acquiree’s AWS Organization Management Account, then all member accounts must be completely removed from the Organization Management Account not suspended ‐ and the Acquiree’s AWS Organization configuration must be deleted.
Hopefully, AWS improves this process in the future. Also see additional considerations from Houston and Matt Fuller .
Get Down to Business: Onboarding AWS Accounts
Time to r un your Landing Zone process to invite each account and onboard your security and observability stack. This is the time to validate your technical controls, ensure logs are flowing, and security integrations are functioning. Depending on the number of accounts and potential setbacks, this step may take some time.
Consider temporarily disabling alerting on low and medium risk misconfiguration items and low confidence threat detection rules in your SIEM to avoid overwhelming your security operations teams. Low severity security items could be things like ensuring S3 bucket policies only accept TLS connections. The goal in this phase is to learn the environment and focus on high and critical severity risks and active threats.
During an acquisition, it is completely normal for the new accounts to come with security technical debt. For lower-severity findings and noise that typical CSPM tools are notorious for, one approach can be to summarize these types of events on a weekly or monthly basis . This can also be a great opportunity to market your organization’s existing IaC templates making it difficult to do this incorrectly in the future.
The most underrated tip in this phase is to a ssign risk scores based on OU to help isolate these new accounts from existing security metrics measured and reported to leadership .You will want to capture this data and compare over time. It can be a great feeling to show the progress made down the road.
Step 4 - Your Shoes Are Untied…Identifying Riskiest Risks (Day 20-35)
Reviewing all the data
By this point, you should have your CSPM, logging, and other familiar tools integrated with enough time to aggregate security telemetry and translate that data into actionable risks. You should have enough elements to put together the pieces and gauge their overall security risk.
Budget at least two weeks to give yourself time to produce a quality cloud risk assessment. Do not overwhelm the acquired team with a spreadsheet of 10,000 findings from Prowler and call it an assessment. Telling them “MFA delete is not enabled for every S3 bucket” will not prevent initial access on a major security breach.
Instead, take it a step further and relate findings to a realistic attack path based on your threat model from Step 2. Use a tool like CloudFox which can quickly perform heavy lifting and steer you on the right path. See Nick Frichette’s hackingthe.cloud or Lizzie Moratti’s Ramp Up Guide to AWS Pentest Methodology for inspiration on how to avoid “scan and rebrand” or risk damaging trust.
Invest time identifying and reducing risk related to:
- Long-lived static access keys with permissive access.
- Resources that should not directly be public such as EC2, EKS, SNS, SQS, and RDS.
- Internet accessible compute and data resources with high-severity misconfigurations, poor patch hygiene, and/or EOL operating systems.
- Lack of security visibility and monitoring. Onboarding their accounts into your organization helped resolve this.
These items are easiest for attackers to find and exploit and often the root cause of several major cloud security breaches. Learn from past incidents . Remember the acquired company may not have personnel focused on cyber security. They will need help prioritizing ROI on security risks.
Common security issues and themes observed:
IAM
- Root Access Keys existing, and worse, active use in workloads.
- Use of IAM Users by humans attached to highly permissive custom or AWS managed policies such as AdministratorAccess or PowerUser. These users may or may not have MFA, or even strong password policies applied.
-
Access keys used both by humans and services, often dating back as old as
the acquisition company has existed. IAM users may have two active access
keys.
- Bonus to look for these keys hardcoded in all kinds of places: AMIs, source control, S3 buckets, Lambda environment variables, Cloudformation parameters, container images, etc.
- Overly permissive execution roles (ECS, Lambda, etc.) or resource policies (S3 bucket, SNS, SQS, and others) granting service:* or attached to risky AWS managed policies.
- Misconfigured cross-account role assumption leading to potential confused deputy .
- Former third-party companies with active IAM Roles.
-
IAM Role trust relationships and resource policies making them publicly
accessible.
- If you are faced with a circumstance granting principal:* level access, a quick win to reduce risk from critical to far less severe would be to use the aws:PrincipalOrgID global condition to at least reduce access to just your organization. These situations can be time consuming to sift through logs and lock down to least-privilege.
Compute
- Hardcoded IAM User Access Keys in workloads such as EC2 instances rather than using Instance Profiles.
- Lack of a patching strategy, or any consistent patching.
- Misconfigured EC2 Bastion hosts.
Network / Data
- Security Groups with wide access over administration ports. Often the same Security Groups are attached to several resources.
- Publicly accessible resources such as S3 buckets (non-intentional), EC2 instances with internet-routable ENIs attached, public RDS instances.
Work with the team to prioritize with strict focus on high and critical-risk items. Establish a central location to track issues and be willing to work with their preferred systems initially. It may even help to devote time to remediate flaws, which can go a long way in building relationships and establishing a positive security culture.
Step 5 - Meetings That Should Not Be An E-Mail…Sustaining Momentum (Day 45-90)
Hopefully, the assessment was well received and the team is working through critical-risk items. Below are some examples of activities that should be occurring in this phase to demonstrate success:
- Remediate all critical risk security items with a direct attack path.
- Convert human access to your own SSO/IDP practices.
- Replace IAM User Access Keys with IAM Roles.
- If the workload does not support it: r otate all Access Keys as you never know who may have been in possession of them, reduce policies to least privilege, and attach restrictive IAM conditions to limit how they can be used such as aws:SourceIP .
- Install any agent based security tools such as EDR and CNAPP.
- Tune anomalous behavior-based threat detections that trip existing SIEM rules to reduce alert fatigue.
Establish a regular cadence to sustain momentum. Depending on the state of security upon onboarding, it could be a project of projects. There may also be uncertainty given the acquisition; some workloads will be seen as redundant and be retired, others may go through their own merger with existing systems, and others may have to live on forever in its current state. These are factors that should be part of the decision process for risk remediation efforts.
Step 6 - The New Normal (Day 90+)
If you made it this far into the journey (and blog post), congratulations! Things should start to become more “normal” and routine. The previous phases were aimed at discovering and remediating foundational and high-risk security items. Hopefully you have succeeded at that. You should start to be able to start addressing lower priority items. Below are items that should occur during the final phase of the acquisition journey:
- Move the account(s) to an OU with the rest of the SCPs attached.
- Onboard the acquired organization into your ongoing vulnerability and issue management program.
- Adopt your IaC deployment practices.
- Safely enable any automated reactive remediation controls you may have.
- Migrate workloads into new AWS accounts created organically in your organization.
- Seize any opportunity to migrate from long-lived access keys where possible.
- Secure integration of software or services into your company; after all, they were acquired for a reason.
The steps in this article are designed to contain and gain control on security during an M&A with AWS workloads. While it would be nice to suggest closing the accounts, in reality, that may not happen for a long time, if ever. Never give up or feel discouraged, there are several great wins from the journey that can go unnoticed…until the next acquisition includes a GCP Organization with 428 Projects.
These were lessons learned from professional experience, and do not represent the views of my current or previous employers.