FedRAMP Pen Test Scope vs. Rules of Engagement Explained

FedRAMP has strict requirements for the security of the companies looking to earn their certification. Among the many requirements you need to navigate are tests from your C3PAO, simulating malicious actors and common threat vectors.

In order to understand what you need to do to pass, it’s worth going over what penetration testing is, what red teaming is, what the scope of FedRAMP pen testing includes, and what the rules of engagement encompass.

Two Kinds of Assessments

First, let’s talk about two common terms, which are often used interchangeably, though they aren’t actually the same.

The first is penetration testing. Penetration testing is what it sounds like: tests run against a system, process, application, network, or team with the goal of gaining unauthorized access to the systems beyond.

A red team assessment is similar; it’s simulated attacks against your organization using various threat vectors.

So, what is the difference? The key is in awareness. With penetration testing, the people involved are aware it’s happening. The goal is to see how processes and security in place will handle a threat, whether passwords are strong enough, whether MFA is in place, whether systems are vulnerable to injection attacks, whether employees react properly to phishing attacks, and so on.

Red teaming, meanwhile, is done without the awareness of most of the people involved. High-level leadership will be aware and will authorize the testing, and the red team won’t actually compromise, leak, or damage systems or processes. However, the people lower in the chain aren’t aware that the tests are going to happen, so it’s much more like a real attack in terms of how they act and react.

Both kinds of testing are encompassed within the overall banner of pen testing, which is why the terms are used much the same way; the differences are defined in the scoping process and rules of engagement for any individual test.

FedRAMP Testing Rules of Engagement

The Rules of Engagement for penetration testing and red teaming is your testing plan. These kinds of assessments aren’t performed without awareness, buy-in, and a limited scope. A red team doesn’t simply attack via any vector, for any system, in hopes of finding a way to break in; they have specific goals and tests to perform.

According to the FedRAMP penetration testing guidance document, the rules of engagement stipulate that a testing plan must include:

• A documented description of the planned attacks, including the approach taken, the constraints of the attacks, and the methodologies used.
• A test schedule that specifies when tests will start and end and the content of each testing period.
• The point of contact for the testing and a backup of the subsystems or applications that may be included in case damage is done.

There are also official guidelines for the rules of engagement. As always, these are specified by the National Institute of Standards and Technology. In fact, there are four relevant NIST documents to learn for penetration testing and assessments. They are:

• NIST SP 800-115 – Technical Guide to Information Security Testing and Assessment
• NIST SP 800-145 – The NIST Definition of Cloud Computing
• NIST SP 800-53 – Security and Privacy Controls for Information Systems and Organizations
• NIST SP 800-53A – Assessing Security and Privacy Controls in Information Systems and Organizations

These outline things like the definition of cloud systems, the limitations and scope of information systems, and the guidelines for how tests should be planned, documented, conducted, and reported.

The rules of engagement also need to include information about the local incident response team, the physical constraints of testing, any acceptable boundaries or pretexts for social engineering attacks, and customization to suit the organization and systems being tested.

The Defined Scope of Penetration Tests

One key to effective penetration testing is the defined scope of the testing itself. One of the biggest differences between real attacks and penetration tests is this limitation in scope.

Scope is also defined by the NIST documents above. Broadly speaking, any systems, services, processes, and devices that are part of FedRAMP-secured systems are in scope. Anything outside of the FedRAMP boundary is not to be tested because it doesn’t need to be held to the same standard of security. This is also why scoping is important in FedRAMP in general. The less you have to secure, the fewer systems need to be maintained, and the less there is to attack.

In addition to the broad FedRAMP boundary, individual penetration tests will have certain systems that are in scope while the rest are out of scope. These are defined in the planning phase and help keep different kinds of attacks focused and on target.

Scoping may also require buy-in from other parties. For example, an ISP may need to be made aware of a red team attack and remove filters or limitations that would hinder the attack. Leaseholders for physical facilities, similarly, may need to be made aware so third-party security doesn’t get involved and escalate a situation unintentionally.

Red teams will also generally work with you to find ways to test with minimal disruption to actual working systems and customers; they will avoid attacks such as DDoS attacks that are indistinguishable from real attacks.

The Six Attack Vectors

While attacks are many and vary, most of them can be defined as one of six types of attack along a particular vector. FedRAMP outlines these six vectors as mandatory for penetration testing. More information about each of these vectors is available in the FedRAMP guidance for penetration testing document.

Attack #1: External to Corporate

The first kind of attack is something nearly all of us are at least passingly familiar with: phishing attacks. Whether more generic phishing or more targeted spear phishing, these are a kind of social engineering attempting to target specific system administrators or management personnel whose access can compromise systems if they aren’t paying attention.

There are a few quirks with this kind of attack testing.

• Even if your systems are normally very effective at blocking phishing attempts, in order to properly conduct a test, the phishing must be allowed through to see how employees will actually respond to suspicious requests.
• Companies will generally provide a list of employee names and emails for the test, even if that data is usually not readily available.
• Any response to a phishing message, even if the data is intentionally faked, is considered a failure.

Since phishing attacks are extremely commonplace, but they can also get intensely sophisticated, it’s critical to have well-trained employees who know the proper response, hence the importance of this test.

Attack #2: External to Target System

This is a two-part vector of attack, checking specific systems within your business. It involves testing two kinds of attacks: external threats and internal threats. External threats involve the penetration testing team doing reconnaissance, scanning for vulnerabilities, and testing various exploits to see if any internet-facing connections are vulnerable to attack.

Internal threats assess similar potential access and consequences from an attacker that has internal access to your network.

Attack #3: Tenant to Management System

The third kind of attack is a full-scale application pen test, which attempts to access your general management systems. They can examine all kinds of different flaws and potential attack vectors, including flawed system design, abused functionality, low-code or no-code software deployment, CLI attacks, misconfiguration, and more.

Since this test originates from a tenant account, your testing team will be provided with privileged accounts for the environment from which to conduct the testing. These accounts can’t be overly restricted and must have access equivalent to the highest level allowed to customers.

Attack #4: Tenant to Tenant

This is similar to the previous attack, where your testing team is given access to a high-permission customer account. The difference is that rather than attacking the host system, the testing team attempts to compromise another customer account.

To avoid causing issues with actual customers, a second customer account needs to be set up for the testing team.

Attack #5: Mobile to System

This test simulates attacks on target systems using mobile applications where relevant.

Since not all CSPs have mobile apps on either iOS or Android, this testing may be less useful, but most CSPs these days are at least accessible via a mobile device and thus potentially vulnerable to mobile-only attacks.

Attack #6: Client Application to Target System

The final mandatory attack is an attack from client-side applications to your target systems. You need to provide a list of in-scope client-side applications, such as customer components and other locally installed elements of your customer environment. Since these are considered within the boundary for FedRAMP activity, they need to be tested.

The testing team will generally create their own virtual machine version of the environment for testing.

Exploring a Red Team Assessment

So, what does the overall process of a penetration test or red team assessment look like?

Step 1: Planning. Each individual test needs to be planned, with the rules of engagement defined, the scope and goals of the test enumerated, and the overall goal of the assessment outlined. For FedRAMP assessments, the scope is generally the FedRAMP boundary, so CSP systems outside of the FedRAMP boundary are exempt from the testing. This is also where communication protocols are established for escalations and disruptions in testing.

Step 2: Reconnaissance. Your red team will now start using intelligence software and techniques to gather data that can help them perform their attacks. While you do have to give them certain information for some testing, they will also gather an information profile to see what is available from the outside regardless. Note that privileged information is generally not allowed outside of specific replication of customer accounts as outlined in the rules of engagement. Testing is done with a “black box” approach using only publicly-discoverable information whenever possible.

Step 3: Discovery. Red teams now start to use their testing techniques to explore and discover vulnerabilities that may exist, both in terms of technical flaws and personnel or training issues.

Step 4: Persistence. The goal of attackers is generally not to perform one-and-done attacks but to set up backdoors and establish persistent access that they can reuse as necessary. Your red team will simulate this as much as they can with any vulnerabilities they find, with mechanisms that can survive reboots or incident response actions.

Step 5: Lateral Movement. Once red teams have established a foothold, they will attempt to move laterally to other systems, access credentials, perform discovery, and generally see how broadly they can access and scrape information within the FedRAMP boundary.

Step 6: Exfiltration. Once they have done as much exploration and compromised as many systems as they can, the red team will attempt to extract whatever critical data they can. For the same of business operations, this will use mock data rather than real data, but the process and results would be the same as if the data was accessed normally.

Step 7: Reset. When the testing is complete, the red team will go through and undo any damage, changes, or access they created. They will remove files, reset configurations, restore accounts, and generally remove traces of their presence. After all, they don’t want to have their leftovers used by actual malicious attackers.

Step 8: Reporting. A full report on what they found – both the positives, in terms of tests that failed, employees that passed, avenues that were blocked off, and the negatives they found – will be documented. This report is given to stakeholders and provides a comprehensive account of vulnerabilities. Red teams also use their expertise to instruct the CSP on how to fix the vulnerabilities wherever possible.

At Ignyte, we can help. The Ignyte Assurance Platform helps you maintain documentation for your FedRAMP compliance and auditing, including the reports generated by pen testing. We’re also very familiar with everything involved in FedRAMP authorization and can help you with the process along the way. From performing audits to gathering documentation to achieving your ATO, we’re here to help, so just reach out and see what we can do for you.