This blog post covers EOP Exchange Online Protection Architecture, and explain in great details how the internal components of EOP work. As things are changing so fast in the cloud, the information in this blog post represent the current state of the service at the time of writing this post. The information presented in this blog post is based on my own understanding and experience of how the service work.

I will be covering the following topics:

• EOP Introduction
• Message Headers
• EOP Exchange Online Protection Architecture
• Appendixes

Note: The information presented in this blog post is based on my own understanding and experience of how the service work and are not based on Microsoft documentation.

EOP Introduction

Exchange Online Protection is an online service from Microsoft that helps protect your organization against spam and malware. You can use this solution even if your email system is hosted on-premises and even if you are not using Microsoft email system on-premises.

While most anti-spam solutions I worked on during the past years are good and effective, they lack the simplicity of configuration and integration. Microsoft Exchange Online Protection offers a simple admin interface to configure and trace different protection features through a simple clean web interface.

Although the admin portal looks simple, EOP has a lot to offer when it comes to anti-spam, zero day attacks, and reporting capabilities. In this blog post, I will go deep behind the scenes, and show you how EOP actually works to ensure such high protection measures.

Message Headers

The first thing to discuss in the EOP Exchange Online Protection Architecture is the message headers.

Introduction

Any email message contains a message body, and a message header. Usually, any SMTP server that participate in the message routing will add its own information into the message header. This information is what makes it possible for you to analyze any message header, and inspect what happened to the message during delivery.

Message headers play a big role in any anti-spam solution. in fact, the whole anti-spam solution depends on message header to tag certain information related to various tests performed by the anti-spam engines.

It goes like this. A message is delivered to the anti-spam solution for inspection, before it is forwarded to the recipient email system. The result of the anti-spam inspection will be tagged into the message header, before it goes to the recipient mailbox or junk folder. By looking at the message header, you can easily see the results of the anti-spam inspection. Exchange SCL and EOP headers play a big role in understanding the mechanics of EOP.

P1 and P2 headers

To understand EOP Exchange Online Protection Architecture (or any other anti-spam solution) works, you must fully understand what Email Message P1 and P2 headers mean.

 

Header Sections

To better understand EOP Exchange Online Protection Architecture, let us learn about the three important header sections to pay attention to:

• X-Forefront-Antispam-Report: This one is important, and it shows many things including the SCL and SFV values.
• X-Microsoft-Antispam: shows the punishing and bulk email confident levels.
• Authentication-Results : showing the result of message authentication.

Finally, I want to give a quick review on couple of things you will see inside the message headers:

• SCL (Spam confidence level): indicates how likely the message is spam. The key thing to remember that along the way, many entities can change the value of SCL. The final change is what matters. For example, you can use Exchange transport rules to force an SCL value for the message based on a condition.
• Spam Filtering Verdict (SFV): It is a dream coming true to every email admin, as it gives you a good reason and explanation, why this message was treated in a specific way.  You might find the value SFV = SFE , which indicates that “Filtering was skipped and the message was let through because it was sent from an address on an individual’s safe sender list.” Check my SFV sheet for a quick listing of all SFV values.
• Connecting IP (CIP): IP address of the message sender. Why this is an important value? If you want to allow or block a sender by IP, to obtain the IP address of the sender whose messages you want to allow or block in the EOP IP Connection Filter, you can check the CIP header value.
• IPV: it shows anti-spam checks related to the IP address of the sender. Values of this can be either:
  • IPV:CAL > The message was allowed through the spam filters because the IP address was specified in an IP Allow list in the connection filter.
  • IPV:NLI > The IP address was not listed on any IP reputation list.
• CTRY: “The country from which the message connected to the service. This is determined by the connecting IP address, which may not be the same as the originating sending IP address”.
• LANG : “The language in which the message was written, as specified by the country code (for example, ru_RU for Russian)”.
• PCL: The Phishing Confidence Level (PCL) value of the message.

SCL Value Sheet

To master EOP Exchange Online Protection Architecture, we should know more about SCL values

SCL Rating	Spam Confidence Interpretation	Default Action
-1	Non-spam coming from a safe sender, safe recipient, or safe listed IP address (trusted partner)	Deliver the message to the recipients’ inbox.
0,1	Non-spam because the message was scanned and determined to be clean	Deliver the message to the recipients’ inbox.
5,6	Spam [suspected spam]	Deliver the message to the recipients’ Junk Email folder.
7,8,9	High confidence spam	Deliver the message to the recipients’ Junk Email folder.

SFV Value Sheet

To master EOP Exchange Online Protection Architecture, we should know more about SFV values.

Category	Header	Description
IP INFORMATION	CIP [Connecting IP]	Connecting IP [This one that shold be put in the connection filter if you want to allow a sender.
	IPV:CAL [IP Verdict]	The message was allowed through the spam filters because the IP address was specified in an IP Allow list in the connection filter.
	IPV:NLI [IP Verdict]	The IP address was not listed on any IP reputation list.
	CTRY [Country]	The Country from which the message connected to the service. This is determined by the connecting IP address, which may not be the same as the originating sending IP address.
	LANG [Language]	The language in which the message was written, as specified by the country code (for example, ru_RU for Russian).
Inspected by Content Filter	SFV:SPM [Spam Filtering Verdict]	The message was marked as spam by the Content Filter.
	SFV:NSPM [Spam Filtering Verdict]	The message was marked as non-spam by the Content Filter and was sent to the intended recipients.
Spam Filter Allow/Block lists	SFV:SKA [Spam Filtering Verdict]	The message skipped Content Filtering and was delivered to the inbox because it matched an allow list in the Spam Filter policy, such as the Sender allow list inside the Spam Filter Policy Allow List.
	SFV:SKB [Spam Filtering Verdict]	The message was marked as spam because it matched a block list in the spam filter policy, such as the Sender block list inside the Spam Filter Block List.
User Mailbox Junk Folder Allow/Block	SFV:SFE [Spam Filtering Verdict]	Filtering was skipped and the message was let through because it was sent from an address on an individual’s safe sender list.
	SFV:BLK [Spam Filtering Verdict]	Filtering was skipped and the message was blocked because it was sent from an address on an individual’s blocked sender list.
SKIP SPAM FILTER	SFV:SKN [Spam Filtering Verdict]	The message was marked as non-spam prior to being processed by the content filter. This includes messages where the message matched a transport rule to automatically mark it as non-spam and bypass all additional filtering or Connection Filter Allow List.
	SFV:SKI [Spam Filtering Verdict]	Similar to SFV:SKN, the message skipped filtering for another reason such as being intra-organizational email within a tenant. This include messages exchanged inside the organization.
Release from Quarantine	SFV:SKQ [Spam Filtering Verdict]	The message was released from the quarantine and was sent to the intended recipients.
FORCE BEING SPAM	SFV:SKS [Spam Filtering Verdict]	The message was marked as spam prior to being processed by the content filter. This includes messages where the message matched a Transport rule to automatically mark it as spam and bypass all additional filtering.
	SCL	The Spam Confidence Level (SCL) value of the message
	H [helostring]	The HELO or EHLO string of the connecting mail server.
	PTR [ReverseDNS]	The PTR record, or pointer record, of the sending IP address, also known as the reverse DNS address.
	X-CustomSpam: [ASFOption]	The message matched an advanced spam filtering (ASF) option.
	SRV:BULK	The message was identified as a bulk email message. If the Block all bulk email messages advanced spam filtering option is enabled, it will be marked as spam. If it is not enabled, it will only be marked as spam if the rest of the filtering rules determine that the message is spam.

EOP Exchange Online Protection Architecture

I will divide the architecture to sections, started from how emails are being delivered to EOP, and how a message goes through each protection and inspection phase in great details. To learn more about how EOP routing and connectivity works, have a look at the Exchange Online Protection Overview blog post.

Inbound Connections

EOP Exchange Online Protection Architecture starts by analyzing how messages are being sent to EOP. Senders to Exchange Online Protection can be:

• On-Premise senders (Hybrid setup) via TLS connections.
• Partner organization via TLS connections.
• Safe Senders: This is an external internet sender that is labeled by the user as a safe sender from his Outlook. This information is propagated to Exchange Online Protection via AADConnect or DirSync.
• Internet Sender.

Further more, if you are using Hybrid Exchange model, where you have Exchange environment on premise, and couple of mailboxes in Office 365, then things can be interesting. In this case, you will have TLS SMTP inbound and outbound connectors between your on-premise Exchange and Office 365. The interesting part is how emails coming though that connector to Office 365 are treated.

So imagine you have an on-premise environment with Exchange servers and on-premise mailboxes. You also have a connector to Office 365 by running the hybrid wizard. There are two types of email flow on your on-premise network:

• A mailbox user opening his Activesync, Outlook, or OWA and sending emails. This user is:
  • Authenticated before sending emails.
  • Is treated by Exchange as AuthAs: Internal  and in the transport rules, it will be considered Inside the Organization  traffic.
  • When that email is sent via the connector to EOP, it will have:
    • SFV: SKI [Similar to SFV:SKN, the message skipped filtering for another reason such as being intra-organizational email within a tenant. This include messages exchanged inside the organization.]
    • SCL = -1

• Printer or an application that needs to send SMTP email without authentication:
  • Not authenticated.
  • Is treated by Exchange as AuthAs: Anonymous  and in transport rules, it will be considered Outside the Organization  traffic.
  • When that email is sent via the connector to EOP, and assuming you added your on-premise public IPs to EOP Connection Filter in the Allow List:
    • SFV:SKN [The message was marked as non-spam prior to being processed by the content filter. This includes messages where the message matched a transport rule to automatically mark it as non-spam and bypass all additional filtering or Connection Filter Allow List.]
    • SCL=-1
    • IPV:CAL  [The message was allowed through the spam filters because the IP address was specified in an IP Allow list in the connection filter.]

Bonus: download my EOP header reference table to better understand all values of SFV. Also, have a look at Exchange SCL and EOP Headers blog post.

EOP Perimeter Protection 

EOP Exchange Online Protection Architecture continues with the EOP Perimeter Protection. The first thing that EOP does after receiving an email message, is to add CIP value to the X-FOREFRONT-ANTISPAM-REPORT message header.CIP stands for Client IP, and this represents the client IP that EOP received the message from. This attribute become handy when you want to add the sender IP to the EOP connection filter allow or block list. You can open the message headers, search for CIP value, and here you can find the IP to be added. The message now looks like the figure below, with both P1 and P2 header, and ready for processing by EOP.

The next step would be the perimeter protection phase. Most inspections here rely on the P1 header information of the message. The phases of the EOP Perimeter Protection are:

• Directory Based Edge Blocking [Dictionary Attack]: If the recipient domain is set to authoritative in EOP accepted domains, then EOP will apply the Directory Based Edge Blocking inspection. It goes like this. If the recipient email address does not exist in Azure AD, the message will be dropped. This technique helps mitigate the dictionary attack.
• Connection Filter Allow/Block List [Static Entries by EOP Admin]:
  • If the CIP  value in the message header matches an entry in the IP Connection Filter block list, then the message will be deleted.
  • If the CIP  value in the message header matches an entry in the IP Connection Filter Allow list, then the IPV  value in the message header will be set to IPV:CAL . This means [The message was allowed through the spam filters because the IP address was specified in an IP Allow list in the connection filter]. Also SCL will be set to SCL=-1  and SFV will be set to SFV:SKN [This means the message will never be inspected by EOP SPAM content Filter]. Then the message then exits the EOP Perimeter Protection phase.
  • If the CIP value in the message header does not match any entry in the IP Connection Allow/Block lists, then move to the next phase.
• Microsoft IP Safe List: In EOP IP Connection Filter settings, there is an option called (Enable Safe List). Microsoft subscribes to third-party sources of trusted senders. Using this safe list means that these trusted senders aren’t mistakenly marked as spam. If the CIP in the message header matches an entry in the Microsoft IP safe list, the IPV  value is set to IPV=CAL.Also SCL will be set to SCL=-1  and SFV will be set to SFV:SKN  [This means the message will never be inspected by EOP SPAM content Filter]. Then the message then exits the EOP Perimeter Protection phase. Else, the message will be send to next phase which is reputation block.
• Reputation Block: This is one of the most important phase in the EOP Perimeter Protection pipeline. It is the reputation list, where Microsoft maintains a list of reputation that is updated and maintained by Microsoft, and help dynamically determine if the message is coming from a bad reputation address. You cannot see what is in the list as it is maintained by Microsoft.
  • If the message was blocked due to this reputation list, the message will be blocked and deleted. You can go to https://sender.office.com and submit a form to whitelist the IP of the message sender (the IP of the sender is available in the CIP  value).
  • If the message is not listed in Microsoft reputation list, then the message will be move on safely to the next phase of EOP pipeline, and the IPV  value in the message header will be set to IPV: NLI (The IP address was not listed on any IP reputation list.) Note that the message SCL and SFV values are not altered here.

Trick: if the message was blocked due to the reputation filter in the EOP Perimeter Protection, and you want the message not to be blocked by the reputation filter, then you can easily do that by making sure the message never reach that reputation inspection box. This can be adding the sender IP to the IP Connection Filter Allow list.

Trick2: Furthermore to the solution proposed in the previous trick, adding the sender IP address to the EOP IP Connection allow list, not only will make the message by pass the reputation block phase, but will stamp the message with SCL=-1  and SFV=SKN . This means that the message will also skip Spam filter. This might be the desired case for you. But sometimes, you want the message to survive the reputation block phase but still being inspected by the Spam content filter. To do that you can later on create a transport rule to set the message SCL to SCL=0 so that it will pass through Spam Content Filter.

Summary: Most messages coming out the perimeter protection phase will have IPV:NLI  and no SCL or SFV value assigned to them. This means they will normally be inspected by spam content filter. If the message is in an IP allow list, it will have IPV=CAL  and both SCL=-1  and SFV=SKN  (SKN means the message was marked as non-spam prior to being processed by the content filter. This includes messages where the message matched a transport rule to automatically mark it as non-spam and bypass all additional filtering or Connection Filter Allow List.

Be careful when adding an IP in the EOP IP Connection Filter allow list. This means that the message will have SCL=-1 and will not go through the spam content filtering.

EOP also will perform some authentication checks on the message and will add the result in the Authentication-Results header section. This includes SPF, DKIM and DMARC checks.

EOP HUB Phase 1

 EOP HUB Phase 1 includes the following:

• Anti-Malware inspection: EOP will use multiple AV engines to inspect the message for malware. the Anti Malware section contains the (Common Attachment Blocking) filter, which will block attachments if they match a predefined set of types, so you can say that you do not want any attachment that contains .exe or .dll.
• Resolver: The resolver engine will do two things:
  • Will take the recipient email address and map it to a recipient in Azure AD, and then will pick the primary SMTP for that recipient and put it in as the recipient address. So suppose that we have a recipient called bob. Bob has two email address, a primary smtp address called bob@contoso.com and a secondary smtp address bob.smith@contoso.com. Now, if the message is sent to bob.smith@contoso.com, the resolver will replace that with bob@contoso.com.
  • If the recipient is group, and the group is synced to Azure AD, then group expansion will happen here.
• Phishing & Bulk email Inspection: EOP in this phase will add the X-Microsoft-Antispam filter with two values BCL  and PCL .
  • The Bulk Complaint Level (BCL) : is a score that is assigned by EOP to indicate if this message is considered a bulk message or not. A complete overview about this value can be found here.
  • The Phishing Confidence Level (PCL):  is a score that is assigned by EOP to which indicates whether it’s a phishing message.

Finally, there is the ZAP that integrates with the Anti-Malware inspection phase. More about ZAP later in this blog post.

EOP HUB Phase 2 – Exchange Transport Rule (ETR)

Next in EOP is the Exchange Transport Rules (ETR). This is the first phase where we can do a quarantine for the message. All previous phases will either allow or delete the message. Now, we can allow the message to pass, delete the message, quarantine the message, or do other things like changing the SCL value of the message even it was set before by EOP.

The important thing to note is that the message quarantine that this phase deals with is called Admin quarantine, and messages are stayed there for 7 days only. End user cannot see content in the admin quarantine. Only admins can look at this quarantine.

Here we can do a lot of things like:

• Allow specific sender address or block it.
• Move the message to the user junk folder by putting the SCL=5 or 6.
• Block executable ( by looking at the file content and not just extension)
• Many other things.

EOP HUB Phase 3 – SPAM Protection

Spam protection include the Spam Content Filter, which is so expensive engine in terms of processing. Usually, the message should pass all previous checks to reach here. Messages also might be configured to bypass spam filtering due to many factors like IP Allow list, or ETR rule.

Since Spam content filtering is so expensive and processing intensive, the message will be inspected before forwarded to spam content filtering:

• Outlook Safe/Blocked Senders check:
  • If the sender is one of the Safe Senders that the user configured in his outlook, then the message SCL will be set to SCL=-1  and SFV=SFE . The message will then exit the whole Spam Protection pipeline and will not be inspected by Spam Content Filtering.
  • If the sender is one of the Blocked Senders that the user configured in his outlook, then the message SCL will be set to SCL=6  and SFV=BLK . The message will then exit the whole Spam Protection pipeline and will not be inspected by Spam Content Filtering. The message will most likely be delivered to the recipient junk folder.
  • Note: if the user configured a safe domain in his outlook like *contoso.com, then safe domains are not synced to EOP. Only safe and blocked recipients are synced.
  • Note: This is important to note that even if the message has previously SCL=-1 or any other value, the Outlook Safe/Blokced Senders phase will ignore that value, and will write a new SCL value if a match happen from within the recipient safe/blocked lists. In other words, if the message makes it all the way through all previous EOP pipeline checks and reaches the Spam Protection phase without being deleted, then the Outlook Safe/Blocked senders check will have the final word when it comes to SCL value. No one after that will change the SCL value of the message inside EOP.
  • Note: Outlook Safe/Blocked Senders check is unique per recipient. For example, you might have someone that blocked bob@contoso.com sender, and other people adding bob@contoso.com as a safe sender inside their outlook. EOP will respect individual choices and will treat them differently according to the user’s preference.
• Previously cleared check
  • If the message sender does not match an entry in the outlook safe/blocked senders check, then the SCL for the message is inspected. the SCL cloud be set by ETR or by IP Connection Filter for example. IF SCL=-1 , then the message will then exit the whole Spam Protection pipeline and will not be inspected by Spam Content Filtering.
  • If the SCL was not set before to -1 by ETR or EOP Perimeter protection, then the message will be checked if SCL was set previously to 5 or 6. You as an admin, might use Exchange Transport Rule to set the SCL to 5 or 6 (force spam) to ensure the message will be ended to the user’s junk folder if it matches a certain criteria. Remember that even if you do that, the message should first pass by the Outlook Safe/Blocked Senders check. When you use the ETR rules to set the SCL to 5 or 6, then the SFV will be automatically be set to SFV=SKS
• Allow/Blocked Senders/Domains
  • There is a place inside EOP Spam Filter configuration called (Allow Lists, Block Lists). This is a place where you can whitelist senders or block senders , not by their IP address like we used to do inside the EOP IP Connection Filter, but by using SMTP addresses and domains.
  • This stage comes only after passing through all previous checks. That is, if the user (Alice) from his outlook added bob@contoso.com as a safe sender, and you as an admin added bob@contoso.com as one of the block lists here, then the message will be send to the user mailbox with SCL=-1. The reason is EOP will not even bother checking since the Outlook Safe/Blocked senders comes first.
  • If the message reaches here, then one of three things will happen:
    • If the sender matches an entry in the Allow Lists inside the Allow.Blocked Senders/Domains phase, then the message will have SCL=-1  and SFV=SKA . The message will then be delivered to the user inbox folder and will not be inspected by the Spam Content Filter.
    • If the sender matches an entry in the Block Lists inside the Allow/Blocked Senders/Domains phase, then the message will have SCL=9  and SFV=SKB . The message will be then sent to the EOP User Quarantine area, where it will stay there for 15 days. End user can access this quarantine area and perhaps release messages from there.
    • If the sender does not match any entry in the Allow Lists nor the Block Lists, then the message will be sent to the Spam Content Filter finally.
  • Note: It is very dangerous to add your own domain names in the Allow list here, as this mean that the message will not be inspected by the Spam Content Filter and therefore, there will not be any spoof detection applied. This might lead to some one sending emails and impersonate white-listed domains.

• Spam Content Filter
  • If the message does not have any SCL at this point, then it will go through the spam content filter to assign an SCL.
  • Spam content filtering is where you can configure the Spam Filter Advanced Options like:
    • how to deal with empty messages
    • SPF hard fail.
    • NDR backscatter.
    • Image links to remote sites.
    • Numeric IP address in URL.
    • Many others [Go to EOP> Spam Filter> Advanced Options]
  • The result of the content filter is simply assigning an SCL to the message:
    • SCL=-1  , SFV = NSPM  : if the message is not spam.
    • SCL=5  , SFV = SPM  : if the message is suspect spam: most likely it will go to the user’s junk folder.
    • SCL=9  , SFV = SPM  : if the message is spam : it will go to EOP user quarantined.
• SPAM Action Phase
  • If the message SCL=9 > Move it to the EOP end user quarantine area. [it will stay there for 15 days, and end user can access that quarantine].

 

Exchange ATP (Advanced Threat Protection) 

If you have a license for Exchange Advance Threat Protection, then the message will be inspected by Exchange ATP. ATP is a very long subject, so I will just list the main three features that ATP does here:

• Safe Links
• Safe Attachements
• Dynamic Delivery

One note here is that ATP does not necessary kick off here, I am note sure in which phase ATP will start processing the message, but for illustration purpose, I added ATP to be after spam filter.

Final Delivery

The message will not be prepared to be sent out to the final recipient as shown in the below figure.

Appendixes

Authentication

P1, P2 Headers

Misc

 

 

 

 

Download the whole EOP Architecture Diagram

You can download the full complete EOP Exchange Online Protection Architecture diagram that I made from here. It has a lot of information, so it might need a lot of zoom and loading times. You can learn more about Office365 SPF DKIM AND DMARC from my fellow MVP Ahmad Mahmoud blog post

 

MICROSOFT 365 ADVANCED THREAT PROTECTION VIDEO

Here is a video I published on my YouTube channel. It is about “How to secure your modern workplace with Microsoft Advanced Threat Protection”. Please subscribe to the channel to get updates on my new videos.

“I will introduce you to Microsoft 365’s threat protection services and demonstrate how Microsoft 365’s threat protection leverages strength of signal, integration, machine learning and AI to help secure the modern workplace from a advanced persistent threats or APT.”