HomeLog4j Zero-Day Vulnerability Response

Log4j Zero-Day Vulnerability Response

Last Updated: January 7, 2022

Change Log

Date	Updates
01/07/22	Updates to Executive Summary including recommended actions and FTC Log4Shell remediation warning
12/30/21	Added CVE-2021-44832 to Executive Summary and a note on Next-Generation Firewalls to Additional Actions
12/23/21	Added CIS and MS/EI-ISAC Products and Log4j and indexed table of contents
12/22/21	Added Change Log and updated Playbook graphic based on member feedback
12/21/21	Added CIS Products and Log4j Vulnerability link
12/20/21	Minor updates to Recommendations
12/18/21	Rewrote Executive Overview to include new CVEs and updates related to version 2.17.0
12/17/21	Minor updates to Executive Overview and Recommendations; added Yahoo tool
12/16/21	Added tools to Appendix A and added new sources
12/15/21	Updated content and sources; included note that initial patch for CVE-2021-44228 was incomplete and workarounds are ineffective; included CVE-2021-45056
12/14/21	Page created with Exec Summary, Recommendations, Playbook, and sources

Executive Summary

On December 9, 2021, security researchers discovered a flaw in the code of a software library used for logging. The software library, Log4j, is built on a popular coding language, Java, that has widespread use in other software and applications used worldwide. This flaw in Log4j is estimated to be present in over 100 million instances globally. This vulnerability and associated attacks against it are being characterized as Log4Shell in the cybersecurity community.

The flaw, also known as a vulnerability by the security community, was rated a 10 out of 10 on the Common Vulnerability Scoring System, or CVSS, due to the potential impact that it can have if leveraged by attackers. Details of the vulnerability can be found in the National Vulnerability Database (NVD) under the heading CVE-2021-44228.

As of Dec. 14, researchers discovered that the fix developed for CVE-2021-44228 was incomplete and the vendor, Apache, released a new fix. On Dec. 17, two new issues were confirmed and the next day, Apache released another fix. We expect this cycle of vulnerability-fix vulnerability-fix will continue as attackers and researchers continue to focus on Log4j.

To simplify things, the current list of vulnerabilities and recommended fixes is listed here:

CVE-2021-44228 (CVSS score: 10.0) - A remote code execution vulnerability affecting Log4j versions from 2.0-beta9 to 2.14.1 (Fixed in version 2.15.0)
CVE-2021-45046 (CVSS score: 9.0) - An information leak and remote code execution vulnerability affecting Log4j versions from 2.0-beta9 to 2.15.0, excluding 2.12.2 (Fixed in version 2.16.0)
CVE-2021-45105 (CVSS score: 7.5) - A denial-of-service vulnerability affecting Log4j versions from 2.0-beta9 to 2.16.0 (Fixed in version 2.17.0)
CVE-2021-4104 (CVSS score: 8.1) - An untrusted deserialization flaw affecting Log4j version 1.2 (No fix available; Upgrade to version 2.17.0)
CVE-2021-44832 (CVSS score: 6.6) - Remote code execution vulnerability affecting Log4j2 versions 2.0-beta7 through 2.17.0, excluding security fixes for 2.3.2 and 2.12.4. (Fixed by upgrading to 2.3.2 (for Java 6), 2.12.4 (for Java 7), or 2.17.1 (for Java 8 and later).

We recommend following the advice of Apache, which recommends updating to the latest version of Log4j immediately.

Please note that all 1.x versions of Log4j are considered “End of Life,” which means they are no longer supported by Apache. While the vulnerabilities (CVEs) listed above are all associated directly with Log4Shell, there are several other known vulnerabilities in 1.x versions of Log4j that can leave users open to a wide range of attacks. It is for this reason that we recommend all Log4j users update to the latest 2.x version available immediately.

When the initial vulnerability was made public, it was described as a zero-day (or 0day), which means it was being targeted and potentially acted upon prior to the software developers knowing that it existed. In other words, the developers had zero days to implement a fix for the vulnerability before it may have been used in an attack.

Due to the widespread prevalence of Log4j, the high impact of an attack against it, and evidence that malicious actors are actively targeting organizations with vulnerable versions of Log4j, CIS is encouraging all organizations to mitigate risk as soon as possible. This page is intended to help all organizations, regardless of technical maturity, to find resources for mitigating risks associated with Log4j.

Security patches are available and CIS encourages all organizations to implement these updates as quickly as possible. Additionally, any organization that relies on outside providers for services that may use Log4j should work with those providers to ensure these third-party relationships do not open them up to undue risk.

It is important to note that simply updating Log4j may not resolve issues if an organization is already compromised. In other words, updating to the newest version of any software will not remove accesses gained by adversaries or additional malicious capabilities dropped in victim environments. Therefore, CIS recommends vigilance in investigating activity in your environment, looking for evidence of unauthorized access, and acting in accordance with incident response best practices to reduce exposure. State, Local, Tribal, and Territorial (SLTT) governments and affiliated organizations, including Elections organizations, may contact the CIS SOC for assistance if you believe you were impacted or compromised.

Threat-focused security organizations have observed state actors begin to leverage the vulnerability in new attacks. CIS recommends heightened vigilance in monitoring networks for abnormal behaviors and invoking immediate response actions as necessary.

The Federal Trade Commission (FTC) recently warned U.S. companies under it's purview to remediate Log4Shell vulnerabilities and that it intends to use the commission's full legal authorities to pursue organizations failing to reduce consumer exposure to these vulnerabilities.

Recommendations

MS-ISAC and EI-ISAC members are encouraged to reach out to the Security Operations Center (SOC) as soon as possible if you suspect you have been impacted by an attack against this vulnerability.

The MS-ISAC is currently working with members who have been impacted. The scope of this vulnerability is more widespread than any other case in recent history. We aim to support all SLTTs and respectfully ask for your patience as we work through this together.

Additionally, we have the ability to scan your external facing infrastructure with up-to-date signatures from leading vulnerability vendors Tenable and Qualys. If you are a member or qualify to join (see here for MS-ISAC, and here for EI-ISAC membership information), please inquire with our SOC about a scan.

The Department of Homeland Security (DHS) Cybersecurity and Infrastructure Security Agency (CISA) will be standing up a single source landing page for guidance, remediation support, and reporting related to this vulnerability.

Immediate Actions

First and foremost, we encourage all organizations to immediately patch any instances of Log4j to the latest supported version available. Apache continues to post new versions on their security update page. It may be helpful to review Apache’s notes on this page.

If you are unable to fully update products that rely on Log4j due to those products being maintained by a third party, reach out to your third-party contacts as soon as possible to get updated guidance. They may have released updates without notification.

If you or any third-party providers are unable to update software in a timely fashion, CIS recommends uninstalling or disabling Log4j until updates can be applied. This is especially critical for any Internet-facing products using Log4j.

The MS-ISAC created this flow chart for members and others to follow for identifying and mitigating risk related to this vulnerability:
Note: Appendices are available below this graphic.

Log4j Chart PNG

CIS and MS/EI-ISAC Products and Log4j

The list of technology products impacted by the Log4j vulnerability continues to grow, given the ubiquity of the code.

CIS has made available a list of our products that have been impacted by the vulnerability as well as the mitigations applied by CIS. The steps for users to update to the latest version(s) are being tracked and updated in near real-time. You can access the most up to date information in our knowledge base here.

Separately, our Security Operations Center (SOC) and engineering teams have evaluated the software products offered to MS-ISAC and EI-ISAC members and have determined that the services are unaffected by the Log4j vulnerability, at this point in time. These services include, but are not limited to:

Albert Network Monitoring
CIS Endpoint Security Services (ESS) and Endpoint Detection and Response (EDR)
Malicious Domain Blocking and Reporting (MDBR)

CIS will continue to provide updates on this page and in our WorkBench knowledge base when there is new information to share, or there are recommended actions for our members to take.

Appendix A: Detecting Vulnerable Software

Section 1: Affected Vendors

Section 2: Source Code Scanning

CERT CC (Carnegie Mellon)
Logpresso
Grype – Searches libraries installed on a system and displays vulnerabilities present
Syft – Searches for installed code and libraries and displays their versions

Section 3: Active/Dynamic Testing

Open source:

CISA Log4j Scanner
NMAP Scripting Engine
CyberReason – (also performs mitigations; however, we recommend exercising caution with automated mitigations)
Huntress Tester
FullHunt
Yahoo Check for Log4j (somewhat intrusive and should be executed with care)

Commercial:

Tenable Nessus Plugin (Note: to run this plugin, an entity must already run Nessus; Tenable is encouraging non-customer organizations to download a community license for Nessus scanner)

Appendix B: Manual Mitigation

NOTE: There is strong evidence that the workarounds are not effective. Therefore, we encourage everyone to update to the latest version from Apache.

According to Section 3 of the Trusted Sec playbook, “If it is not possible to upgrade, there are several mitigation tactics that can be performed.”

For Log4j versions >= 2.10, set the log4j2.formatMsgNoLookups system property to true on both client- and server-side components.

This can be done in multiple ways:

Add -Dlog4j2.formatMsgNoLookups=true to the startup scripts of Java programs; or
Set the following environment variable: LOG4J_FORMAT_MSG_NO_LOOKUPS=”true”

For Log4j versions from 2.0-beta9 through 2.10.0, remove the JndiLookup class from the classpath. For example:

zip -q -d log4j-core-*.jar
org/apache/logging/log4j/core/lookup/JndiLookup.class

Additionally, they recommend the following actions:

Isolate systems into their own restricted DMZs or VLANs
Block all outbound network connections from servers or limit outbound network connections to trusted hosts and network ports
Turn on any endpoint or network security signatures for Log4j exploitation
Monitor networks and servers closely for suspicious or unexpected behavior

TrustedSec further recommends testing and evaluation “after ALL mitigation steps above have been completed to ensure that vulnerabilities do not persist.”

Appendix C: Detecting Exploitation

Open Source

Commercial

Splunk

Appendix D: Engaging MS-ISAC CIRT

If you require CIRT assistance, please request support through the SOC. The CIRT will reach out and will require information from your organization. Please be prepared to provide as many of the following items as possible:

A list of known affected systems
A list of patched systems
A Kape capture of the known affected system or systems (pre-patched state if possible)
Download the contents of the Kape tool file from our Sharefile
Place folder contents into a folder on the affected machine(s)
Run the Kape executable
Output will be found in same folder where Kape tool was placed
Provide output to CIRT analysts via Sharefile folder which will be provided by CIRT once case is acknowledged and a specific Sharefile folder for your incident has been created
Any relevant logs collected, to include, but not exclusive to:
- Network logs
- Endpoint Detection logs
- IDS/IPS logs
- Suspicious outbound connections
- Apache logs

Additional Actions

We recommend the following actions in addition to the immediate actions above:

If you use an outside IT or cybersecurity provider (e.g., internal SOC, Managed Security Service Provider [MSSP], etc.) ensure they are aware of, monitoring, and taking appropriate actions on any alerts associated with the presence of Log4j in your environment
1. Ask them specifically for evidence of outbound traffic as a result of a Log4j exploitation attempt

Install or modify an existing Web Application Firewall (WAF) with rules that automatically update with the latest information Note: Palo Alto recommends adding a Next-Generation Firewall (e.g., a firewall powered by Machine Learning algorithms) for the best protection against these kinds of attacks. WAFs alone can be noisy and may miss attacks that deviate from the rules, while Next-Gen Firewalls block activity based on deviations from normal or expected behavior.
1. CISA will be maintaining a repository of information and affected software here
2. Another useful and trusted repository is being maintained by the Dutch National Cyber Security Center
3. Other sources of information are listed in the Resources section below; however, CIS cannot vouch for the consistency and reliability of data in all resources shared

All affected organizations are encouraged to report compromises to CISA and the F BI. For MS-ISAC and EI-ISAC members, we recommend notifying the SOC and seeking assistance through our incident response team, which is a no-cost service available to all SLTT and Elections organizations in the U.S.