Examining the Log4j2 vulnerability and our response

A serious vulnerability recently discovered in the Log4j2 logging package can be exploited in many Java-based applications, and may even allow an attacker to execute code remotely.

Because of Log4j’s popularity and widespread use in web-based services and enterprise networks, the impacts are far-reaching. Indeed, Microsoft has reported on nation-state groups from China, Iran, North Korea, and Turkey abusing the vulnerability. A handful of security firms have tracked numerous malware groups quickly taking advantage as well.

But the effect on each product that uses Log4j can vary.

The vulnerability, also sometimes referred to as Log4Shell, affects Log4j between versions 2.0-beta-9 and 2.14.1. This vulnerability depends on a feature we’ll call “embedded commands,” which includes message lookup.

The initial vulnerability, CVE-2021-44228 (published December 9), led to the subsequent discoveries of CVE-2021-45046 (published December 14) and CVE-2021-45105 (published December 18). All three are in the Log4j2 Java library. BlueCat Address Manager and BlueCat DNS/DHCP Server (BDDS), both part of BlueCat Integrity, use this library for logging that originates from Java components.

This post will first explore how the Log4j2 vulnerability works. Then, it will look at how severe it is, exactly. Finally, it will detail its potential effects on BlueCat customers and outline BlueCat’s efforts to immediately fix the vulnerability.

BlueCat has released hotfix patches for Integrity 9.1.1, 9.2.1, and 9.3.1.

How the Log4j2 vulnerability works

Widely used by enterprise applications and cloud services, Log4j is an open-source, Java-based logging package developed by the Apache Software Foundation. It’s essentially the de facto way to do event logging in Java.

Log messages often contain a mix of boilerplate and event-specific content. For example, assembling the message “The file Foo could be not found” would happen by using standard boilerplate error text and inserting the file name.

The Log4j interface includes embedded command strings that can result in it invoking additional code when specific character sequences are present in a log message.

Event-specific content can come from user-provided sources. This can include submission through a web-based user interface, sending it in a network packet, or storing it in a file. However, logging features often process text from untrusted sources, like usernames and http(s) request URLs. When data that comes from outside the system can cause code to run, bad actors can insert malicious content. This can cause a server to do something unintended.

The role of JNDI and LDAP

The Java Naming and Directory Interface (JNDI) is a key element of Java leveraged in this vulnerability. With JNDI, a Java program can find data in the form of a Java object through a number of directory services, including the Lightweight Directory Access Protocol (LDAP). JDNI and LDAP work together to find Java objects.

A very popular service, an LDAP server could potentially be running anywhere. Most organizations have an LDAP server, but Java clients aren’t restricted to using their organization’s approved server. This means that if a bad actor can control the LDAP’s URL, they can make a Java program load an object from a server under their control, too.

As Apache has noted on its Log4j security vulnerabilities page, in versions 2.14.1 and earlier, “JNDI features used in configuration, log messages, and parameters do not protect against attacker controlled LDAP and other JNDI related endpoints. An attacker who can control log messages or log message parameters can execute arbitrary code loaded from LDAP servers when message lookup substitution is enabled.”

The vulnerability sequence and Apache patches

Version 2.15.0 of Log4j fixes CVE-2021-44228, the initial vulnerability, by disabling embedded commands by default.

However, 2.15.0 didn’t address another issue, as identified by the second CVE-2021-45046 vulnerability. That one allowed a remote attacker with control over the Thread Context Map to insert malicious input using a JNDI Lookup pattern and remotely execute code in some environments. Version 2.16.0 addressed that problem by removing support for message lookup patterns and disabling JNDI functionality entirely.

The latest bug, CVE-2021-45105, identified that 2.16.0 did not protect from uncontrolled recursion from nested expansions. As a result, an attacker could launch a denial of service attack. Log4j 2.17.0 fixed this issue.

A long-known issue in other programming languages

This is a long-known issue in other programming languages, such as printf string exploits and SQL injection. However, in the path to resolution in printf and SSL, each requires that the programmer make sure that the submission of user-supplied content occurs in a way that it can’t interpret it as a command.

Log4j doesn’t have an equivalent mechanism yet. As a result, admins have to disable the embedded command ability. In Log4j 2.16, developers removed the capability entirely because it creates too great a risk.

Just how severe is the Log4j2 vulnerability?

The Common Vulnerability Scoring System (CVSS) score lists this vulnerability as 10.0 out of 10. But before letting panic take hold, let’s ask just how severe this vulnerability really is, exactly.

Analysis of vulnerabilities under CVSS occurs abstractly, with only information about the coding flaws themselves. There is no consideration for how the code is incorporated into a larger codebase. Nor is there consideration for the business context that determines the impact that the flaw could have.

The 10.0 number listed for CVE-2021-44228 is a “Base Exploitability Metric.” Value assignment occurs in a risk-conservative way based on the worst possible impact it could have.

For example, it would be easy to exploit a server on the internet running Log4j that generates log messages that include attempted user logins. Many security guidelines and compliance frameworks require logging failed login attempts, which means these inputs are likely to reach the Log4j vulnerability. On the other hand, a server inside of a corporate network has risk mitigations. In this case, the attacker can not access it and insert a malicious login string before first bypassing the corporate firewall or VPN service.

Additionally, the server might not have a login interface or might record failed login attempts using something other than Log4j logging. Those cases lessen the real vulnerability. The Log4j CVE might have a base CVSS metric of 10.0 but a specific codebase or product might actually have a rating of 4.9 or lower.

NIST’s CVSS calculator can show you how this works. The base 10.0 (for AV:N/AC:L/Au:N) becomes 4.9 when you take the access complexity of getting into the corporate network into account (AV:N/AC:H/Au:N).

The effects of Log4j2 on BlueCat customers

Many customers reached out to BlueCat after the first disclosure of the vulnerability.

BlueCat did not know about the vulnerability prior to the public disclosure. Many vendors communicate privately about security vulnerabilities in advance. This allows time to create and test fixed versions of products.

Officials published the CVE on December 10, shortly after the first public disclosure and exploits against public servers such as Minecraft. (Short for Common Vulnerabilities and Exposures, a CVE is a publicly disclosed computer security flaw listed by the MITRE Corporation on behalf of the U.S. federal government.) However, Cloudflare and Cisco Talos have subsequently reported that the first attacks occurred on December 1 and 2.

Investigations into the impact on BlueCat’s products and services began immediately after CVE publication.

Vulnerable versions of the Log4j library are present in all supported versions of BlueCat Integrity and BlueCat Edge. There is no vulnerable code in BlueCat Gateway, which does not use Log4j.

A Known Issues (KI) article in the BlueCat Care Community provides additional and continually updated details.

Impacts on BlueCat Integrity

So far, BlueCat identified an insider-threat-only path to exploit the vulnerability in BlueCat Address Manager (BAM).

BAM servers located behind your corporate firewall are only subject to insider threats. For Log4j, the identified exploit path requires access to the web interface.

BlueCat has identified ways to exploit the vulnerability on BDDS if CommandServer’s Log4j is set in DEBUG mode. CommandServer is the Java component on BDDSes. Other debug features that customers may use are independent of setting CommandServer Log4j into debug mode. It must be done in the shell, while editing a file, or directly on a BDDS.

BlueCat has not confirmed any way to exploit the vulnerability against a production BDDS under normal log settings. Customers do not normally enable Log4j DEBUG mode, except under explicit direction from the BlueCat Care teams, and then only for a limited time.

BlueCat released REL-250, hotfix patches for Integrity 9.1.1, 9.2.1, and 9.3.1, to address CVE-2021-44228 and CVE-2021-45046. As the latest CVE-2021-45105 vulnerability cannot be exploited on BlueCat appliances once the REL-250 patch is applied, BlueCat will include the fix for CVE-2021-45105 in the next cumulative maintenance release.

Impacts on BlueCat Edge

BlueCat Edge uses the Log4j library in both the cloud and in the Service Point virtual machine (VM).

The Service Point VM does not log any user-provided strings through Log4j. BlueCat has found no paths to successfully exploit Log4j at this time.

BlueCat Edge services do not log any strings from unauthenticated users or DNS queries. This limits the risk to insider threats.

BlueCat is preparing patches for both the BlueCat Edge and Service Point images. When patched, these will use the latest version of the Log4j library, remediating the vulnerability completely.

How BlueCat can help with prevention

When it comes to your risk of attack from this kind of vulnerability, BlueCat can help protect your networks.

Specifically,  BlueCat believes in restricting DNS on purpose-built servers to only known and allowed DNS records. Using BlueCat Edge would render many exploits for this vulnerability and others moot. The vulnerability would still be there, of course, but it wouldn’t be able to leverage DNS to communicate and exfiltrate data or download arbitrary code. DNS segmentation can add another layer to a network segmentation strategy.

Learn more about how BlueCat can enhance your network security.