KubeSphere Recommendations for Responding to Apache Log4j 2 Vulnerabilities

Apache Log4j 2 is an open-source logging tool that is used in a wide range of frameworks. Recently, Apache Log4j 2 vulnerabilities have been reported. This article provides KubeSphere users with recommendations for fixing the vulnerabilities.

In Log4j 2, the lookup functionality allows developers to read specific environment configurations by using some protocols. However, it does not scrutinize the input during implementation, and this is where the vulnerabilities come in. A large number of Java-based applications have been affected, including Apache Solr, srping-boot-strater-log4j2, Apache Struts2, ElasticSearch, Dubbo, Redis, Logstash, Kafka, and so on. For more information, see Log4j 2 Documentation.

Apache Log4j versions 2.x to 2.15.0-rc2 are affected. Currently, Apache has released Apache 2.15.0-rc2 to fix the vulnerabilities. However, this release is not stable. If you plan to upgrade to Apache 2.15.0-rc2, we recommend that you back up your data first.

The KubeSphere team provides the following three workarounds to fix the vulnerabilities.

• Set the value of environment variable FORMAT_MESSAGES_PATTERN_DISABLE_LOOKUPS to true.
• Add log4j2.formatMsgNoLookups=True to the configmap file.
• Set the -Dlog4j2.formatMsgNoLookups=true JVM option.

Workaround 1: Change the value of the environment variable

KubeSphere uses Elasticsearch to collect logs by default, so it's necessary to fix the vulnerabilities on KubeSphere. The following describes how to fix Elasticsearch.

Run the following commands to edit the Elasticsearch YAML files.

kubectl edit  statefulset  elasticsearch-logging-data -n kubesphere-logging-system
kubectl edit  statefulset  elasticsearch-logging-discovery  -n kubesphere-logging-system

Set the value of FORMAT_MESSAGES_PATTERN_DISABLE_LOOKUPS to true.

env:       
- name: FORMAT_MESSAGES_PATTERN_DISABLE_LOOKUPS
  value: "true"

Workaround 2: Change Log4j 2 configurations

Run the following command to edit the configmap file.

kubectl edit configmaps elasticsearch-logging  -n kubesphere-logging-system

Add log4j2.formatMsgNoLookups=True to the log4j2.properties section.

log4j2.properties: |-
    status=error
    appender.console.type=Console
    appender.console.name=console
    appender.console.layout.type=PatternLayout
    appender.console.layout.pattern=[%d{ISO8601}][%-5p][%-25c{1.}] %marker%m%n
    rootLogger.level=info
    rootLogger.appenderRef.console.ref=console
    logger.searchguard.name=com.floragunn
    logger.searchguard.level=info
    # Add the parameter here.    
    log4j2.formatMsgNoLookups=true    

Note:

1. After you add the parameter, check whether it has been mounted successfully. If not, restart the pod.

2. If you have re-installed the KubeSphere logging component, configmap configurations may be reset. In this case, add the parameter again according to Workaround 2, or you can use Workaround 1.

Workaround 3: Change the JVM parameter of Elasticsearch

You can also set the JVM option -Dlog4j2.formatMsgNoLookups=true. For more information, see the Elasticsearch announcement.

Reference

Artifacts using Apache Log4j Core: https://mvnrepository.com/artifact/org.apache.logging.log4j/log4j-core/usages?p=1