Guide to the Secure Configuration of Red Hat OpenShift Container Platform 4
with profile CIS Red Hat OpenShift Container Platform 4 BenchmarkThis profile defines a baseline that aligns to the Center for Internet Security® Red Hat OpenShift Container Platform 4 Benchmark™, V1.5. This profile includes Center for Internet Security® Red Hat OpenShift Container Platform 4 CIS Benchmarks™ content. Note that this part of the profile is meant to run on the Platform that Red Hat OpenShift Container Platform 4 runs on top of. This profile is applicable to OpenShift versions 4.12 and greater.
https://www.open-scap.org/security-policies/scap-security-guide
scap-security-guide package which is developed at
https://www.open-scap.org/security-policies/scap-security-guide.
Providing system administrators with such guidance informs them how to securely configure systems under their control in a variety of network roles. Policy makers and baseline creators can use this catalog of settings, with its associated references to higher-level security control catalogs, in order to assist them in security baseline creation. This guide is a catalog, not a checklist, and satisfaction of every item is not likely to be possible or sensible in many operational scenarios. However, the XCCDF format enables granular selection and adjustment of settings, and their association with OVAL and OCIL content provides an automated checking capability. Transformations of this document, and its associated automated checking content, are capable of providing baselines that meet a diverse set of policy objectives. Some example XCCDF Profiles, which are selections of items that form checklists and can be used as baselines, are available with this guide. They can be processed, in an automated fashion, with tools that support the Security Content Automation Protocol (SCAP). The NIST National Checklist Program (NCP), which provides required settings for the United States Government, is one example of a baseline created from this guidance.
Evaluation Characteristics
| Evaluation target | ocp4-cis-api-checks-pod |
|---|---|
| Benchmark URL | #scap_org.open-scap_comp_ssg-ocp4-xccdf.xml |
| Benchmark ID | xccdf_org.ssgproject.content_benchmark_OCP-4 |
| Benchmark version | 0.1.75 |
| Profile ID | xccdf_org.ssgproject.content_profile_cis |
| Started at | 2025-03-12T03:09:53+00:00 |
| Finished at | 2025-03-12T03:09:55+00:00 |
| Performed by | 1000970000 |
| Test system | cpe:/a:redhat:openscap:1.3.10 |
CPE Platforms
- cpe:/a:redhat:openshift_container_platform_on_ovn:4
- cpe:/a:redhat:openshift_container_platform:4.16
- cpe:/a:redhat:openshift_container_platform:4.1
- cpe:/a:redhat:openshift_container_platform_node_on_ovn:4
- cpe:/a:redhat:openshift_container_platform_node_on_sdn:4
- cpe:/o:redhat:openshift_container_platform_node:4
- cpe:/a:redhat:openshift_container_platform_on_aws:4
- cpe:/a:redhat:openshift_container_platform_on_azure:4
- cpe:/a:redhat:openshift_container_platform_on_gcp:4
- cpe:/a:redhat:openshift_container_platform_on_sdn:4
- cpe:/a:redhat:openshift_container_platform:4.10
- cpe:/a:redhat:openshift_container_platform:4.11
- cpe:/a:redhat:openshift_container_platform:4.12
- cpe:/a:redhat:openshift_container_platform:4.13
- cpe:/a:redhat:openshift_container_platform:4.14
- cpe:/a:redhat:openshift_container_platform:4.15
- cpe:/a:redhat:openshift_container_platform:4.17
- cpe:/a:redhat:openshift_container_platform:4.18
- cpe:/a:redhat:openshift_container_platform:4.6
- cpe:/a:redhat:openshift_container_platform:4.7
- cpe:/a:redhat:openshift_container_platform:4.8
- cpe:/a:redhat:openshift_container_platform:4.9
Addresses
- IPv4 127.0.0.1
- IPv4 10.132.0.152
- IPv6 0:0:0:0:0:0:0:1
- IPv6 fe80:0:0:0:858:aff:fe84:98
- MAC 00:00:00:00:00:00
- MAC 0A:58:0A:84:00:98
Compliance and Scoring
Rule results
Severity of failed rules
Score
| Scoring system | Score | Maximum | Percent |
|---|---|---|---|
| urn:xccdf:scoring:default | 70.714287 | 100.000000 |
Rule Overview
Result Details
Restrict Automounting of Service Account Tokens
| Rule ID | xccdf_org.ssgproject.content_rule_accounts_restrict_service_account_tokens | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:53+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Service accounts tokens should not be mounted in pods except where the workload
running in the pod explicitly needs to communicate with the API server.
To ensure pods do not automatically mount tokens, set
automountServiceAccountToken to false. | ||||||||||||||
| Rationale | Mounting service account tokens inside pods can provide an avenue
for privilege escalation attacks where an attacker is able to
compromise a single pod in the cluster. | ||||||||||||||
Ensure Usage of Unique Service Accounts
| Rule ID | xccdf_org.ssgproject.content_rule_accounts_unique_service_account | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:53+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Using the default service account prevents accurate application
rights review and audit tracing. Instead of default, create
a new and unique service account with the following command:
$ oc create sa service_account_name
where service_account_name is the name of a service account
that is needed in the project namespace. | ||||||||||||||
| Rationale | Kubernetes provides a default service account which is used by
cluster workloads where no specific service account is assigned to the pod.
Where access to the Kubernetes API from a pod is required, a specific service account
should be created for that pod, and rights granted to that service account.
This increases auditability of service account rights and access making it
easier and more accurate to trace potential malicious behaviors to a specific
service account and project. | ||||||||||||||
Disable the AlwaysAdmit Admission Control Plugin
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_alwaysadmit | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_alwaysadmit:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:53+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84148-6 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure OpenShift only responses to requests explicitly allowed by the
admission control plugin. Check that the config ConfigMap object does not
contain the AlwaysAdmit plugin. | ||||||||||||
| Rationale | Enabling the admission control plugin AlwaysAdmit allows all
requests and does not provide any filtering. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_alwaysadmit:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_admission_control_plugin_alwaysadmit:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments["enable-admission-plugins"][:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145'). oval:ssg-test_file_for_api_server_admission_control_plugin_alwaysadmit:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145 | regular | 1000970000 | 1000970000 | 9736 | rw------- |
Ensure that the Admission Control Plugin AlwaysPullImages is not set
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_alwayspullimages | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_alwayspullimages:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:53+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| References: |
| ||||||||||||
| Description | The AlwaysPullImages admission control plugin should be disabled,
since it can introduce new failure modes for control plane components if an
image registry is unreachable. | ||||||||||||
| Rationale | Setting admission control policy to AlwaysPullImages forces every new pod
to pull the required images every time. In a multi-tenant cluster users can
be assured that their private images can only be used by those who have the
credentials to pull them. Without this admission control policy, once an
image has been pulled to a node, any pod from any user can use it simply by
knowing the image’s name, without any authorization check against the image
ownership. When this plug-in is enabled, images are always pulled prior to
starting containers, which means valid credentials are required.
However, turning on this admission plugin can introduce new kinds of
cluster failure modes. OpenShift 4 master and infrastructure components are
deployed as pods. Enabling this feature can result in cases where loss of
contact to an image registry can cause a redeployed infrastructure pod
(oauth-server for example) to fail on an image pull for an image that is
currently present on the node. We use PullIfNotPresent so that a loss of
image registry access does not prevent the pod from starting. If it
becomes PullAlways, then an image registry access outage can cause key
infrastructure components to fail.
The pull policy can be managed per container, using
imagePullPolicy. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_alwayspullimages:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_admission_control_plugin_alwayspullimages:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments["enable-admission-plugins"][:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145'). oval:ssg-test_file_for_api_server_admission_control_plugin_alwayspullimages:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145 | regular | 1000970000 | 1000970000 | 9736 | rw------- |
Enable the NamespaceLifecycle Admission Control Plugin
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_namespacelifecycle | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_namespacelifecycle:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83854-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | OpenShift enables the NamespaceLifecycle plugin by default. | ||||||||||||
| Rationale | Setting admission control policy to NamespaceLifecycle ensures that
objects cannot be created in non-existent namespaces, and that namespaces
undergoing termination are not used for creating new objects. This
is recommended to enforce the integrity of the namespace termination process
and also for the availability of new objects. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_namespacelifecycle:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["enable-admission-plugins"][:] | CertificateApproval CertificateSigning CertificateSubjectRestriction DefaultIngressClass DefaultStorageClass DefaultTolerationSeconds LimitRanger MutatingAdmissionWebhook NamespaceLifecycle NodeRestriction OwnerReferencesPermissionEnforcement PersistentVolumeClaimResize PersistentVolumeLabel PodNodeSelector PodTolerationRestriction Priority ResourceQuota RuntimeClass ServiceAccount StorageObjectInUseProtection TaintNodesByCondition ValidatingAdmissionWebhook ValidatingAdmissionPolicy authorization.openshift.io/RestrictSubjectBindings authorization.openshift.io/ValidateRoleBindingRestriction config.openshift.io/DenyDeleteClusterConfiguration config.openshift.io/ValidateAPIServer config.openshift.io/ValidateAuthentication config.openshift.io/ValidateConsole config.openshift.io/ValidateFeatureGate config.openshift.io/ValidateImage config.openshift.io/ValidateOAuth config.openshift.io/ValidateProject config.openshift.io/ValidateScheduler image.openshift.io/ImagePolicy network.openshift.io/ExternalIPRanger network.openshift.io/RestrictedEndpointsAdmission quota.openshift.io/ClusterResourceQuota quota.openshift.io/ValidateClusterResourceQuota route.openshift.io/IngressAdmission scheduling.openshift.io/OriginPodNodeEnvironment security.openshift.io/DefaultSecurityContextConstraints security.openshift.io/SCCExecRestrictions security.openshift.io/SecurityContextConstraint security.openshift.io/ValidateSecurityContextConstraints storage.openshift.io/CSIInlineVolumeSecurity |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_admission_control_plugin_namespacelifecycle:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Enable the NodeRestriction Admission Control Plugin
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_noderestriction | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_noderestriction:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83753-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | To limit the Node and Pod objects that a kubelet could modify,
ensure that the NodeRestriction plugin on kubelets is enabled in
the api-server configuration by running the following command:
$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"' | ||||||||||||
| Rationale | Using the NodeRestriction plugin ensures that the kubelet is
restricted to the Node and Pod objects that it could
modify as defined. Such kubelets will only be allowed to modify their
own Node API object, and only modify Pod API objects
that are bound to their node. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_noderestriction:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["enable-admission-plugins"][:] | CertificateApproval CertificateSigning CertificateSubjectRestriction DefaultIngressClass DefaultStorageClass DefaultTolerationSeconds LimitRanger MutatingAdmissionWebhook NamespaceLifecycle NodeRestriction OwnerReferencesPermissionEnforcement PersistentVolumeClaimResize PersistentVolumeLabel PodNodeSelector PodTolerationRestriction Priority ResourceQuota RuntimeClass ServiceAccount StorageObjectInUseProtection TaintNodesByCondition ValidatingAdmissionWebhook ValidatingAdmissionPolicy authorization.openshift.io/RestrictSubjectBindings authorization.openshift.io/ValidateRoleBindingRestriction config.openshift.io/DenyDeleteClusterConfiguration config.openshift.io/ValidateAPIServer config.openshift.io/ValidateAuthentication config.openshift.io/ValidateConsole config.openshift.io/ValidateFeatureGate config.openshift.io/ValidateImage config.openshift.io/ValidateOAuth config.openshift.io/ValidateProject config.openshift.io/ValidateScheduler image.openshift.io/ImagePolicy network.openshift.io/ExternalIPRanger network.openshift.io/RestrictedEndpointsAdmission quota.openshift.io/ClusterResourceQuota quota.openshift.io/ValidateClusterResourceQuota route.openshift.io/IngressAdmission scheduling.openshift.io/OriginPodNodeEnvironment security.openshift.io/DefaultSecurityContextConstraints security.openshift.io/SCCExecRestrictions security.openshift.io/SecurityContextConstraint security.openshift.io/ValidateSecurityContextConstraints storage.openshift.io/CSIInlineVolumeSecurity |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_admission_control_plugin_noderestriction:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Enable the SecurityContextConstraint Admission Control Plugin
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_scc | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_scc:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83602-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure pod permissions are managed, make sure that the
SecurityContextConstraint admission control plugin is used. | ||||||||||||
| Rationale | A Security Context Constraint is a cluster-level resource that controls the actions
which a pod can perform and what the pod may access. The
SecurityContextConstraint objects define a set of conditions that a pod
must run with in order to be accepted into the system. Security Context Constraints
are comprised of settings and strategies that control the security features
a pod has access to and hence this must be used to control pod access
permissions. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_scc:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["enable-admission-plugins"][:] | CertificateApproval CertificateSigning CertificateSubjectRestriction DefaultIngressClass DefaultStorageClass DefaultTolerationSeconds LimitRanger MutatingAdmissionWebhook NamespaceLifecycle NodeRestriction OwnerReferencesPermissionEnforcement PersistentVolumeClaimResize PersistentVolumeLabel PodNodeSelector PodTolerationRestriction Priority ResourceQuota RuntimeClass ServiceAccount StorageObjectInUseProtection TaintNodesByCondition ValidatingAdmissionWebhook ValidatingAdmissionPolicy authorization.openshift.io/RestrictSubjectBindings authorization.openshift.io/ValidateRoleBindingRestriction config.openshift.io/DenyDeleteClusterConfiguration config.openshift.io/ValidateAPIServer config.openshift.io/ValidateAuthentication config.openshift.io/ValidateConsole config.openshift.io/ValidateFeatureGate config.openshift.io/ValidateImage config.openshift.io/ValidateOAuth config.openshift.io/ValidateProject config.openshift.io/ValidateScheduler image.openshift.io/ImagePolicy network.openshift.io/ExternalIPRanger network.openshift.io/RestrictedEndpointsAdmission quota.openshift.io/ClusterResourceQuota quota.openshift.io/ValidateClusterResourceQuota route.openshift.io/IngressAdmission scheduling.openshift.io/OriginPodNodeEnvironment security.openshift.io/DefaultSecurityContextConstraints security.openshift.io/SCCExecRestrictions security.openshift.io/SecurityContextConstraint security.openshift.io/ValidateSecurityContextConstraints storage.openshift.io/CSIInlineVolumeSecurity |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_admission_control_plugin_scc:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Enable the ServiceAccount Admission Control Plugin
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_admission_control_plugin_service_account | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_admission_control_plugin_service_account:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83791-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure ServiceAccount objects must be created and granted
before pod creation is allowed, follow the documentation and create
ServiceAccount objects as per your environment.
Ensure that the plugin is enabled in the api-server configuration:
$ oc -n openshift-kube-apiserver get configmap config -o json | jq -r '.data."config.yaml"' | jq '.apiServerArguments."enable-admission-plugins"' | ||||||||||||
| Rationale | When a pod is created, if a service account is not specified, the pod
is automatically assigned the default service account in the same
namespace. OpenShift operators should create unique service accounts
and let the API Server manage its security tokens. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_admission_control_plugin_service_account:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["enable-admission-plugins"][:] | CertificateApproval CertificateSigning CertificateSubjectRestriction DefaultIngressClass DefaultStorageClass DefaultTolerationSeconds LimitRanger MutatingAdmissionWebhook NamespaceLifecycle NodeRestriction OwnerReferencesPermissionEnforcement PersistentVolumeClaimResize PersistentVolumeLabel PodNodeSelector PodTolerationRestriction Priority ResourceQuota RuntimeClass ServiceAccount StorageObjectInUseProtection TaintNodesByCondition ValidatingAdmissionWebhook ValidatingAdmissionPolicy authorization.openshift.io/RestrictSubjectBindings authorization.openshift.io/ValidateRoleBindingRestriction config.openshift.io/DenyDeleteClusterConfiguration config.openshift.io/ValidateAPIServer config.openshift.io/ValidateAuthentication config.openshift.io/ValidateConsole config.openshift.io/ValidateFeatureGate config.openshift.io/ValidateImage config.openshift.io/ValidateOAuth config.openshift.io/ValidateProject config.openshift.io/ValidateScheduler image.openshift.io/ImagePolicy network.openshift.io/ExternalIPRanger network.openshift.io/RestrictedEndpointsAdmission quota.openshift.io/ClusterResourceQuota quota.openshift.io/ValidateClusterResourceQuota route.openshift.io/IngressAdmission scheduling.openshift.io/OriginPodNodeEnvironment security.openshift.io/DefaultSecurityContextConstraints security.openshift.io/SCCExecRestrictions security.openshift.io/SecurityContextConstraint security.openshift.io/ValidateSecurityContextConstraints storage.openshift.io/CSIInlineVolumeSecurity |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_admission_control_plugin_service_account:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Ensure that anonymous requests to the API Server are authorized
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_anonymous_auth | ||||||||||||||
| Result | pass | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-api_server_anonymous_auth:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | By default, anonymous access to the OpenShift API is enabled, but at
the same time, all requests must be authorized. If no authentication
mechanism is used, the request is assigned the system:anonymous
virtual user and the system:unauthenticated virtual group.
This allows the authorization layer to determine which requests, if any,
is an anonymous user authorized to make.
To verify the authorization rules for anonymous requests run the following:
$ oc describe clusterrolebindingsand inspect the bindings of the system:anonymous
virtual user and the system:unauthenticated virtual group.
To test that an anonymous request is authorized to access the readyz
endpoint, run:
$ oc get --as="system:anonymous" --raw='/readyz?verbose'In contrast, a request to list all projects should not be authorized: $ oc get --as="system:anonymous" projects | ||||||||||||||
| Rationale | When enabled, requests that are not rejected by other configured
authentication methods are treated as anonymous requests. These requests
are then served by the API server. If you are using RBAC authorization,
it is generally considered reasonable to allow anonymous access to the
API Server for health checks and discovery purposes, and hence this
recommendation is not scored. However, you should consider whether
anonymous discovery is an acceptable risk for your purposes. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/rbac.authorization.k8s.io/v1/clusterrolebindings API endpoint to the local /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterrolebindings file. |
In the file '/apis/rbac.authorization.k8s.io/v1/clusterrolebindings' find only one object at path '.items[:]['subjects'][:].name'. oval:ssg-test_api_server_anonymous_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value | Value |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterrolebindings | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1 | clusterrolebindings | .items[:]['subjects'][:].name | alertmanager-main | argo | assisted-installer-controller | backstage-read-only-sa | system:authenticated | cloud-controller-manager | cloud-credential-operator | cloud-node-manager | system:masters | admin | system:cluster-admins system:admin | cluster-autoscaler | cluster-autoscaler-operator | cluster-baremetal-operator | cluster-monitoring-operator | cluster-network-operator | cluster-node-tuning-operator | tuned | system:cluster-readers | cluster-samples-operator | cluster-samples-operator | cluster-samples-operator | system:authenticated | cluster-storage-operator | default | containergroup-service-account | codeflare-operator-controller-manager | resultscollector | remediation-aggregator | profileparser | rerunner | resultserver | compliance-operator | api-resource-collector | api-resource-collector | compliance-operator | remediation-aggregator | console | console | system:authenticated | console-operator | console-operator | control-plane-machine-set-operator | csi-snapshot-controller-operator | csi-snapshot-controller-operator | csi-snapshot-webhook | machine-config-operator | data-science-pipelines-operator-controller-manager | data-science-pipelines-operator-controller-manager | cluster-image-registry-operator | prometheus-k8s | ds-pipeline-ui-dspa ds-pipeline-dspa ds-pipeline-metadata-envoy-dspa | controller-manager | system:authenticated | operator | operator | gather | gather | istio-operator | istiod-data-science-smcp | klusterlet | knative-openshift | knative-operator | activator | controller | controller | kserve-localmodel-controller-manager | kserve-controller-manager | kserve-controller-manager | kube-apiserver system:kube-apiserver | kube-state-metrics | kubeflow-training-operator | kuberay-operator | rhods-admins odh-admins dedicated-admins | system:authenticated | kueue-controller-manager | kueue-controller-manager | L1Support | L2Support | L3Support | llama-factory-test-oauth-proxy-c80326bb | lvms-operator | vg-manager | lvms-operator | machine-api-controllers | machine-api-controllers | machine-api-operator | machine-api-controllers | machine-config-controller | machine-config-daemon | machine-config-server | machine-os-builder | machine-os-builder | marketplace-operator | metrics-daemon-sa | metrics-server | multus-ac | multus-ancillary-tools | system:cluster-readers | system:multus | multus | multus-ancillary-tools | network-diagnostics | network-node-identity | node-exporter | notebook-controller-service-account | governance-policy-framework-sa | odh-model-controller | odh-notebook-controller-manager | olm-operator-serviceaccount | application-manager | cert-policy-controller-sa | config-policy-controller-sa | governance-policy-framework-sa | klusterlet-addon-search | klusterlet-addon-workmgr | klusterlet-addon-workmgr-log | klusterlet-work-sa | klusterlet-work-sa | klusterlet-work-sa | klusterlet-work-sa | klusterlet-work-sa | managed-serviceaccount | klusterlet-work-sa | csi-snapshot-controller | dns | dns-operator | pruner | ingress-operator | router | iptables-alerter | ovn-kubernetes-control-plane | system:ovn-nodes | ovn-kubernetes-node | openshift-state-metrics | cloud-controller-manager | ossm-cni | olm-operator-serviceaccount | prometheus-k8s | prometheus-k8s | prometheus-k8s | prometheus-operator | odh-model-controller | prometheus-k8s | activator | controller | prometheus-k8s | rhbk-operator | rhods-dashboard | rhods-dashboard | rhods-dashboard | redhat-ods-operator-controller-manager | redhat-ods-operator-controller-manager | prometheus-k8s | nfs-provisioner | system:authenticated | system:authenticated:oauth | knative-openshift | knative-operator | knative-openshift-ingress | istio-operator | kube-storage-version-migrator-sa | node-bootstrapper | system:nodes | system:authenticated | system:authenticated | system:authenticated | system:authenticated | attachdetach-controller | certificate-controller | clusterrole-aggregation-controller | cronjob-controller | daemon-set-controller | deployment-controller | disruption-controller | endpoint-controller | endpointslice-controller | endpointslicemirroring-controller | ephemeral-volume-controller | expand-controller | generic-garbage-collector | horizontal-pod-autoscaler horizontal-pod-autoscaler | job-controller | legacy-service-account-token-cleaner | namespace-controller | node-controller | persistent-volume-binder | pod-garbage-collector | pv-protection-controller | pvc-protection-controller | replicaset-controller | replication-controller | resourcequota-controller | root-ca-cert-publisher | route-controller | service-account-controller | service-ca-cert-publisher | service-controller | statefulset-controller | ttl-after-finished-controller | ttl-controller | default-rolebindings-controller | system:authenticated | default-rolebindings-controller | default-rolebindings-controller | system:kube-controller-manager | kube-dns | system:kube-scheduler | system:masters | metrics-server | system:monitoring | system:master system:kube-apiserver system:node-admins | system:master system:node-admins | node-bootstrapper | system:kube-proxy | system:nodes | system:authenticated | build-config-change-controller | build-controller | cluster-csr-approver-controller | cluster-quota-reconciliation-controller | default-rolebindings-controller | deployer-controller | deploymentconfig-controller | horizontal-pod-autoscaler | image-import-controller | image-trigger-controller | system:serviceaccount:openshift-kube-apiserver:check-endpoints | system:serviceaccount:openshift-kube-apiserver:check-endpoints | system:serviceaccount:openshift-kube-apiserver:check-endpoints | machine-approver-sa | namespace-security-allocation-controller | origin-namespace-controller | podsecurity-admission-label-syncer-controller | privileged-namespaces-psa-label-syncer | pv-recycler-controller | resourcequota-controller | service-ca | service-ingress-ip-controller | serviceaccount-controller | serviceaccount-pull-secrets-controller | template-instance-controller | template-instance-controller | template-instance-finalizer-controller | template-instance-finalizer-controller | template-service-broker | unidling-controller | system:authenticated | oauth-apiserver-sa | openshift-apiserver-sa | oauth-openshift | openshift-controller-manager-sa | image-trigger-controller | ingress-to-route-controller | openshift-controller-manager-sa build-config-change-controller | route-controller-manager-sa | authentication-operator | cluster-cloud-controller-manager | openshift-kube-scheduler-operator | etcd-backup-sa | etcd-operator | kube-apiserver-operator | localhost-recovery-client | kube-controller-manager-operator | localhost-recovery-client | localhost-recovery-client | openshift-kube-scheduler-sa | kube-storage-version-migrator-operator | openshift-apiserver-operator | openshift-config-operator | openshift-controller-manager-operator | installer-sa | installer-sa | installer-sa | installer-sa | service-ca-operator | system:authenticated system:unauthenticated | system:authenticated | openshift-controller-manager-sa | route-controller-manager-sa | system:authenticated:oauth | system:authenticated system:unauthenticated | system:authenticated | system:nodes | system:serviceaccounts | system:kube-scheduler | system:authenticated | telemeter-client | telemeter-client | thanos-querier | trustyai-service-operator-controller-manager | trustyai-service-operator-controller-manager |
Find the file to be checked ('/apis/rbac.authorization.k8s.io/v1/clusterrolebindings'). oval:ssg-test_file_for_api_server_anonymous_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterrolebindings | regular | 1000970000 | 1000970000 | 315890 | rw------- |
Configure the Kubernetes API Server Maximum Retained Audit Logs
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_audit_log_maxbackup | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_audit_log_maxbackup:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | low | ||||||||||||
| Identifiers: | CCE-83739-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To configure how many rotations of audit logs are retained,
edit the openshift-kube-apiserver configmap
and set the audit-log-maxbackup parameter to
10 or to an organizationally appropriate value:
"apiServerArguments":{
...
"audit-log-maxbackup": [10],
...
| ||||||||||||
| Rationale | OpenShift automatically rotates the log files. Retaining old log files ensures
OpenShift Operators will have sufficient log data available for carrying out
any investigation or correlation. For example, if the audit log size is set to
100 MB and the number of retained log files is set to 10, OpenShift Operators
would have approximately 1 GB of log data to use during analysis. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["audit-log-maxbackup"][:]'. oval:ssg-test_api_server_audit_log_maxbackup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["audit-log-maxbackup"][:] | 10 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_audit_log_maxbackup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure Kubernetes API Server Maximum Audit Log Size
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_audit_log_maxsize | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_audit_log_maxsize:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83607-2 | ||||||||||||
| References: |
| ||||||||||||
| Description | To rotate audit logs upon reaching a maximum size,
edit the openshift-kube-apiserver configmap
and set the audit-log-maxsize parameter to
an appropriate size in MB. For example, to set it to 100 MB:
"apiServerArguments":{
...
"audit-log-maxsize": ["100"],
...
| ||||||||||||
| Rationale | OpenShift automatically rotates log files. Retaining old log files ensures that
OpenShift Operators have sufficient log data available for carrying out any
investigation or correlation. If you have set file size of 100 MB and the number of
old log files to keep as 10, there would be approximately 1 GB of log data
available for use in analysis. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["audit-log-maxsize"][:]'. oval:ssg-test_api_server_audit_log_maxsize:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["audit-log-maxsize"][:] | 200 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_audit_log_maxsize:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the Audit Log Path
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_audit_log_path | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_audit_log_path:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-84020-7 | ||||||||||||
| References: |
| ||||||||||||
| Description | To enable auditing on the Kubernetes API Server, the audit log path must be set.
Edit the openshift-kube-apiserver configmap
and set the audit-log-path to a suitable path and file
where audit logs should be written. For example:
"apiServerArguments":{
...
"audit-log-path":"/var/log/kube-apiserver/audit.log",
...
| ||||||||||||
| Rationale | Auditing of the Kubernetes API Server is not enabled by default. Auditing the API Server
provides a security-relevant chronological set of records documenting the sequence
of activities that have affected the system by users, administrators, or other
system components. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["audit-log-path"][:]'. oval:ssg-test_api_server_audit_log_path:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["audit-log-path"][:] | /var/log/kube-apiserver/audit.log |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_audit_log_path:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
The authorization-mode cannot be AlwaysAllow
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_auth_mode_no_aa | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_auth_mode_no_aa:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84207-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | Do not always authorize all requests. | ||||||||||||
| Rationale | The API Server, can be configured to allow all requests. This mode should not be used on any production cluster. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145' find only one object at path '.apiServerArguments["authorization-mode"][:]'. oval:ssg-test_api_server_auth_mode_no_aa:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_auth_mode_no_aa:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments["authorization-mode"][:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145'). oval:ssg-test_file_for_api_server_auth_mode_no_aa:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145 | regular | 1000970000 | 1000970000 | 9736 | rw------- |
Ensure authorization-mode RBAC is configured
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_auth_mode_rbac | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_auth_mode_rbac:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84102-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure OpenShift restricts different identities to a defined set
of operations they are allowed to perform, check that the API server's
authorization-mode configuration option list contains RBAC. | ||||||||||||
| Rationale | Role Based Access Control (RBAC) allows fine-grained control over the
operations that different entities can perform on different objects in
the cluster. Enabling RBAC is critical in regulating access to an
OpenShift cluster as the RBAC rules specify, given a user, which operations
can be executed over a set of namespaced or cluster-wide resources. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["authorization-mode"][:]'. oval:ssg-test_api_server_auth_mode_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["authorization-mode"][:] | Scope SystemMasters RBAC Node |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_auth_mode_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Disable basic-auth-file for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_basic_auth | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_basic_auth:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83936-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | Basic Authentication should not be used for any reason. If needed, edit API
Edit the openshift-kube-apiserver configmap
and remove the basic-auth-file parameter:
"apiServerArguments":{
...
"basic-auth-file":[
"/path/to/any/file"
],
...
Alternate authentication mechanisms such as tokens and certificates will need to be
used. Username and password for basic authentication will be disabled. | ||||||||||||
| Rationale | Basic authentication uses plaintext credentials for authentication.
Currently the basic authentication credentials last indefinitely, and
the password cannot be changed without restarting the API Server. The
Basic Authentication is currently supported for convenience and is
not intended for production workloads. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments[:]'. oval:ssg-test_api_server_basic_auth:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_basic_auth:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments[:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_basic_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Ensure that the bindAddress is set to a relevant secure port
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_bind_address | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_bind_address:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | low | ||||||||||||
| Identifiers: | CCE-83646-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | The bindAddress is set by default to 0.0.0.0:6443, and listening with TLS enabled. | ||||||||||||
| Rationale | The OpenShift API server is served over HTTPS with authentication and authorization;
the secure API endpoint is bound to 0.0.0.0:6443 by default. In OpenShift, the only
supported way to access the API server pod is through the load balancer and then through
the internal service. The value is set by the bindAddress argument under the servingInfo
parameter. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Variable test to check XCCDF variable oval:ssg-test_api_server_bind_address:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Var ref | Value |
|---|---|---|
| true | oval:ssg-local_variable_api_server_bind_address:var:1 | 0.0.0.0:6443 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_bind_address:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the Client Certificate Authority for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_client_ca | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_client_ca:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84284-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | Certificates must be provided to fully setup TLS client certificate
authentication. To ensure the API Server utilizes its own TLS certificates, the
clientCA must be configured. Verify
that servingInfo has the clientCA configured in
the openshift-kube-apiserver
config configmap
to something similar to:
"apiServerArguments": {
...
"client-ca-file": [
"/etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt"
],
...
| ||||||||||||
| Rationale | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS traffic.
If -clientCA is set, any request presenting a client
certificate signed by one of the authorities in the client-ca-file
is authenticated with an identity corresponding to the CommonName of
the client certificate. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#d56e72c377d8f85e0601a704d4218064a0ea4a2235ceee82d20db6cdafc74608' find only one object at path '[:]'. oval:ssg-test_api_server_client_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#d56e72c377d8f85e0601a704d4218064a0ea4a2235ceee82d20db6cdafc74608 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#d56e72c377d8f85e0601a704d4218064a0ea4a2235ceee82d20db6cdafc74608 | [:] | /etc/kubernetes/static-pod-certs/configmaps/client-ca/ca-bundle.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#d56e72c377d8f85e0601a704d4218064a0ea4a2235ceee82d20db6cdafc74608'). oval:ssg-test_file_for_api_server_client_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#d56e72c377d8f85e0601a704d4218064a0ea4a2235ceee82d20db6cdafc74608 | regular | 1000970000 | 1000970000 | 71 | rw------- |
Configure the Encryption Provider Cipher
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_encryption_provider_cipher | ||||||||||||||
| Result | fail | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-api_server_encryption_provider_cipher:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| Identifiers: | CCE-83585-0 | ||||||||||||||
| References: |
| ||||||||||||||
| Description | When you enable etcd encryption, the following OpenShift API server and Kubernetes API server resources are encrypted:
When you enable etcd encryption, encryption keys are created. These keys are rotated on a weekly basis. You must have these keys in order to restore from an etcd backup.
To ensure the correct cipher, set the encryption type to
spec:
encryption:
type: aescbc
For more information, follow the relevant documentation. | ||||||||||||||
| Rationale | etcd is a highly available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are
sensitive in nature and should be encrypted at rest to avoid any
disclosures. Where etcd encryption is used, it is important to ensure that the
appropriate set of encryption providers is used. Currently, aescbc
and aesgcm are the only types supported by OCP. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/apis/config.openshift.io/v1/apiservers/cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb' find only one object at path '[:]'. oval:ssg-test_api_server_encryption_provider_cipher:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| false | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers | cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb | [:] | (null) |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb'). oval:ssg-test_file_for_api_server_encryption_provider_cipher:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster#a1d4b20a86b76e7e2d634dbeff420b1a80df6800836dad1b552314d1b24a18cb | regular | 1000970000 | 1000970000 | 6 | rw------- |
Configure the etcd Certificate Authority for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_etcd_ca | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_etcd_ca:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84216-1 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure etcd is configured to make use of TLS encryption for client
connections, follow the OpenShift documentation and setup the TLS
connection between the API Server and etcd. Then, verify
that apiServerArguments has the etcd-cafile configured in
the openshift-kube-apiserver
config configmap to something
similar to:
"apiServerArguments": {
...
"etcd-cafile": [
"/etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt"
],
...
| ||||||||||||
| Rationale | etcd is a highly-available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are
sensitive in nature and should be protected by client authentication. This
requires the API Server to identify itself to the etcd server using
a SSL Certificate Authority file. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#33769e7a3c14dd6dc237eb2b13a72140eeadf2ce49578f57bc9e0fd096cf4e9a' find only one object at path '[:]'. oval:ssg-test_api_server_etcd_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#33769e7a3c14dd6dc237eb2b13a72140eeadf2ce49578f57bc9e0fd096cf4e9a | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#33769e7a3c14dd6dc237eb2b13a72140eeadf2ce49578f57bc9e0fd096cf4e9a | [:] | /etc/kubernetes/static-pod-resources/configmaps/etcd-serving-ca/ca-bundle.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#33769e7a3c14dd6dc237eb2b13a72140eeadf2ce49578f57bc9e0fd096cf4e9a'). oval:ssg-test_file_for_api_server_etcd_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#33769e7a3c14dd6dc237eb2b13a72140eeadf2ce49578f57bc9e0fd096cf4e9a | regular | 1000970000 | 1000970000 | 81 | rw------- |
Configure the etcd Certificate for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_etcd_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_etcd_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83876-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure etcd is configured to make use of TLS encryption for client
communications, follow the OpenShift documentation and setup the TLS
connection between the API Server and etcd. Then, verify
that apiServerArguments has the etcd-certfile configured in
the openshift-kube-apiserver configmap to something similar to:
...
"etcd-certfile": [
"/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.crt"
],
...
| ||||||||||||
| Rationale | etcd is a highly-available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are sensitive
in nature and should be protected by client authentication. This requires the
API Server to identify itself to the etcd server using a client certificate
and key. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["etcd-certfile"][:]'. oval:ssg-test_api_server_etcd_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["etcd-certfile"][:] | /etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_etcd_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the etcd Certificate Key for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_etcd_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_etcd_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83546-2 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure etcd is configured to make use of TLS encryption for client
communications, follow the OpenShift documentation and setup the TLS
connection between the API Server and etcd. Then, verify
that apiServerArguments has the etcd-keyfile configured in
the openshift-kube-apiserver configmap to something similar to:
...
"etcd-keyfile": [
"/etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.key"
],
...
| ||||||||||||
| Rationale | etcd is a highly-available key-value store used by OpenShift deployments
for persistent storage of all REST API objects. These objects are sensitive
in nature and should be protected by client authentication. This requires the
API Server to identify itself to the etcd server using a client certificate
and key. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["etcd-keyfile"][:]'. oval:ssg-test_api_server_etcd_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["etcd-keyfile"][:] | /etc/kubernetes/static-pod-resources/secrets/etcd-client/tls.key |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_etcd_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Ensure that the --kubelet-https argument is set to true
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_https_for_kubelet_conn | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_https_for_kubelet_conn:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | The kube-apiserver ensures https to the kubelet by default. The apiserver
flag "--kubelet-https" is deprecated and should be either set to "true" or
omitted from the argument list. | ||||||||||||
| Rationale | Connections from the kube-apiserver to kubelets could potentially carry
sensitive data such as secrets and keys. It is thus important to use
in-transit encryption for any communication between the apiserver and
kubelets. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments[:]'. oval:ssg-test_api_server_https_for_kubelet_conn:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_https_for_kubelet_conn:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments[:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_https_for_kubelet_conn:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Disable Use of the Insecure Bind Address
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_insecure_bind_address | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_insecure_bind_address:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83955-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | OpenShift should not bind to non-loopback insecure addresses.
Edit the openshift-kube-apiserver configmap
and remove the insecure-bind-address if it exists:
"apiServerArguments":{
...
"insecure-bind-address":[
"127.0.0.1"
],
...
| ||||||||||||
| Rationale | If the API Server is bound to an insecure address the installation would
be susceptible to unauthenticated and unencrypted access to the master node(s).
The API Server does not perform authentication checking for insecure
binds and the traffic is generally not encrypted. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#95b5b27bb6ea2b122e810c99c17c2430c4845596942804847dd677557cfed88e' find only one object at path '.apiServerArguments[:]'. oval:ssg-test_api_server_insecure_bind_address:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_insecure_bind_address:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments[:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#95b5b27bb6ea2b122e810c99c17c2430c4845596942804847dd677557cfed88e'). oval:ssg-test_file_for_api_server_insecure_bind_address:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#95b5b27bb6ea2b122e810c99c17c2430c4845596942804847dd677557cfed88e | regular | 1000970000 | 1000970000 | 6981 | rw------- |
Configure the kubelet Certificate File for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_kubelet_client_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_kubelet_client_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-84080-1 | ||||||||||||
| References: |
| ||||||||||||
| Description | To enable certificate based kubelet authentication,
edit the config configmap in the openshift-kube-apiserver
namespace and set the below parameter in the config.yaml key if
it is not already configured:
"apiServerArguments":{
...
"kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}
| ||||||||||||
| Rationale | By default the API Server does not authenticate itself to the kubelet's
HTTPS endpoints. Requests from the API Server are treated anonymously.
Configuring certificate-based kubelet authentication ensures that the
API Server authenticates itself to kubelets when submitting requests. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865' find only one object at path '[:]'. oval:ssg-test_api_server_kubelet_client_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | [:] | /etc/kubernetes/static-pod-certs/secrets/kubelet-client/tls.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865'). oval:ssg-test_file_for_api_server_kubelet_client_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | regular | 1000970000 | 1000970000 | 67 | rw------- |
Configure the kubelet Certificate File for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_kubelet_client_cert_pre_4_9 | ||||||
| Result | notapplicable | ||||||
| Multi-check rule | no | ||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||
| Severity | high | ||||||
| Identifiers: | CCE-85890-2 | ||||||
| References: |
| ||||||
| Description | To enable certificate based kubelet authentication,
edit the config configmap in the openshift-kube-apiserver
namespace and set the below parameter in the config.yaml key if
it is not already configured:
"apiServerArguments":{
...
"kubelet-client-certificate":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.crt",
...
}
Note that this particular rule is only valid for OCP releases up to and including 4.8 | ||||||
| Rationale | By default the API Server does not authenticate itself to the kubelet's
HTTPS endpoints. Requests from the API Server are treated anonymously.
Configuring certificate-based kubelet authentication ensures that the
API Server authenticates itself to kubelets when submitting requests. | ||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. |
Configure the kubelet Certificate Key for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_kubelet_client_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_kubelet_client_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-83591-8 | ||||||||||||
| References: |
| ||||||||||||
| Description | To enable certificate based kubelet authentication,
edit the config configmap in the openshift-kube-apiserver
namespace and set the below parameter in the config.yaml key if
it is not already configured:
"apiServerArguments":{
...
"kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}
| ||||||||||||
| Rationale | By default the API Server does not authenticate itself to the kubelet's
HTTPS endpoints. Requests from the API Server are treated anonymously.
Configuring certificate-based kubelet authentication ensures that the
API Server authenticates itself to kubelets when submitting requests. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69' find only one object at path '[:]'. oval:ssg-test_api_server_kubelet_client_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | [:] | /etc/kubernetes/static-pod-certs/secrets/kubelet-client/tls.key |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69'). oval:ssg-test_file_for_api_server_kubelet_client_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | regular | 1000970000 | 1000970000 | 67 | rw------- |
Configure the kubelet Certificate Key for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_kubelet_client_key_pre_4_9 | ||||||
| Result | notapplicable | ||||||
| Multi-check rule | no | ||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||
| Severity | high | ||||||
| Identifiers: | CCE-90794-9 | ||||||
| References: |
| ||||||
| Description | To enable certificate based kubelet authentication,
edit the config configmap in the openshift-kube-apiserver
namespace and set the below parameter in the config.yaml key if
it is not already configured:
"apiServerArguments":{
...
"kubelet-client-key":"/etc/kubernetes/static-pod-resources/secrets/kubelet-client/tls.key",
...
}
Note that this particular rule is only valid for OCP releases up to and including 4.8 | ||||||
| Rationale | By default the API Server does not authenticate itself to the kubelet's
HTTPS endpoints. Requests from the API Server are treated anonymously.
Configuring certificate-based kubelet authentication ensures that the
API Server authenticates itself to kubelets when submitting requests. | ||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /api/v1/namespaces/openshift-kube-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config file. |
Ensure the openshift-oauth-apiserver service uses TLS
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_oauth_https_serving_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_oauth_https_serving_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | By default, the OpenShift OAuth API Server uses TLS. HTTPS should be
used for connections between openshift-oauth-apiserver and kube-apiserver.
By default, the OpenShift OAuth API Server uses Intermediate profile which
requires a minimum TLS version of 1.2. | ||||||||||||
| Rationale | Connections between the kube-apiserver and the extension
openshift-oauth-apiserver could potentially carry sensitive data such
as secrets and keys. It is important to use in-transit encryption
for any communication between the kube-apiserver and the extension
openshift-apiserver. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path '.spec.tlsSecurityProfile.type'. oval:ssg-test_api_server_tls_security_profile_not_old:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_tls_security_profile_not_old:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.tlsSecurityProfile.type |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_api_server_tls_security_profile_not_old:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path '.spec.tlsSecurityProfile.custom.minTLSVersion'. oval:ssg-test_api_server_tls_security_profile_custom_min_tls_version:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_tls_security_profile_custom_min_tls_version:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.tlsSecurityProfile.custom.minTLSVersion |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_api_server_tls_security_profile_custom_min_tls_version:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
Ensure the openshift-oauth-apiserver service uses TLS
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_openshift_https_serving_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_openshift_https_serving_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | By default, the OpenShift API Server uses TLS. HTTPS should be
used for connections between openshift-apiserver and kube-apiserver.
By default, the OpenShift OAuth API Server uses Intermediate profile which
requires a minimum TLS version of 1.2. | ||||||||||||
| Rationale | Connections between the kube-apiserver and the extension
openshift-apiserver could potentially carry sensitive data such
as secrets and keys. It is important to use in-transit encryption
for any communication between the kube-apiserver and the extension
openshift-apiserver. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path '.spec.tlsSecurityProfile.type'. oval:ssg-test_api_server_tls_security_profile_not_old:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_tls_security_profile_not_old:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.tlsSecurityProfile.type |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_api_server_tls_security_profile_not_old:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path '.spec.tlsSecurityProfile.custom.minTLSVersion'. oval:ssg-test_api_server_tls_security_profile_custom_min_tls_version:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_tls_security_profile_custom_min_tls_version:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.tlsSecurityProfile.custom.minTLSVersion |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_api_server_tls_security_profile_custom_min_tls_version:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
Profiling is protected by RBAC
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_profiling_protected_by_rbac | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_profiling_protected_by_rbac:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84212-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | Ensure that the cluster-debugger cluster role includes the /metrics
resource URL. This demonstrates that profiling is protected by RBAC, with a
specific cluster role to allow access. | ||||||||||||
| Rationale | Profiling allows for the identification of specific performance bottlenecks.
It generates a significant amount of program data that could potentially be
exploited to uncover system and program details.
To ensure the collected data is not exploited, profiling endpoints are secured
via RBAC (see cluster-debugger role). By default, the profiling endpoints are
accessible only by users bound to cluster-admin or cluster-debugger role.
Profiling can not be disabled. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger API endpoint to the local /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger file. |
In the file '/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger' find only one object at path '.rules[0].nonResourceURLs[:]'. oval:ssg-test_api_server_profiling_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles | cluster-debugger | .rules[0].nonResourceURLs[:] | /debug/pprof /debug/pprof/* /metrics |
Find the file to be checked ('/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger'). oval:ssg-test_file_for_api_server_profiling_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | regular | 1000970000 | 1000970000 | 667 | rw------- |
Configure the API Server Minimum Request Timeout
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_request_timeout | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_request_timeout:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | The API server minimum request timeout defines the minimum number of
seconds a handler must keep a request open before timing it out. To set this,
edit the openshift-kube-apiserver configmap and set
min-request-timeout under the apiServerArguments field:
"apiServerArguments":{
...
"min-request-timeout":[
3600
],
...
| ||||||||||||
| Rationale | Setting global request timeout allows extending the API Server request
timeout limit to a duration appropriate to the user's connection speed. By
default, it is set to 1800 seconds which might not be suitable for some
environments. Setting the limit too low may result in excessive timeouts,
and a limit that is too large may exhaust the API Server resources making
it prone to Denial-of-Service attack. It is recommended to set this limit
as appropriate and change the default limit of 1800 seconds only if needed. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Variable test to check XCCDF variable oval:ssg-test_api_server_request_timeout:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Var ref | Value |
|---|---|---|
| true | oval:ssg-local_variable_api_server_request_timeout:var:1 | 3600 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_request_timeout:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Ensure that the service-account-lookup argument is set to true
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_service_account_lookup | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_service_account_lookup:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83370-7 | ||||||||||||
| References: |
| ||||||||||||
| Description | Validate service account before validating token. | ||||||||||||
| Rationale | If service-account-lookup is not enabled, the apiserver
only verifies that the authentication token is valid, and
does not validate that the service account token mentioned
in the request is actually present in etcd. This allows
using a service account token even after the corresponding
service account is deleted. This is an example of time of
check to time of use security issue. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["service-account-lookup"][:]'. oval:ssg-test_api_server_service_account_lookup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["service-account-lookup"][:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_service_account_lookup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the Service Account Public Key for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_service_account_public_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_service_account_public_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83350-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the API Server utilizes its own key pair,
edit the openshift-kube-apiserver configmap
and set the serviceAccountPublicKeyFiles parameter to the public
key file for service accounts:
... "serviceAccountPublicKeyFiles":[ "/etc/kubernetes/static-pod-resources/configmaps/sa-token-signing-certs" ], ... | ||||||||||||
| Rationale | By default if no service-account-key-file is specified
to the apiserver, it uses the private key from the TLS serving
certificate to verify service account tokens. To ensure that the
keys for service account tokens are rotated as needed, a
separate public/private key pair should be used for signing service
account tokens. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.serviceAccountPublicKeyFiles[:]'. oval:ssg-test_api_server_service_account_public_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .serviceAccountPublicKeyFiles[:] | /etc/kubernetes/static-pod-resources/configmaps/sa-token-signing-certs /etc/kubernetes/static-pod-resources/configmaps/bound-sa-token-signing-certs |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_service_account_public_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the Certificate for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_tls_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_tls_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83779-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the API Server utilizes its own TLS certificates, the
tls-cert-file must be configured. Verify
that the apiServerArguments section has the tls-cert-file configured in
the config configmap in the openshift-kube-apiserver namespace
similar to:
"apiServerArguments":{
...
"tls-cert-file": [
"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt"
],
...
}
| ||||||||||||
| Rationale | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS
traffic. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#bca394347bab5b9902f1d1568d4f5d6e5498b01ec27ddf8231443e376b18757d' find only one object at path '[:]'. oval:ssg-test_api_server_tls_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#bca394347bab5b9902f1d1568d4f5d6e5498b01ec27ddf8231443e376b18757d | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#bca394347bab5b9902f1d1568d4f5d6e5498b01ec27ddf8231443e376b18757d | [:] | /etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#bca394347bab5b9902f1d1568d4f5d6e5498b01ec27ddf8231443e376b18757d'). oval:ssg-test_file_for_api_server_tls_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#bca394347bab5b9902f1d1568d4f5d6e5498b01ec27ddf8231443e376b18757d | regular | 1000970000 | 1000970000 | 84 | rw------- |
Use Strong Cryptographic Ciphers on the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_tls_cipher_suites | ||||||||||
| Result | pass | ||||||||||
| Multi-check rule | no | ||||||||||
| OVAL Definition ID | oval:ssg-api_server_tls_cipher_suites:def:1 | ||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||
| Severity | medium | ||||||||||
| References: |
| ||||||||||
| Description | To ensure that the API Server is configured to only use strong
cryptographic ciphers, verify the openshift-kube-apiserver
configmap contains the following set of ciphers, with no additions:
"servingInfo":{
...
"cipherSuites": [
"TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256",
"TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384",
"TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256",
"TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256",
],
...
| ||||||||||
| Rationale | TLS ciphers have had a number of known vulnerabilities and weaknesses,
which can reduce the protection provided. By default, OpenShift supports
a number of TLS ciphersuites including some that have security concerns,
weakening the protection provided. | ||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
warning
Once configured, API Server clients that cannot support modern
cryptographic ciphers will not be able to make connections to the API
server. |
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.servingInfo.cipherSuites[:]'. oval:ssg-test_api_server_tls_cipher_suites:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .servingInfo.cipherSuites[:] | TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256 TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_api_server_tls_cipher_suites:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Configure the Certificate Key for the API Server
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_tls_private_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_tls_private_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84282-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the API Server utilizes its own TLS certificates, the
tls-private-key-file must be configured. Verify
that the apiServerArguments section has the tls-private-key-file configured in
the config configmap in the openshift-kube-apiserver namespace
similar to:
"apiServerArguments":{
...
"tls-private-key-file": [
"/etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key"
],
...
}
| ||||||||||||
| Rationale | API Server communication contains sensitive parameters that should remain
encrypted in transit. Configure the API Server to serve only HTTPS
traffic. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#8c69c1fe6742f70a3a16c09461f57a19ef2a695143301cede2f2f5d307aa3508' find only one object at path '[:]'. oval:ssg-test_api_server_tls_private_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#8c69c1fe6742f70a3a16c09461f57a19ef2a695143301cede2f2f5d307aa3508 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#8c69c1fe6742f70a3a16c09461f57a19ef2a695143301cede2f2f5d307aa3508 | [:] | /etc/kubernetes/static-pod-certs/secrets/service-network-serving-certkey/tls.key |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#8c69c1fe6742f70a3a16c09461f57a19ef2a695143301cede2f2f5d307aa3508'). oval:ssg-test_file_for_api_server_tls_private_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#8c69c1fe6742f70a3a16c09461f57a19ef2a695143301cede2f2f5d307aa3508 | regular | 1000970000 | 1000970000 | 84 | rw------- |
Disable Token-based Authentication
| Rule ID | xccdf_org.ssgproject.content_rule_api_server_token_auth | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-api_server_token_auth:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-83481-2 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure OpenShift does not accept token-based authentication,
follow the OpenShift documentation and configure alternate mechanisms for
authentication. Then, edit the API Server pod specification file
Edit the openshift-kube-apiserver configmap
and remove the token-auth-file parameter:
"apiServerArguments":{
...
"token-auth-file":[
"/path/to/any/file"
],
...
| ||||||||||||
| Rationale | The token-based authentication utilizes static tokens to authenticate
requests to the API Server. The tokens are stored in clear-text in a file
on the API Server, and cannot be revoked or rotated without restarting the
API Server. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145' find only one object at path '.apiServerArguments["enable-admission-plugins"][:]'. oval:ssg-test_api_server_token_auth:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_api_server_token_auth:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments["enable-admission-plugins"][:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145'). oval:ssg-test_file_for_api_server_token_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#ffe65d9fac11909686e59349c6a0111aaf57caa26bd2db3e7dcb1a0a22899145 | regular | 1000970000 | 1000970000 | 9736 | rw------- |
Ensure that Audit Log Forwarding Is Enabled
| Rule ID | xccdf_org.ssgproject.content_rule_audit_log_forwarding_enabled | ||||||||||||||
| Result | fail | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-audit_log_forwarding_enabled:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| Identifiers: | CCE-84076-9 | ||||||||||||||
| References: |
| ||||||||||||||
| Description | OpenShift audit works at the API server level, logging all requests coming to the server.
Audit is on by default and the best practice is to ship audit logs off the cluster for retention.
The cluster-logging-operator is able to do this with the ClusterLogForwardersresource. The forementioned resource can be configured to logs to different third party systems. For more information on this, please reference the official documentation: https://docs.openshift.com/container-platform/4.6/logging/cluster-logging-external.html | ||||||||||||||
| Rationale | Retaining logs ensures the ability to go back in time to investigate or correlate any events.
Offloading audit logs from the cluster ensures that an attacker that has access to the cluster will not be able to
tamper with the logs because of the logs being stored off-site. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance API endpoint to the local /kubernetes-api-resources/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance file. true
|
In the file '/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance' find only one object at path 'spec.pipelines[:].inputRefs[:]'. oval:ssg-test_audit_log_forwarding_enabled:tst:1 false
No items have been found conforming to the following objects:
Object oval:ssg-object_audit_log_forwarding_enabled:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| spec.pipelines[:].inputRefs[:] |
Find the file to be checked ('/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance'). oval:ssg-test_file_for_audit_log_forwarding_enabled:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/logging.openshift.io/v1/namespaces/openshift-logging/clusterlogforwarders/instance | regular | 1000970000 | 1000970000 | 25 | rw------- |
Ensure that Audit Log Webhook Is Configured
| Rule ID | xccdf_org.ssgproject.content_rule_audit_log_forwarding_webhook | ||||||
| Result | notapplicable | ||||||
| Multi-check rule | no | ||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||
| Severity | medium | ||||||
| Identifiers: | CCE-86103-9 | ||||||
| References: |
| ||||||
| Description | Audit is on by default and the best practice is to ship audit logs off an cluster for retention.
HyperShift is able to do this with the a audit webhook, which is configured in the HostedCluster
custom resource. The forementioned resource can be configured to log to different third party systems.
For more information on this, please reference the official documentation:
https://hypershift-docs.netlify.app/reference/api/
| ||||||
| Rationale | Retaining logs ensures the ability to go back in time to investigate or correlate any events.
Offloading audit logs from the cluster ensures that an attacker that has access to the cluster will not be able to
tamper with the logs because of the logs being stored off-site. | ||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Configure An Identity Provider
| Rule ID | xccdf_org.ssgproject.content_rule_idp_is_configured | ||||||||||||||
| Result | pass | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-idp_is_configured:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| Identifiers: | CCE-84088-4 | ||||||||||||||
| References: |
| ||||||||||||||
| Description | For users to interact with OpenShift Container Platform, they must first authenticate to the cluster. The authentication layer identifies the user associated with requests to the OpenShift Container Platform API. The authorization layer then uses information about the requesting user to determine if the request is allowed. Understanding authentication | Authentication | OpenShift Container Platform The OpenShift Container Platform includes a built-in OAuth server for token-based authentication. Developers and administrators obtain OAuth access tokens to authenticate themselves to the API. It is recommended for an administrator to configure OAuth to specify an identity provider after the cluster is installed. User access to the cluster is managed through the identity provider. Understanding identity provider configuration | Authentication | OpenShift Container Platform OpenShift includes built-in role based access control (RBAC) to determine whether a user is allowed to perform a given action within the cluster. Roles can have cluster scope or local (i.e. project) scope. Using RBAC to define and apply permissions | Authentication | OpenShift Container Platform | ||||||||||||||
| Rationale | With any authentication mechanism the ability to revoke credentials if they are compromised or no longer required, is a key control. Kubernetes client certificate authentication does not allow for this due to a lack of support for certificate revocation. OpenShift's built-in OAuth server allows credential revocation by relying on the Identity provider, as well as giving the administrators the ability to revoke any tokens given to a specific user. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/apis/config.openshift.io/v1/oauths/cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e' find only one object at path '.identityProviders[:].type'. oval:ssg-test_idp_is_configured:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/config.openshift.io/v1/oauths/cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e | /kubernetes-api-resources/apis/config.openshift.io/v1/oauths | cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e | .identityProviders[:].type | HTPasswd OpenID |
Find the file to be checked ('/apis/config.openshift.io/v1/oauths/cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e'). oval:ssg-test_file_for_idp_is_configured:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/oauths/cluster#489c53adb0325a207f2120d4dee0ef775dad56dceaa74bafc10bf32c1da46e9e | regular | 1000970000 | 1000970000 | 542 | rw------- |
Ensure Controller insecure port argument is unset
| Rule ID | xccdf_org.ssgproject.content_rule_controller_insecure_port_disabled | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-controller_insecure_port_disabled:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | low | ||||||||||||
| Identifiers: | CCE-83578-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the Controller Manager service is bound to secure loopback
address and a secure port,
set the RotateKubeletServerCertificate option to true
in the openshift-kube-controller-manager configmap on the master
node(s):
"extendedArguments": {
...
"port": ["0"],
...
It is also acceptable for a system to deprecate the insecure port:
"extendedArguments": {
...
...
| ||||||||||||
| Rationale | The Controller Manager API service is used for health and metrics
information and is available without authentication or encryption. As such, it
should only be bound to a localhost interface to minimize the cluster's
attack surface. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#9f09cca56dc1e9f9605eb5a94aed74de554fd209513a9222e4fe9c0ed669aeee' find only one object at path '[:]'. oval:ssg-test_controller_insecure_port_disabled:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#9f09cca56dc1e9f9605eb5a94aed74de554fd209513a9222e4fe9c0ed669aeee | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps | config#9f09cca56dc1e9f9605eb5a94aed74de554fd209513a9222e4fe9c0ed669aeee | [:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#9f09cca56dc1e9f9605eb5a94aed74de554fd209513a9222e4fe9c0ed669aeee'). oval:ssg-test_file_for_controller_insecure_port_disabled:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#9f09cca56dc1e9f9605eb5a94aed74de554fd209513a9222e4fe9c0ed669aeee | regular | 1000970000 | 1000970000 | 6 | rw------- |
Ensure Controller secure-port argument is set
| Rule ID | xccdf_org.ssgproject.content_rule_controller_secure_port | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-controller_secure_port:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | low | ||||||||||||
| Identifiers: | CCE-83861-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the Controller Manager service is bound to secure loopback
address using a secure port,
set the RotateKubeletServerCertificate option to true
in the openshift-kube-controller-manager configmap on the master
node(s):
"extendedArguments": {
...
"secure-port": ["10257"],
...
| ||||||||||||
| Rationale | The Controller Manager API service is used for health and metrics
information and is available without authentication or encryption. As such, it
should only be bound to a localhost interface to minimize the cluster's
attack surface. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#8241ce1009dc5dd166436d0311b60b96aa3a2f591ba43a26e2b9d0bfc9071414' find only one object at path '[:]'. oval:ssg-test_controller_secure_port:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#8241ce1009dc5dd166436d0311b60b96aa3a2f591ba43a26e2b9d0bfc9071414 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps | config#8241ce1009dc5dd166436d0311b60b96aa3a2f591ba43a26e2b9d0bfc9071414 | [:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#8241ce1009dc5dd166436d0311b60b96aa3a2f591ba43a26e2b9d0bfc9071414'). oval:ssg-test_file_for_controller_secure_port:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#8241ce1009dc5dd166436d0311b60b96aa3a2f591ba43a26e2b9d0bfc9071414 | regular | 1000970000 | 1000970000 | 6 | rw------- |
Configure the Service Account Certificate Authority Key for the Controller Manager
| Rule ID | xccdf_org.ssgproject.content_rule_controller_service_account_ca | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-controller_service_account_ca:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84244-3 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the API Server utilizes its own key pair, set the masterCA
parameter to the public key file for service accounts in the openshift-kube-controller-manager configmap on the master
node(s):
"extendedArguments": {
...
"root-ca-file": [
"/etc/kubernetes/static-pod-resources/configmaps/serviceaccount-ca/ca-bundle.crt"
],
...
| ||||||||||||
| Rationale | Service accounts authenticate to the API using tokens signed by a private RSA
key. The authentication layer verifies the signature using a matching public RSA key.
Configuring the certificate authority file ensures that the API server's signing
certificates are validated. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#e27218fb5fb7cd68a9911eb2db6bf715ca959f639e56cb60f90be782ddd7fcf8' find only one object at path '[:]'. oval:ssg-test_controller_service_account_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#e27218fb5fb7cd68a9911eb2db6bf715ca959f639e56cb60f90be782ddd7fcf8 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps | config#e27218fb5fb7cd68a9911eb2db6bf715ca959f639e56cb60f90be782ddd7fcf8 | [:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#e27218fb5fb7cd68a9911eb2db6bf715ca959f639e56cb60f90be782ddd7fcf8'). oval:ssg-test_file_for_controller_service_account_ca:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#e27218fb5fb7cd68a9911eb2db6bf715ca959f639e56cb60f90be782ddd7fcf8 | regular | 1000970000 | 1000970000 | 6 | rw------- |
Configure the Service Account Private Key for the Controller Manager
| Rule ID | xccdf_org.ssgproject.content_rule_controller_service_account_private_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-controller_service_account_private_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83526-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the API Server utilizes its own key pair, set the privateKeyFile
parameter to the public key file for service accounts in the openshift-kube-controller-manager configmap on the master
node(s):
"extendedArguments": {
...
"service-account-private-key-file": [
"/etc/kubernetes/static-pod-resources/secrets/service-account-private-key/service-account.key"
],
...
| ||||||||||||
| Rationale | By default if no private key file is specified to the
API Server, the API Server uses the private key from the TLS serving
certificate to verify service account tokens. To ensure that the keys
for service account tokens could be rotated as needed, a separate
public/private key pair should be used for signing service account
tokens. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#407a17f0f401ae8c92955bc382bc80ee34a9afd51ab787e405bf524d03ebf3c8' find only one object at path '[:]'. oval:ssg-test_controller_service_account_private_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#407a17f0f401ae8c92955bc382bc80ee34a9afd51ab787e405bf524d03ebf3c8 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps | config#407a17f0f401ae8c92955bc382bc80ee34a9afd51ab787e405bf524d03ebf3c8 | [:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#407a17f0f401ae8c92955bc382bc80ee34a9afd51ab787e405bf524d03ebf3c8'). oval:ssg-test_file_for_controller_service_account_private_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#407a17f0f401ae8c92955bc382bc80ee34a9afd51ab787e405bf524d03ebf3c8 | regular | 1000970000 | 1000970000 | 6 | rw------- |
Ensure that use-service-account-credentials is enabled
| Rule ID | xccdf_org.ssgproject.content_rule_controller_use_service_account | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-controller_use_service_account:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84208-8 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure individual service account credentials are used,
set the use-service-account-credentials option to true
in the openshift-kube-controller-manager configmap on the master
node(s):
"extendedArguments": {
...
"use-service-account-credentials": [
"true"
],
...
| ||||||||||||
| Rationale | The controller manager creates a service account per controller in
kube-system namespace, generates an API token and credentials for it,
then builds a dedicated API client with that service account credential
for each controller loop to use. Setting the
use-service-account-credentials to true runs each
control loop within the controller manager using a separate service
account credential. When used in combination with RBAC, this ensures
that the control loops run with the minimum permissions required to
perform their intended tasks. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#be4ff4c2d3e706eb3b2f17921e5163bca81082bd313ff067ef625af9e6cb61ff' find only one object at path '[:]'. oval:ssg-test_controller_use_service_account:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#be4ff4c2d3e706eb3b2f17921e5163bca81082bd313ff067ef625af9e6cb61ff | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps | config#be4ff4c2d3e706eb3b2f17921e5163bca81082bd313ff067ef625af9e6cb61ff | [:] | true |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#be4ff4c2d3e706eb3b2f17921e5163bca81082bd313ff067ef625af9e6cb61ff'). oval:ssg-test_file_for_controller_use_service_account:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-controller-manager/configmaps/config#be4ff4c2d3e706eb3b2f17921e5163bca81082bd313ff067ef625af9e6cb61ff | regular | 1000970000 | 1000970000 | 6 | rw------- |
Disable etcd Self-Signed Certificates
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_auto_tls | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_auto_tls:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84199-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is not using self-signed
certificates, run the following command:
$ oc get cm/etcd-pod -n openshift-etcd -o yamlThe etcd pod configuration contained in the configmap should not contain the --auto-tls=true flag. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. Using self-signed
certificates ensures that the certificates are never validated
against a certificate authority and could lead to compromised
and invalidated data. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_auto_tls:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_auto_tls:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Ensure That The etcd Client Certificate Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_cert_file | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_cert_file:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83553-8 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to clients,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
--cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.crt. Note that the [a-z]+is being used since the directory might change between OpenShift versions. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_cert_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_cert_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Enable The Client Certificate Authentication
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_client_cert_auth | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_client_cert_auth:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:54+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84077-7 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to clients,
make sure the etcd-pod*
ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-client-cert-auth="the parameter should be set to true. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_client_cert_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_client_cert_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Ensure That The etcd Key File Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_key_file | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_key_file:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83745-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to clients,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-serving-NODE_NAME.key". Note that the [a-z]+is being used since the directory might change between OpenShift versions. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_key_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_key_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Disable etcd Peer Self-Signed Certificates
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_peer_auto_tls | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_peer_auto_tls:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84184-1 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is not using self-signed
certificates, run the following command:
$ oc get cm/etcd-pod -n openshift-etcd -o yamlThe etcd pod configuration contained in the configmap should not contain the --peer-auto-tls=true flag. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. Using self-signed
certificates ensures that the certificates are never validated
against a certificate authority and could lead to compromised
and invalidated data. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_peer_auto_tls:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_peer_auto_tls:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Ensure That The etcd Peer Client Certificate Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_peer_cert_file | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_peer_cert_file:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83847-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to peers,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
--peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.crtNote that the [a-z]+is being used since the directory might change between OpenShift versions. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_peer_cert_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_peer_cert_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Enable The Peer Client Certificate Authentication
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_peer_client_cert_auth | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_peer_client_cert_auth:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83465-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to clients,
make sure the etcd-pod*
ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-client-cert-auth="the parameter should be set to true. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_peer_client_cert_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_peer_client_cert_auth:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Ensure That The etcd Peer Key File Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_etcd_peer_key_file | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-etcd_peer_key_file:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83711-2 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the etcd service is serving TLS to peers,
make sure the etcd-pod* ConfigMaps in the
openshift-etcd namespace contain the following argument
for the etcd binary in the etcd pod:
oc get -nopenshift-etcd cm etcd-pod -oyaml | grep "\-\-peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-[a-z]+/etcd-peer-NODE_NAME.key"Note that the [a-z]+is being used since the directory might change between OpenShift versions. | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027' find only one object at path '[:]'. oval:ssg-test_etcd_peer_key_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps | etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | [:] | apiVersion: v1 kind: Pod metadata: name: etcd namespace: openshift-etcd annotations: kubectl.kubernetes.io/default-container: etcd target.workload.openshift.io/management: '{"effect": "PreferredDuringScheduling"}' labels: app: etcd k8s-app: etcd etcd: "true" revision: "REVISION" spec: initContainers: - name: setup image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh echo -n "Fixing etcd log permissions." mkdir -p /var/log/etcd && chmod 0700 /var/log/etcd securityContext: privileged: true resources: requests: memory: 50Mi cpu: 5m volumeMounts: - mountPath: /var/log/etcd name: log-dir - name: etcd-ensure-env-vars image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail : "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST?not set}" : "${NODE_NODE_ENVVAR_NAME_ETCD_NAME?not set}" : "${NODE_NODE_ENVVAR_NAME_IP?not set}" # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_IP}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_IP}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected node IP to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_IP}" >&2 exit 1 fi # check for ipv4 addresses as well as ipv6 addresses with extra square brackets if [[ "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "${NODE_IP}" && "${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" != "[${NODE_IP}]" ]]; then # echo the error message to stderr echo "Expected etcd url host to be ${NODE_IP} got ${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}" >&2 exit 1 fi resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: NODE_IP valueFrom: fieldRef: fieldPath: status.podIP - name: etcd-resources-copy image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail rm -f $(grep -l '^### Created by cluster-etcd-operator' /usr/local/bin/*) cp -p /etc/kubernetes/static-pod-certs/configmaps/etcd-scripts/*.sh /usr/local/bin resources: requests: memory: 60Mi cpu: 10m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /usr/local/bin name: usr-local-bin containers: # The etcdctl container should always be first. It is intended to be used # to open a remote shell via `oc rsh` that is ready to run `etcdctl`. - name: etcdctl image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - "/bin/bash" - "-c" - "trap TERM INT; sleep infinity & wait" resources: requests: memory: 60Mi cpu: 10m volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" - name: etcd image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail etcdctl member list || true # this has a non-zero return code if the command is non-zero. If you use an export first, it doesn't and you # will succeed when you should fail. ETCD_INITIAL_CLUSTER=$(discover-etcd-initial-cluster \ --cacert=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --cert=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --key=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --endpoints=${ALL_ETCD_ENDPOINTS} \ --data-dir=/var/lib/etcd \ --target-peer-url-host=${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST} \ --target-name=NODE_NAME) export ETCD_INITIAL_CLUSTER # we cannot use the "normal" port conflict initcontainer because when we upgrade, the existing static pod will never yield, # so we do the detection in etcd container itself. echo -n "Waiting for ports 2379, 2380 and 9978 to be released." time while [ -n "$(ss -Htan '( sport = 2379 or sport = 2380 or sport = 9978 )')" ]; do echo -n "." sleep 1 done export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} env | grep ETCD | grep -v NODE set -x # See https://etcd.io/docs/v3.4.0/tuning/ for why we use ionice exec nice -n -19 ionice -c2 -n0 etcd \ --logger=zap \ --log-level=info \ --experimental-initial-corrupt-check=true \ --snapshot-count=10000 \ --initial-advertise-peer-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2380 \ --cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt \ --client-cert-auth=true \ --peer-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --peer-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --peer-trusted-ca-file=/etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --peer-client-cert-auth=true \ --advertise-client-urls=https://${NODE_NODE_ENVVAR_NAME_IP}:2379 \ --listen-client-urls=https://0.0.0.0:2379,unixs://${NODE_NODE_ENVVAR_NAME_IP}:0 \ --listen-peer-urls=https://0.0.0.0:2380 \ --metrics=extensive \ --listen-metrics-urls=https://0.0.0.0:9978 || mv /etc/kubernetes/etcd-backup-dir/etcd-member.yaml /etc/kubernetes/manifests env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 600Mi cpu: 300m readinessProbe: httpGet: port: 9980 path: readyz scheme: HTTPS timeoutSeconds: 30 failureThreshold: 5 periodSeconds: 5 successThreshold: 1 livenessProbe: httpGet: path: healthz port: 9980 scheme: HTTPS timeoutSeconds: 30 periodSeconds: 5 successThreshold: 1 failureThreshold: 5 startupProbe: httpGet: port: 9980 path: readyz scheme: HTTPS initialDelaySeconds: 10 timeoutSeconds: 1 periodSeconds: 10 successThreshold: 1 failureThreshold: 18 securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/manifests name: static-pod-dir - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-metrics image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail export ETCD_NAME=${NODE_NODE_ENVVAR_NAME_ETCD_NAME} exec nice -n -18 etcd grpc-proxy start \ --endpoints https://${NODE_NODE_ENVVAR_NAME_ETCD_URL_HOST}:9978 \ --metrics-addr https://0.0.0.0:9979 \ --listen-addr 127.0.0.1:9977 \ --advertise-client-url "" \ --key /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key \ --key-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.key \ --cert /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt \ --cert-file /etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-metrics-NODE_NAME.crt \ --cacert /etc/kubernetes/static-pod-certs/configmaps/etcd-peer-client-ca/ca-bundle.crt \ --trusted-ca-file /etc/kubernetes/static-pod-certs/configmaps/etcd-metrics-proxy-serving-ca/ca-bundle.crt \ --listen-cipher-suites TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_AES_128_GCM_SHA256,TLS_AES_256_GCM_SHA384,TLS_CHACHA20_POLY1305_SHA256 env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" - name: "ETCD_STATIC_POD_VERSION" value: "REVISION" resources: requests: memory: 200Mi cpu: 40m securityContext: privileged: true volumeMounts: - mountPath: /etc/kubernetes/static-pod-resources name: resource-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir - mountPath: /var/lib/etcd/ name: data-dir - name: etcd-readyz image: quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:9b36a3036fac079c596c2d476feee5d213ee2c1dc5840489e1f44d1e8d697031 imagePullPolicy: IfNotPresent terminationMessagePolicy: FallbackToLogsOnError command: - /bin/sh - -c - | #!/bin/sh set -euo pipefail exec nice -n -18 cluster-etcd-operator readyz \ --target=https://localhost:2379 \ --listen-port=9980 \ --serving-cert-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.crt \ --serving-key-file=/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-serving-NODE_NAME.key \ --client-cert-file=$(ETCDCTL_CERT) \ --client-key-file=$(ETCDCTL_KEY) \ --client-cacert-file=$(ETCDCTL_CACERT) securityContext: privileged: true ports: - containerPort: 9980 name: readyz protocol: TCP resources: requests: memory: 50Mi cpu: 10m env: - name: "ALL_ETCD_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_API" value: "3" - name: "ETCDCTL_CACERT" value: "/etc/kubernetes/static-pod-certs/configmaps/etcd-serving-ca/ca-bundle.crt" - name: "ETCDCTL_CERT" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.crt" - name: "ETCDCTL_ENDPOINTS" value: "https://192.168.50.23:2379" - name: "ETCDCTL_KEY" value: "/etc/kubernetes/static-pod-certs/secrets/etcd-all-certs/etcd-peer-NODE_NAME.key" - name: "ETCD_CIPHER_SUITES" value: "TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256,TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384,TLS_ECDHE_ECDSA_WITH_CHACHA20_POLY1305_SHA256,TLS_ECDHE_RSA_WITH_CHACHA20_POLY1305_SHA256" - name: "ETCD_DATA_DIR" value: "/var/lib/etcd" - name: "ETCD_ELECTION_TIMEOUT" value: "1000" - name: "ETCD_ENABLE_PPROF" value: "true" - name: "ETCD_EXPERIMENTAL_MAX_LEARNERS" value: "1" - name: "ETCD_EXPERIMENTAL_WARNING_APPLY_DURATION" value: "200ms" - name: "ETCD_EXPERIMENTAL_WATCH_PROGRESS_NOTIFY_INTERVAL" value: "5s" - name: "ETCD_HEARTBEAT_INTERVAL" value: "100" - name: "ETCD_IMAGE" value: "quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b062108a04010ad31450204b00f1a69172deee0e7c27ff241ff078333504925d" - name: "ETCD_INITIAL_CLUSTER_STATE" value: "existing" - name: "ETCD_QUOTA_BACKEND_BYTES" value: "8589934592" - name: "ETCD_SOCKET_REUSE_ADDRESS" value: "true" - name: "NODE_master_01_demo_ETCD_NAME" value: "master-01-demo" - name: "NODE_master_01_demo_ETCD_URL_HOST" value: "192.168.50.23" - name: "NODE_master_01_demo_IP" value: "192.168.50.23" volumeMounts: - mountPath: /var/log/etcd/ name: log-dir - mountPath: /etc/kubernetes/static-pod-certs name: cert-dir hostNetwork: true priorityClassName: system-node-critical tolerations: - operator: "Exists" volumes: - hostPath: path: /etc/kubernetes/manifests name: static-pod-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-pod-REVISION name: resource-dir - hostPath: path: /etc/kubernetes/static-pod-resources/etcd-certs name: cert-dir - hostPath: path: /var/lib/etcd type: "" name: data-dir - hostPath: path: /usr/local/bin name: usr-local-bin - hostPath: path: /var/log/etcd name: log-dir |
Find the file to be checked ('/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027'). oval:ssg-test_file_for_etcd_peer_key_file:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-etcd/configmaps/etcd-pod#72b7530e9fb0f39686f598b00d791485841e98be902ba16431a5629726dd7027 | regular | 1000970000 | 1000970000 | 23207 | rw------- |
Apply Security Context to Your Pods and Containers
| Rule ID | xccdf_org.ssgproject.content_rule_general_apply_scc | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Apply Security Context to your Pods and Containers | ||||||||||||
| Rationale | A security context defines the operating system security settings (uid, gid,
capabilities, SELinux role, etc..) applied to a container. When designing your
containers and pods, make sure that you configure the security context for your
pods, containers, and volumes. A security context is a property defined in the
deployment yaml. It controls the security parameters that will be assigned to
the pod/container/volume. There are two levels of security context: pod level
security context, and container level security context. | ||||||||||||
The default namespace should not be used
| Rule ID | xccdf_org.ssgproject.content_rule_general_default_namespace_use | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Kubernetes provides a default namespace, where objects are placed if no
namespace is specified for them. Placing objects in this namespace makes
application of RBAC and other controls more difficult. | ||||||||||||
| Rationale | Resources in a Kubernetes cluster should be segregated by namespace, to allow
for security controls to be applied at that level and to make it easier to
manage resources. | ||||||||||||
Ensure Seccomp Profile Pod Definitions
| Rule ID | xccdf_org.ssgproject.content_rule_general_default_seccomp_profile | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Enable default seccomp profiles in your pod definitions. | ||||||||||||
| Rationale | Seccomp (secure computing mode) is used to restrict the set of system calls
applications can make, allowing cluster administrators greater control over
the security of workloads running in the cluster. Kubernetes disables
seccomp profiles by default for historical reasons. You should enable it to
ensure that the workloads have restricted actions available within the
container. | ||||||||||||
Create administrative boundaries between resources using namespaces
| Rule ID | xccdf_org.ssgproject.content_rule_general_namespaces_in_use | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Use namespaces to isolate your Kubernetes objects. | ||||||||||||
| Rationale | Limiting the scope of user permissions can reduce the impact of mistakes or
malicious activities. A Kubernetes namespace allows you to partition created
resources into logically named groups. Resources created in one namespace can
be hidden from other namespaces. By default, each resource created by a user
in Kubernetes cluster runs in a default namespace, called default. You can
create additional namespaces and attach resources and users to them. You can
use Kubernetes Authorization plugins to create policies that segregate access
to namespace resources between different users. | ||||||||||||
Ensure that the kubeadmin secret has been removed
| Rule ID | xccdf_org.ssgproject.content_rule_kubeadmin_removed | ||||||||||||||||
| Result | fail | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| OVAL Definition ID | oval:ssg-kubeadmin_removed:def:1 | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| Identifiers: | CCE-90387-2 | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | The kubeadmin user is meant to be a temporary user used for
bootstrapping purposes. It is preferable to assign system
administrators whose users are backed by an Identity Provider.
Make sure to remove the user as described in the documentation | ||||||||||||||||
| Rationale | The kubeadmin user has an auto-generated password
and a self-signed certificate, and has effectively cluster-adminpermissions; therefore, it's considered a security liability. | ||||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /api/v1/namespaces/kube-system/secrets/kubeadmin API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/kube-system/secrets/kubeadmin file. |
tests the presence of '# kube-api-error=NotFound' setting in the /api/v1/namespaces/kube-system/secrets/kubeadmin file oval:ssg-test_kubeadmin_removed:tst:1 false
No items have been found conforming to the following objects:
Object oval:ssg-obj_kubeadmin_removed:obj:1 of type textfilecontent54_object
| Filepath | Pattern | Instance | ||
|---|---|---|---|---|
| # kube-api-error=NotFound | 1 |
Find the file to be checked ('/api/v1/namespaces/kube-system/secrets/kubeadmin'). oval:ssg-test_file_for_kubeadmin_removed:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/kube-system/secrets/kubeadmin | regular | 1000970000 | 1000970000 | 529 | rw------- |
This is a helper rule to fetch the required api resource for detecting HyperShift OCP version
| Rule ID | xccdf_org.ssgproject.content_rule_version_detect_in_hypershift |
| Result | notchecked |
| Multi-check rule | no |
| Time | 2025-03-12T03:09:55+00:00 |
| Severity | medium |
| Description | no description |
| Rationale | no rationale |
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
This is a helper rule to fetch the required api resource for detecting OCP version
| Rule ID | xccdf_org.ssgproject.content_rule_version_detect_in_ocp |
| Result | notchecked |
| Multi-check rule | no |
| Time | 2025-03-12T03:09:55+00:00 |
| Severity | medium |
| Description | no description |
| Rationale | no rationale |
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Ensure That The kubelet Client Certificate Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cert | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-kubelet_configure_tls_cert:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83396-2 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the kubelet TLS client certificate is configured, edit the
kubelet configuration file /etc/kubernetes/kubelet.conf
and configure the kubelet certificate file.
tlsCertFile: /path/to/TLS/cert.key | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865' find only one object at path '[:]'. oval:ssg-test_kubelet_configure_tls_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | [:] | /etc/kubernetes/static-pod-certs/secrets/kubelet-client/tls.crt |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865'). oval:ssg-test_file_for_kubelet_configure_tls_cert:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#e5500055b4aa2fcf00dc09ad0e66e44b6b42d67f8d53d1e72ff81b32f0e09865 | regular | 1000970000 | 1000970000 | 67 | rw------- |
Ensure that the Ingress Controller only makes use of Strong Cryptographic Ciphers
| Rule ID | xccdf_org.ssgproject.content_rule_kubelet_configure_tls_cipher_suites_ingresscontroller | ||||
| Result | fail | ||||
| Multi-check rule | no | ||||
| OVAL Definition ID | oval:ssg-kubelet_configure_tls_cipher_suites_ingresscontroller:def:1 | ||||
| Time | 2025-03-12T03:09:55+00:00 | ||||
| Severity | medium | ||||
| References: |
| ||||
| Description | Ensure that the Ingress Controller is configured to only use strong cryptographic ciphers. | ||||
| Rationale | TLS ciphers have had a number of known vulnerabilities and weaknesses,
which can reduce the protection provided by them. By default Kubernetes
supports a number of TLS ciphersuites including some that have security
concerns, weakening the protection provided. | ||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default API endpoint to the local /kubernetes-api-resources/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default file. |
In the file '/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default' find only one object at path '.status.tlsProfile.ciphers[:]'. oval:ssg-test_kubelet_configure_tls_cipher_suites_ingresscontroller:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| false | /kubernetes-api-resources/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default | /kubernetes-api-resources/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers | default | .status.tlsProfile.ciphers[:] | ECDHE-ECDSA-AES128-GCM-SHA256 ECDHE-RSA-AES128-GCM-SHA256 ECDHE-ECDSA-AES256-GCM-SHA384 ECDHE-RSA-AES256-GCM-SHA384 ECDHE-ECDSA-CHACHA20-POLY1305 ECDHE-RSA-CHACHA20-POLY1305 DHE-RSA-AES128-GCM-SHA256 DHE-RSA-AES256-GCM-SHA384 TLS_AES_128_GCM_SHA256 TLS_AES_256_GCM_SHA384 TLS_CHACHA20_POLY1305_SHA256 |
Find the file to be checked ('/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default'). oval:ssg-test_file_for_kubelet_configure_tls_cipher_suites_ingresscontroller:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/operator.openshift.io/v1/namespaces/openshift-ingress-operator/ingresscontrollers/default | regular | 1000970000 | 1000970000 | 4656 | rw------- |
Ensure That The kubelet Server Key Is Correctly Set
| Rule ID | xccdf_org.ssgproject.content_rule_kubelet_configure_tls_key | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-kubelet_configure_tls_key:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-90614-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the kubelet TLS private server key certificate is configured, edit the
kubelet configuration file /etc/kubernetes/kubelet.conf
and configure the kubelet private key file.
tlsPrivateKeyFile: /path/to/TLS/private.key | ||||||||||||
| Rationale | Without cryptographic integrity protections, information can be
altered by unauthorized users without detection. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69' find only one object at path '[:]'. oval:ssg-test_kubelet_configure_tls_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | [:] | /etc/kubernetes/static-pod-certs/secrets/kubelet-client/tls.key |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69'). oval:ssg-test_file_for_kubelet_configure_tls_key:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#1e2b7c1158e0b9a602cb20d62c82b4660907bb57b63dac11c6c7c64211c49c69 | regular | 1000970000 | 1000970000 | 67 | rw------- |
kubelet - Disable the Read-Only Port
| Rule ID | xccdf_org.ssgproject.content_rule_kubelet_disable_readonly_port | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-kubelet_disable_readonly_port:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83427-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | To disable the read-only port, edit the kubelet configuration
Edit the openshift-kube-apiserver configmap
and set the kubelet-read-only-port parameter to 0:
"apiServerArguments":{
...
"kubelet-read-only-port":[
"0"
],
...
| ||||||||||||
| Rationale | OpenShift disables the read-only port ( 10255) on all nodes by setting the
read-only port kubelet flag to 0. This ensures only
authenticated connections are able to receive information about the OpenShift
system. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments["kubelet-read-only-port"][:]'. oval:ssg-test_kubelet_disable_readonly_port:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps | config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | .apiServerArguments["kubelet-read-only-port"][:] | 0 |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_kubelet_disable_readonly_port:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Ensure that API server audit logging is enabled
| Rule ID | xccdf_org.ssgproject.content_rule_audit_logging_enabled | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-audit_logging_enabled:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-90619-8 | ||||||||||||
| References: |
| ||||||||||||
| Description | OpenShift has the ability to audit API server requests. Audit provides a
security-relevant chronological set of records documenting the sequence of
activities that have affected system by individual users, administrators, or
other components of the system. Audit works at the API server level, logging
all requests coming to the server.
Verify that audit logging is enabled by checking that the API server audit
log configuration is not set to `None`, which explicitly disables the
functionality.
For more information on how to configure the audit profile, please visit
the documentation
| ||||||||||||
| Rationale | Logging is an important detective control for all systems, to detect potential
unauthorised access. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path 'spec.audit.profile'. oval:ssg-test_audit_logging_enabled:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers | cluster | spec.audit.profile | Default |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_audit_logging_enabled:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
Ensure that the cluster's audit profile is properly set
| Rule ID | xccdf_org.ssgproject.content_rule_audit_profile_set | ||||||||||||||
| Result | fail | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-audit_profile_set:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| Identifiers: | CCE-83577-7 | ||||||||||||||
| References: |
| ||||||||||||||
| Description | OpenShift can audit the details of requests made to the API server through the standard Kubernetes audit capabilities. In OpenShift, auditing of the API Server is on by default. Audit provides a security-relevant chronological set of records documenting the sequence of activities that have affected system by individual users, administrators, or other components of the system. Audit works at the API server level, logging all requests coming to the server. Each audit log contains two entries: The request line containing:
The response line containing:
For more information on how to configure the audit profile, please visit the documentation | ||||||||||||||
| Rationale | Logging is an important detective control for all systems, to detect potential
unauthorised access. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/apiservers/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster file. |
In the file '/apis/config.openshift.io/v1/apiservers/cluster' find only one object at path 'spec.audit.profile'. oval:ssg-test_audit_profile_set:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| false | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers | cluster | spec.audit.profile | Default |
Find the file to be checked ('/apis/config.openshift.io/v1/apiservers/cluster'). oval:ssg-test_file_for_audit_profile_set:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/apiservers/cluster | regular | 1000970000 | 1000970000 | 1223 | rw------- |
Ensure that the CNI in use supports Network Policies
| Rule ID | xccdf_org.ssgproject.content_rule_configure_network_policies | ||||||||||||||||
| Result | pass | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| OVAL Definition ID | oval:ssg-configure_network_policies:def:1 | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | high | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | There are a variety of CNI plugins available for Kubernetes. If the CNI in
use does not support Network Policies it may not be possible to effectively
restrict traffic in the cluster. OpenShift supports Kubernetes NetworkPolicy
using a Kubernetes Container Network Interface (CNI) plug-in. | ||||||||||||||||
| Rationale | Kubernetes network policies are enforced by the CNI plugin in use. As such
it is important to ensure that the CNI plugin supports both Ingress and
Egress network policies. | ||||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/apis/operator.openshift.io/v1/networks/cluster#35e33d6dc1252a03495b35bd1751cac70041a511fa4d282c300a8b83b83e3498' find only one object at path '[:]'. oval:ssg-test_configure_network_policies:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/operator.openshift.io/v1/networks/cluster#35e33d6dc1252a03495b35bd1751cac70041a511fa4d282c300a8b83b83e3498 | /kubernetes-api-resources/apis/operator.openshift.io/v1/networks | cluster#35e33d6dc1252a03495b35bd1751cac70041a511fa4d282c300a8b83b83e3498 | [:] | OVNKubernetes |
Find the file to be checked ('/apis/operator.openshift.io/v1/networks/cluster#35e33d6dc1252a03495b35bd1751cac70041a511fa4d282c300a8b83b83e3498'). oval:ssg-test_file_for_configure_network_policies:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/operator.openshift.io/v1/networks/cluster#35e33d6dc1252a03495b35bd1751cac70041a511fa4d282c300a8b83b83e3498 | regular | 1000970000 | 1000970000 | 17 | rw------- |
Ensure that HyperShift Hosted Namespaces have Network Policies defined.
| Rule ID | xccdf_org.ssgproject.content_rule_configure_network_policies_hypershift_hosted | ||||||
| Result | notapplicable | ||||||
| Multi-check rule | no | ||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||
| Severity | high | ||||||
| Identifiers: | CCE-86104-7 | ||||||
| References: |
| ||||||
| Description | Use network policies to isolate traffic in your cluster network. | ||||||
| Rationale | Running different applications on the same Kubernetes cluster creates a risk of one
compromised application attacking a neighboring application. Network segmentation is
important to ensure that containers can communicate only with those they are supposed
to. When a network policy is introduced to a given namespace, all traffic not allowed
by the policy is denied. However, if there are no network policies in a namespace all
traffic will be allowed into and out of the pods in that namespace. | ||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Ensure that application Namespaces have Network Policies defined.
| Rule ID | xccdf_org.ssgproject.content_rule_configure_network_policies_namespaces | ||||||||||||||||
| Result | fail | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| OVAL Definition ID | oval:ssg-configure_network_policies_namespaces:def:1 | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | high | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Use network policies to isolate traffic in your cluster network. | ||||||||||||||||
| Rationale | Running different applications on the same Kubernetes cluster creates a risk of one
compromised application attacking a neighboring application. Network segmentation is
important to ensure that containers can communicate only with those they are supposed
to. When a network policy is introduced to a given namespace, all traffic not allowed
by the policy is denied. However, if there are no network policies in a namespace all
traffic will be allowed into and out of the pods in that namespace. | ||||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
Find the file to be checked ('/apis/networking.k8s.io/v1/networkpolicies#7400bb301fff2f7fc7b1b0fb7448b8e3f15222a8d23f992204315b19eeefa72f'). oval:ssg-test_file_for_configure_network_policies_namespaces:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/networking.k8s.io/v1/networkpolicies#7400bb301fff2f7fc7b1b0fb7448b8e3f15222a8d23f992204315b19eeefa72f | regular | 1000970000 | 1000970000 | 182 | rw------- |
Find the file to be checked ('/api/v1/namespaces#f673748db2dd4e4f0ad55d10ce5e86714c06da02b67ddb392582f71ef81efab2'). oval:ssg-test_file_for_configure_network_policies_filtered_namespaces:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces#f673748db2dd4e4f0ad55d10ce5e86714c06da02b67ddb392582f71ef81efab2 | regular | 1000970000 | 1000970000 | 46079 | rw------- |
Count elements at both paths and compare oval:ssg-test_elements_count_for_configure_network_policies_namespaces:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Var ref | Value |
|---|---|---|
| false | oval:ssg-local_variable_counter_configure_network_policies_namespaces:var:1 | 9 |
Make sure there are no count for network policies in non-ctlplane namespaces oval:ssg-test_configure_network_policies_namespaces:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/networking.k8s.io/v1/networkpolicies#7400bb301fff2f7fc7b1b0fb7448b8e3f15222a8d23f992204315b19eeefa72f | /kubernetes-api-resources/apis/networking.k8s.io/v1 | networkpolicies#7400bb301fff2f7fc7b1b0fb7448b8e3f15222a8d23f992204315b19eeefa72f | [:] | assisted-installer istio-system knative-serving redhat-ods-applications redhat-ods-monitoring redhat-ods-operator rhoai-model-registries rhods-notebooks wzh-ai-01 |
Make sure there are no count for namespaces in non-ctlplane namespaces oval:ssg-test_configure_network_policies_filtered_namespaces:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces#f673748db2dd4e4f0ad55d10ce5e86714c06da02b67ddb392582f71ef81efab2 | /kubernetes-api-resources/api/v1 | namespaces#f673748db2dd4e4f0ad55d10ce5e86714c06da02b67ddb392582f71ef81efab2 | [:].metadata.name | aap-namespace assisted-installer demo-backup demo-keycloak demo-llm demo-rhdh group-sync-operator istio-system knative-eventing knative-serving knative-serving-ingress nfs-system open-cluster-management-agent open-cluster-management-agent-addon policies posim redhat-ods-applications redhat-ods-monitoring redhat-ods-operator retalix rhoai-model-registries rhods-notebooks sno-demo wzh-ai-01 wzh-demo-01 |
Configure the OpenShift API Server Maximum Retained Audit Logs
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_api_server_audit_log_maxbackup | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-ocp_api_server_audit_log_maxbackup:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | low | ||||||||||||
| Identifiers: | CCE-83977-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | To configure how many rotations of audit logs are retained,
edit the openshift-apiserver configmap
and set the audit-log-maxbackup parameter to
10 or to an organizationally appropriate value:
"apiServerArguments":{
...
"audit-log-maxbackup": [10],
...
| ||||||||||||
| Rationale | OpenShift automatically rotates the log files. Retaining old log files ensures
OpenShift Operators will have sufficient log data available for carrying out
any investigation or correlation. For example, if the audit log size is set to
100 MB and the number of retained log files is set to 10, OpenShift Operators
would have approximately 1 GB of log data to use during analysis. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59' find only one object at path '.apiServerArguments["audit-log-maxbackup"][:]'. oval:ssg-test_ocp_api_server_audit_log_maxbackup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps | config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | .apiServerArguments["audit-log-maxbackup"][:] | 10 |
Find the file to be checked ('/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59'). oval:ssg-test_file_for_ocp_api_server_audit_log_maxbackup:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | regular | 1000970000 | 1000970000 | 3105 | rw------- |
Configure OpenShift API Server Maximum Audit Log Size
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_api_server_audit_log_maxsize | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-ocp_api_server_audit_log_maxsize:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-83687-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | To rotate audit logs upon reaching a maximum size,
edit the openshift-apiserver configmap
and set the audit-log-maxsize parameter to
an appropriate size in MB. For example, to set it to 100 MB:
"apiServerArguments":{
...
"audit-log-maxsize": ["100"],
...
| ||||||||||||
| Rationale | OpenShift automatically rotates log files. Retaining old log files ensures that
OpenShift Operators have sufficient log data available for carrying out any
investigation or correlation. If you have set file size of 100 MB and the number of
old log files to keep as 10, there would be approximately 1 GB of log data
available for use in analysis. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59' find only one object at path '.apiServerArguments["audit-log-maxsize"][:]'. oval:ssg-test_ocp_api_server_audit_log_maxsize:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps | config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | .apiServerArguments["audit-log-maxsize"][:] | 100 |
Find the file to be checked ('/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59'). oval:ssg-test_file_for_ocp_api_server_audit_log_maxsize:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps/config#45ae2c88fe28d39a42f19e165a1612353224e9663eb369000e03c7efcd10ef59 | regular | 1000970000 | 1000970000 | 3105 | rw------- |
Configure the Audit Log Path
| Rule ID | xccdf_org.ssgproject.content_rule_openshift_api_server_audit_log_path | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-openshift_api_server_audit_log_path:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-83547-0 | ||||||||||||
| References: |
| ||||||||||||
| Description | To enable auditing on the OpenShift API Server, the audit log path must be set.
Edit the openshift-apiserver configmap
and set the audit-log-path to a suitable path and file
where audit logs should be written. For example:
"apiServerArguments":{
...
"audit-log-path":"/var/log/openshift-apiserver/audit.log",
...
| ||||||||||||
| Rationale | Auditing of the API Server is not enabled by default. Auditing the API Server
provides a security-relevant chronological set of records documenting the sequence
of activities that have affected the system by users, administrators, or other
system components. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /api/v1/namespaces/openshift-apiserver/configmaps/config API endpoint to the local /kubernetes-api-resources/api/v1/namespaces/openshift-apiserver/configmaps/config file. |
In the file '/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430' find only one object at path '.apiServerArguments[:]'. oval:ssg-test_openshift_api_server_audit_log_path:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_openshift_api_server_audit_log_path:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .apiServerArguments[:] |
Find the file to be checked ('/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430'). oval:ssg-test_file_for_openshift_api_server_audit_log_path:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/api/v1/namespaces/openshift-kube-apiserver/configmaps/config#54842ba5cf821644f2727625c1518eba2de6e6b7ae318043d0bf7ccc9570e430 | regular | 1000970000 | 1000970000 | 9734 | rw------- |
Profiling is protected by RBAC
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_debug_role_protects_pprof | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-rbac_debug_role_protects_pprof:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-84182-5 | ||||||||||||
| References: |
| ||||||||||||
| Description | Ensure that the cluster-debugger cluster role includes the /debug/pprof
resource URL. This demonstrates that profiling is protected by RBAC, with a
specific cluster role to allow access. | ||||||||||||
| Rationale | Profiling allows for the identification of specific performance bottlenecks.
It generates a significant amount of program data that could potentially be
exploited to uncover system and program details. If you are not experiencing
any bottlenecks and do not need the profiler for troubleshooting purposes, it
is recommended to turn it off to reduce the potential attack surface. To
ensure the collected data is not exploited, profiling endpoints are secured
via RBAC (see cluster-debugger role). By default, the profiling endpoints are
accessible only by users bound to cluster-admin or cluster-debugger role.
Profiling can not be disabled. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger API endpoint to the local /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger file. |
In the file '/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger' find only one object at path '.rules[0].nonResourceURLs[:]'. oval:ssg-test_rbac_debug_role_protects_pprof:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles | cluster-debugger | .rules[0].nonResourceURLs[:] | /debug/pprof /debug/pprof/* /metrics |
Find the file to be checked ('/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger'). oval:ssg-test_file_for_rbac_debug_role_protects_pprof:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | regular | 1000970000 | 1000970000 | 667 | rw------- |
Ensure that the RBAC setup follows the principle of least privilege
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_least_privilege | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | high | ||||||||||||
| Identifiers: | CCE-90678-4 | ||||||||||||
| References: |
| ||||||||||||
| Description | Role-based access control (RBAC) objects determine whether a user is
allowed to perform a given action within a project.
If users or groups exist that are bound to roles they must not have, modify the user or group permissions using the following cluster and local role binding commands:
Remove a User from a Cluster RBAC role by executing the following:
oc adm policy remove-cluster-role-from-user role username
Remove a Group from a Cluster RBAC role by executing the following:
oc adm policy remove-cluster-role-from-group role groupname
Remove a User from a Local RBAC role by executing the following:
oc adm policy remove-role-from-user role username
Remove a Group from a Local RBAC role by executing the following:
oc adm policy remove-role-from-group role groupname
NOTE: For additional information. https://docs.openshift.com/container-platform/latest/authentication/using-rbac.html | ||||||||||||
| Rationale | Controlling and limiting users access to system services and resources
is key to securing the platform and limiting the intentional or
unintentional comprimising of the system and its services. OpenShift
provides a robust RBAC policy system that allows for authorization
policies to be as detailed as needed. Additionally there are two layers
of RBAC policies, the first is Cluster RBAC policies which administrators
can control who has what access to cluster level services. The other is
Local RBAC policies, which allow project developers/administrators to
control what level of access users have to a given project or namespace. | ||||||||||||
Ensure that the cluster-admin role is only used where required
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_limit_cluster_admin | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | The RBAC role cluster-admin provides wide-ranging powers over the
environment and should be used only where and when needed. | ||||||||||||
| Rationale | Kubernetes provides a set of default roles where RBAC is used. Some of these
roles such as cluster-admin provide wide-ranging privileges which should
only be applied where absolutely necessary. Roles such as cluster-admin
allow super-user access to perform any action on any resource. When used in
a ClusterRoleBinding, it gives full control over every resource in the
cluster and in all namespaces. When used in a RoleBinding, it gives full
control over every resource in the rolebinding's namespace, including the
namespace itself. | ||||||||||||
Limit Access to Kubernetes Secrets
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_limit_secrets_access | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | The Kubernetes API stores secrets, which may be service
account tokens for the Kubernetes API or credentials used
by workloads in the cluster. Access to these secrets should
be restricted to the smallest possible group of users to
reduce the risk of privilege escalation. To restrict users from
secrets, remove get, list, and watch
access to unauthorized users to secret objects in the cluster. | ||||||||||||
| Rationale | Inappropriate access to secrets stored within the Kubernetes
cluster can allow for an attacker to gain additional access to
the Kubernetes cluster or external resources whose credentials
are stored as secrets. | ||||||||||||
Minimize Access to Pod Creation
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_pod_creation_access | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | The ability to create pods in a namespace can provide a
number of opportunities for privilege escalation. Where
applicable, remove create access to pod
objects in the cluster. | ||||||||||||
| Rationale | The ability to create pods in a cluster opens up the cluster
for privilege escalation. | ||||||||||||
Minimize Wildcard Usage in Cluster and Local Roles
| Rule ID | xccdf_org.ssgproject.content_rule_rbac_wildcard_use | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Kubernetes Cluster and Local Roles provide access to resources
based on sets of objects and actions that can be taken on
those objects. It is possible to set either of these using a
wildcard * which matches all items. This violates the
principle of least privilege and leaves a cluster in a more
vulnerable state to privilege abuse. | ||||||||||||||
| Rationale | The principle of least privilege recommends that users are
provided only the access required for their role and nothing
more. The use of wildcard rights grants is likely to provide
excessive rights to the Kubernetes API. | ||||||||||||||
Allowed registries are configured
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_allowed_registries | ||||||||||
| Result | fail | ||||||||||
| Multi-check rule | no | ||||||||||
| OVAL Definition ID | oval:ssg-ocp_allowed_registries:def:1 | ||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||
| Severity | medium | ||||||||||
| References: |
| ||||||||||
| Description | The configuration registrySources.allowedRegistries determines the
permitted registries that the OpenShift container runtime can access for builds
and pods. This configuration setting ensures that all registries other than
those specified are blocked.
You can set the allowed repositories by applying the following manifest using
oc patch, e.g. if you save the following snippet to /tmp/allowed-registries-patch.yaml
spec:
registrySources:
allowedRegistries:
- my-trusted-registry.internal.example.com
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-registries-patch.yaml)" --type=merge | ||||||||||
| Rationale | Allowed registries should be configured to restrict the registries that the
OpenShift container runtime can access, and all other registries should be
blocked. | ||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. |
In the file '/apis/config.openshift.io/v1/images/cluster' find only one object at path '.spec.registrySources.allowedRegistries[:]'. oval:ssg-test_ocp_allowed_registries:tst:1 false
No items have been found conforming to the following objects:
Object oval:ssg-object_ocp_allowed_registries:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.registrySources.allowedRegistries[:] |
Find the file to be checked ('/apis/config.openshift.io/v1/images/cluster'). oval:ssg-test_file_for_ocp_allowed_registries:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster | regular | 1000970000 | 1000970000 | 1035 | rw------- |
Allowed registries for import are configured
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_allowed_registries_for_import | ||||||||||
| Result | fail | ||||||||||
| Multi-check rule | no | ||||||||||
| OVAL Definition ID | oval:ssg-ocp_allowed_registries_for_import:def:1 | ||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||
| Severity | medium | ||||||||||
| References: |
| ||||||||||
| Description | The configuration allowedRegistriesForImport limits the container
image registries from which normal users may import images. This is important
to control, as a user who can stand up a malicious registry can then import
content which claims to include the SHAs of legitimate content layers.
You can set the allowed repositories for import by applying the following
manifest using oc patch, e.g. if you save the following snippet to /tmp/allowed-import-registries-patch.yaml
spec:
allowedRegistriesForImport:
- domainName: my-trusted-registry.internal.example.com
insecure: false
you would call
oc patch image.config.openshift.io cluster --patch="$(cat /tmp/allowed-import-registries-patch.yaml)" --type=merge | ||||||||||
| Rationale | Allowed registries for import should be specified to limit the registries
from which users may import images. | ||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. |
In the file '/apis/config.openshift.io/v1/images/cluster' find only one object at path '.spec.allowedRegistriesForImport[:].domainName'. oval:ssg-test_ocp_allowed_registries_for_import:tst:1 false
No items have been found conforming to the following objects:
Object oval:ssg-object_ocp_allowed_registries_for_import:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.allowedRegistriesForImport[:].domainName |
Find the file to be checked ('/apis/config.openshift.io/v1/images/cluster'). oval:ssg-test_file_for_ocp_allowed_registries_for_import:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster | regular | 1000970000 | 1000970000 | 1035 | rw------- |
Check configured allowed registries for import uses secure protocol
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_insecure_allowed_registries_for_import | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-ocp_insecure_allowed_registries_for_import:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-86235-9 | ||||||||||||
| References: |
| ||||||||||||
| Description | The configuration allowedRegistriesForImport limits the container
image registries from which normal users may import images. This is a list
of the registries that can be trusted to contain valid images and the image
location configured is assumed to be secured unless configured otherwise. It
is important to allow only secure registries to avoid man in the middle attacks,
as the insecure image import request can be impersonated and could lead to
fetching malicious content.
List all the allowed repositories for import configured with insecure set to true
using the following command:
oc get image.config.openshift.io/cluster -o json | jq '.spec | (.allowedRegistriesForImport[])? | select(.insecure==true)'Remove or edit the listed registries having insecure set by using the command: oc edit image.config.openshift.io/clusterFor more information, follow the relevant documentation. | ||||||||||||
| Rationale | Configured list of allowed registries for import should be from the secure source. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. |
In the file '/apis/config.openshift.io/v1/images/cluster' find only one object at path '.spec.allowedRegistriesForImport[:].insecure'. oval:ssg-test_ocp_insecure_allowed_registries_for_import:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_ocp_insecure_allowed_registries_for_import:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.allowedRegistriesForImport[:].insecure |
Find the file to be checked ('/apis/config.openshift.io/v1/images/cluster'). oval:ssg-test_file_for_ocp_insecure_allowed_registries_for_import:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster | regular | 1000970000 | 1000970000 | 1035 | rw------- |
Check if any insecure registry sources is configured
| Rule ID | xccdf_org.ssgproject.content_rule_ocp_insecure_registries | ||||||||||||
| Result | pass | ||||||||||||
| Multi-check rule | no | ||||||||||||
| OVAL Definition ID | oval:ssg-ocp_insecure_registries:def:1 | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| Identifiers: | CCE-86123-7 | ||||||||||||
| References: |
| ||||||||||||
| Description | The configuration registrySources.insecureRegistries determines the
insecure registries that the OpenShift container runtime can access for builds
and pods. This configuration setting is for accessing the configured registries
without TLS validation which could lead to security breaches and should be
avoided.
Remove any insecureRegistries configured using the following command:
oc patch image.config.openshift.io cluster --type=json -p "[{'op': 'remove', 'path': '/spec/registrySources/insecureRegistries'}]"
For more information, follow
the relevant documentation. | ||||||||||||
| Rationale | Insecure registries should not be configured, which would restrict the possibilities of
OpenShift container runtime accessing registries which cannot be validated. | ||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/config.openshift.io/v1/images/cluster API endpoint to the local /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster file. |
In the file '/apis/config.openshift.io/v1/images/cluster' find only one object at path '.spec.registrySources.insecureRegistries[:]'. oval:ssg-test_ocp_insecure_registries:tst:1 true
No items have been found conforming to the following objects:
Object oval:ssg-object_ocp_insecure_registries:obj:1 of type yamlfilecontent_object
| Filepath | Yamlpath | ||
|---|---|---|---|
| .spec.registrySources.insecureRegistries[:] |
Find the file to be checked ('/apis/config.openshift.io/v1/images/cluster'). oval:ssg-test_file_for_ocp_insecure_registries:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/config.openshift.io/v1/images/cluster | regular | 1000970000 | 1000970000 | 1035 | rw------- |
Drop Container Capabilities
| Rule ID | xccdf_org.ssgproject.content_rule_scc_drop_container_capabilities | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Containers should not enable more capabilities than needed as this
opens the door for malicious use. To disable the
capabilities, the appropriate Security Context Constraints (SCCs)
should set all capabilities as * or a list of capabilities in
requiredDropCapabilities. | ||||||||||||||
| Rationale | By default, containers run with a default set of capabilities as assigned
by the Container Runtime which can include dangerous or highly privileged
capabilities. Capabilities should be dropped unless absolutely critical for
the container to run software as added capabilities that are not required
allow for malicious containers or attackers. | ||||||||||||||
Limit Container Capabilities
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_container_allowed_capabilities | ||||||||||||||
| Result | fail | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| OVAL Definition ID | oval:ssg-scc_limit_container_allowed_capabilities:def:1 | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description |
Containers should not enable more capabilites than needed as this
opens the door for malicious use. To enable only the
required capabilities, the appropriate Security Context Constraints (SCCs)
should set capabilities as a list in
In case an SCC outside the default allow list in the variable
apiVersion: compliance.openshift.io/v1alpha1
kind: TailoredProfile
metadata:
name: cis-additional-scc
spec:
description: Allows an additional scc
setValues:
- name: ocp4-var-sccs-with-allowed-capabilities-regex
rationale: Allow our own custom SCC
value: ^privileged$|^hostnetwork-v2$|^restricted-v2$|^nonroot-v2$|^additional$
extends: ocp4-cis
title: Modified CIS allowing one more SCC
Finally, reference this | ||||||||||||||
| Rationale | By default, containers run with a default set of capabilities as assigned
by the Container Runtime which can include dangerous or highly privileged
capabilities. Capabilities should be dropped unless absolutely critical for
the container to run software as added capabilities that are not required
allow for malicious containers or attackers. | ||||||||||||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the following:
|
In the file '/apis/security.openshift.io/v1/securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0' find only one object at path '[:]'. oval:ssg-test_scc_limit_container_allowed_capabilities:tst:1 false
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/security.openshift.io/v1/securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0 | /kubernetes-api-resources/apis/security.openshift.io/v1 | securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0 | [:] | custom-host-access dify-privileged-scc nfs-provisioner |
Find the file to be checked ('/apis/security.openshift.io/v1/securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0'). oval:ssg-test_file_for_scc_limit_container_allowed_capabilities:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/security.openshift.io/v1/securitycontextconstraints#821f2c3e5c4100d5609ac6070d8a51075295651614a05c65a5f744ba751c15b0 | regular | 1000970000 | 1000970000 | 62 | rw------- |
Limit Access to the Host IPC Namespace
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_ipc_namespace | ||||||||||||||||
| Result | notchecked | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| Identifiers: | CCE-84042-1 | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Containers should not be allowed access to the host's Interprocess Communication (IPC)
namespace. To prevent containers from getting access to a host's
IPC namespace, the appropriate Security Context Constraints (SCCs)
should set allowHostIPC to false. | ||||||||||||||||
| Rationale | A container running in the host's IPC namespace can use IPC
to interact with processes outside the container potentially
allowing an attacker to exploit a host process thereby enabling an
attacker to exploit other services. | ||||||||||||||||
Limit Use of the CAP_NET_RAW
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_net_raw_capability | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Containers should not enable more capabilities than needed as this
opens the door for malicious use. CAP_NET_RAW enables a container
to launch a network attack on another container or cluster. To disable the
CAP_NET_RAW capability, the appropriate Security Context Constraints (SCCs)
should set NET_RAW in requiredDropCapabilities. | ||||||||||||||
| Rationale | By default, containers run with a default set of capabilities as assigned
by the Container Runtime which can include dangerous or highly privileged
capabilities. If the CAP_NET_RAW is enabled, it may be misused
by malicious containers or attackers. | ||||||||||||||
Limit Access to the Host Network Namespace
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_network_namespace | ||||||||||||||||
| Result | notchecked | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| Identifiers: | CCE-83492-9 | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Containers should not be allowed access to the host's network
namespace. To prevent containers from getting access to a host's
network namespace, the appropriate Security Context Constraints (SCCs)
should set allowHostNetwork to false. | ||||||||||||||||
| Rationale | A container running in the host's network namespace could
access the host network traffic to and from other pods
potentially allowing an attacker to exploit pods and network
traffic. | ||||||||||||||||
Limit Containers Ability to Escalate Privileges
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_privilege_escalation | ||||||||||||||
| Result | notchecked | ||||||||||||||
| Multi-check rule | no | ||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||
| Severity | medium | ||||||||||||||
| Identifiers: | CCE-83447-3 | ||||||||||||||
| References: |
| ||||||||||||||
| Description | Containers should be limited to only the privileges required
to run and should not be allowed to escalate their privileges.
To prevent containers from escalating privileges,
the appropriate Security Context Constraints (SCCs)
should set allowPrivilegeEscalation to false. | ||||||||||||||
| Rationale | Privileged containers have access to more of the Linux Kernel
capabilities and devices. If a privileged container were
compromised, an attacker would have full access to the container
and host. | ||||||||||||||
Limit Privileged Container Use
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_privileged_containers | ||||||||||||||||
| Result | notchecked | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Containers should be limited to only the privileges required
to run. To prevent containers from running as privileged containers,
the appropriate Security Context Constraints (SCCs) should set
allowPrivilegedContainer to false. | ||||||||||||||||
| Rationale | Privileged containers have access to all Linux Kernel
capabilities and devices. If a privileged container were
compromised, an attacker would have full access to the container
and host. | ||||||||||||||||
Limit Access to the Host Process ID Namespace
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_process_id_namespace | ||||||||||||||||
| Result | notchecked | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Containers should not be allowed access to the host's process
ID namespace. To prevent containers from getting access to a host's
process ID namespace, the appropriate Security Context Constraints (SCCs)
should set allowHostPID to false. | ||||||||||||||||
| Rationale | A container running in the host's PID namespace can inspect
processes running outside the container which can be used to
escalate privileges outside of the container. | ||||||||||||||||
Limit Container Running As Root User
| Rule ID | xccdf_org.ssgproject.content_rule_scc_limit_root_containers | ||||||||||||||||
| Result | notchecked | ||||||||||||||||
| Multi-check rule | no | ||||||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||||||
| Severity | medium | ||||||||||||||||
| References: |
| ||||||||||||||||
| Description | Containers should run as a random non-privileged user.
To prevent containers from running as root user,
the appropriate Security Context Constraints (SCCs) should set
.runAsUser.type to MustRunAsRange. | ||||||||||||||||
| Rationale | It is strongly recommended that containers running on OpenShift
should support running as any arbitrary UID. OpenShift will then assign
a random, non-privileged UID to the running container instance. This
avoids the risk from containers running with specific uids that could
map to host service accounts, or an even greater risk of running as root
level service. OpenShift uses the default security context constraints
(SCC), restricted, to prevent containers from running as root or other
privileged user ids. Pods may be configured to use an scc policy
that allows the container to run as a specific uid, including root(0)
when approved. Only a cluster administrator may grant the change of
an scc policy. | ||||||||||||||||
Verify that the scheduler API service is protected by RBAC
| Rule ID | xccdf_org.ssgproject.content_rule_scheduler_profiling_protected_by_rbac | ||||
| Result | pass | ||||
| Multi-check rule | no | ||||
| OVAL Definition ID | oval:ssg-scheduler_profiling_protected_by_rbac:def:1 | ||||
| Time | 2025-03-12T03:09:55+00:00 | ||||
| Severity | medium | ||||
| References: |
| ||||
| Description | Do not bind the scheduler service to non-loopback insecure addresses. | ||||
| Rationale | The Scheduler API service which runs on port 10251/TCP by default is used for health and metrics information and is available without authentication or encryption. As such it should only be bound to a localhost interface, to minimize the cluster's attack surface | ||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger API endpoint to the local /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger file. |
In the file '/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger' find only one object at path '.rules[0].nonResourceURLs[:]'. oval:ssg-test_scheduler_profiling_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles | cluster-debugger | .rules[0].nonResourceURLs[:] | /debug/pprof /debug/pprof/* /metrics |
Find the file to be checked ('/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger'). oval:ssg-test_file_for_scheduler_profiling_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | regular | 1000970000 | 1000970000 | 667 | rw------- |
Verify that the scheduler API service is protected by RBAC
| Rule ID | xccdf_org.ssgproject.content_rule_scheduler_service_protected_by_rbac | ||||
| Result | pass | ||||
| Multi-check rule | no | ||||
| OVAL Definition ID | oval:ssg-scheduler_service_protected_by_rbac:def:1 | ||||
| Time | 2025-03-12T03:09:55+00:00 | ||||
| Severity | medium | ||||
| References: |
| ||||
| Description | Do not bind the scheduler service to non-loopback insecure addresses. | ||||
| Rationale | The Scheduler API service which runs on port 10251/TCP by default is used for health and metrics information and is available without authentication or encryption. As such it should only be bound to a localhost interface, to minimize the cluster's attack surface | ||||
| Warnings | warning
This rule's check operates on the cluster configuration dump.
Therefore, you need to use a tool that can query the OCP API, retrieve the /apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger API endpoint to the local /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger file. |
In the file '/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger' find only one object at path '.rules[0].nonResourceURLs[:]'. oval:ssg-test_scheduler_service_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Filepath | Path | Filename | Yamlpath | Value |
|---|---|---|---|---|---|
| true | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles | cluster-debugger | .rules[0].nonResourceURLs[:] | /debug/pprof /debug/pprof/* /metrics |
Find the file to be checked ('/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger'). oval:ssg-test_file_for_scheduler_service_protected_by_rbac:tst:1 true
Following items have been found on the system:
| Result of item-state comparison | Path | Type | UID | GID | Size (B) | Permissions |
|---|---|---|---|---|---|---|
| not evaluated | /kubernetes-api-resources/apis/rbac.authorization.k8s.io/v1/clusterroles/cluster-debugger | regular | 1000970000 | 1000970000 | 667 | rw------- |
Consider external secret storage
| Rule ID | xccdf_org.ssgproject.content_rule_secrets_consider_external_storage | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Consider the use of an external secrets storage and management system,
instead of using Kubernetes Secrets directly, if you have more complex
secret management needs. Ensure the solution requires authentication to
access secrets, has auditing of access to and use of secrets, and encrypts
secrets. Some solutions also make it easier to rotate secrets. | ||||||||||||
| Rationale | Kubernetes supports secrets as first-class objects, but care needs to be
taken to ensure that access to secrets is carefully limited. Using an
external secrets provider can ease the management of access to secrets,
especially where secrets are used across both Kubernetes and non-Kubernetes
environments. | ||||||||||||
Do Not Use Environment Variables with Secrets
| Rule ID | xccdf_org.ssgproject.content_rule_secrets_no_environment_variables | ||||||||||||
| Result | notchecked | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | Secrets should be mounted as data volumes instead of environment
variables. | ||||||||||||
| Rationale | Environment variables are subject and very susceptible to
malicious hijacking methods by an adversary, as such,
environment variables should never be used for secrets. | ||||||||||||
Verify Group Who Owns The Worker Proxy Kubeconfig File
| Rule ID | xccdf_org.ssgproject.content_rule_file_groupowner_proxy_kubeconfig | ||||||||||||
| Result | notapplicable | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the Kubernetes ConfigMap is mounted into the sdn daemonset pods with the
correct ownership, make sure that the sdn-config ConfigMap is mounted using
a ConfigMap at the /config mount point and that the sdn container
points to that configuration using the --proxy-config command line option.
Run:
oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'and ensure the --proxy-config parameter points to
/config/kube-proxy-config.yaml and that the config mount point is
mounted from the sdn-config ConfigMap. | ||||||||||||
| Rationale | The kubeconfig file for kube-proxy provides permissions to the kube-proxy service.
The proxy kubeconfig file contains information about the administrative configuration of the
OpenShift cluster that is configured on the system. Protection of this file is
critical for OpenShift security.
The file is provided via a ConfigMap mount, so the kubelet itself makes sure that the
file permissions are appropriate for the container taking it into use. |
Verify User Who Owns The Worker Proxy Kubeconfig File
| Rule ID | xccdf_org.ssgproject.content_rule_file_owner_proxy_kubeconfig | ||||||||||||
| Result | notapplicable | ||||||||||||
| Multi-check rule | no | ||||||||||||
| Time | 2025-03-12T03:09:55+00:00 | ||||||||||||
| Severity | medium | ||||||||||||
| References: |
| ||||||||||||
| Description | To ensure the Kubernetes ConfigMap is mounted into the sdn daemonset pods with the
correct ownership, make sure that the sdn-config ConfigMap is mounted using
a ConfigMap at the /config mount point and that the sdn container
points to that configuration using the --proxy-config command line option.
Run:
oc get -nopenshift-sdn ds sdn -ojson | jq -r '.spec.template.spec.containers[] | select(.name == "sdn")'and ensure the --proxy-config parameter points to
/config/kube-proxy-config.yaml and that the config mount point is
mounted from the sdn-config ConfigMap. | ||||||||||||
| Rationale | The kubeconfig file for kube-proxy provides permissions to the kube-proxy service.
The proxy kubeconfig file contains information about the administrative configuration of the
OpenShift cluster that is configured on the system. Protection of this file is
critical for OpenShift security.
The file is provided via a ConfigMap mount, so the kubelet itself makes sure that the
file permissions are appropriate for the container taking it into use. |