IngressNightmare Patch and Vulnerability Analysis

Jacob Sandum

24 Mar 2025 — 13 min read

Series of CVEs assigned to ingress-nginx

Last Update: 3/24 8:53 PM CST

Post may be a bit rushed

Patches have been released

GitHub Issues for each of the CVEs

CVE-2025-24513: https://github.com/kubernetes/kubernetes/issues/131005
CVE-2025-24514: https://github.com/kubernetes/kubernetes/issues/131006
CVE-2025-1097: https://github.com/kubernetes/kubernetes/issues/131007
CVE-2025-1098: https://github.com/kubernetes/kubernetes/issues/131008
CVE-2025-1974: https://github.com/kubernetes/kubernetes/issues/131009

CVE-2025-24513

"A security issue was discovered in ingress-nginx where attacker-provided data are included in a filename by the ingress-nginx Admission Controller feature, resulting in directory traversal within the container. This could result in denial of service, or when combined with other vulnerabilities, limited disclosure of Secret objects from the cluster." - GitHub advisory

An attacker with access to the Kubernetes API / kubectl can create a malicious ingress-nginx resources. In the resource annotation for setting up Basic HTTP authentication in the ingress resource, there is a LFI vulnerability. ingress-nginx fails to sanitize path traversal characters in the auth-file annotation.

Impact: Limited LFI / DOS?

Patch

https://github.com/kubernetes/ingress-nginx/pull/13068/commits/cbc159094f6d1b1bf8cf1761eb119138d1f95df1

import (
	"fmt"
	"os"
	"path/filepath"
	"regexp"
	"strings"

@@ -203,16 +204,24 @@ func (a auth) Parse(ing *networking.Ingress) (interface{}, error) {
		return nil, err
	}

	passFilename := fmt.Sprintf("%v/%v-%v-%v.passwd", a.authDirectory, ing.GetNamespace(), ing.UID, secret.UID)
	passFileName := fmt.Sprintf("%v-%v-%v.passwd", ing.GetNamespace(), ing.UID, secret.UID)
	passFilePath := filepath.Join(a.authDirectory, passFileName)

	// Ensure password file name does not contain any path traversal characters.
	if a.authDirectory != filepath.Dir(passFilePath) || passFileName != filepath.Base(passFilePath) {
		return nil, ing_errors.LocationDeniedError{
			Reason: fmt.Errorf("invalid password file name: %s", passFileName),
		}
	}

	switch secretType {
	case fileAuth:
		err = dumpSecretAuthFile(passFilename, secret)
		err = dumpSecretAuthFile(passFilePath, secret)
		if err != nil {
			return nil, err
		}
	case mapAuth:
		err = dumpSecretAuthMap(passFilename, secret)
		err = dumpSecretAuthMap(passFilePath, secret)
		if err != nil {
			return nil, err
		}
@@ -225,9 +234,9 @@ func (a auth) Parse(ing *networking.Ingress) (interface{}, error) {
	return &Config{
		Type:       at,
		Realm:      realm,
		File:       passFilename,
		File:       passFilePath,
		Secured:    true,
		FileSHA:    file.SHA1(passFilename),
		FileSHA:    file.SHA1(passFilePath),
		Secret:     name,
		SecretType: secretType,
	}, nil

Observe the above patch is adding file path checks to the

CVE-2025-24514

"A security issue was discovered in ingress-nginx where the `auth-url` Ingress annotation can be used to inject configuration into nginx. This can lead to arbitrary code execution in the context of the ingress-nginx controller, and disclosure of Secrets accessible to the controller. (Note that in the default installation, the controller can access all Secrets cluster-wide.)" - GitHub Advisory

Patch

@@ -1082,7 +1082,7 @@ stream {
            set $target {{ changeHostPort $externalAuth.URL $authUpstreamName }};
            {{ else }}
            proxy_http_version {{ $location.Proxy.ProxyHTTPVersion }};
            set $target {{ $externalAuth.URL }};
            set $target {{ $externalAuth.URL | quote }};
            {{ end }}
            proxy_pass $target;
        }

CVE-2025-1097

"A security issue was discovered in ingress-nginx where the `auth-tls-match-cn` Ingress annotation can be used to inject configuration into nginx. This can lead to arbitrary code execution in the context of the ingress-nginx controller, and disclosure of Secrets accessible to the controller. (Note that in the default installation, the controller can access all Secrets cluster-wide.)" - GitHub Advisory

CVE-2025-1098

"A security issue was discovered in Kubernetes where under certain conditions, an unauthenticated attacker with access to the pod network can achieve arbitrary code execution in the context of the ingress-nginx controller. This can lead to disclosure of Secrets accessible to the controller. (Note that in the default installation, the controller can access all Secrets cluster-wide.)" - GitHub Advisory

CVE-2025-1974

This is the big one and has the highest severity.

An attacker with access to the ingress-nginx admission controller webhook can create a malicious ingress-nginx Ingress configuration that results in RCE.

Patch

Disabled Functionality

ingress-nginx/internal/ingress/controller/controller.go

	/* Deactivated to mitigate CVE-2025-1974
	// TODO: Implement sandboxing so this test can be done safely
	err = n.testTemplate(content)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}
	*/

ingress-nginx/test/e2e/admission/admission.go

	/* Deactivated to mitigate CVE-2025-1974
	// TODO: Implement sandboxing so this test can be done safely
	ginkgo.It("should return an error if there is an error validating the ingress definition", func() {
		disableSnippet := f.AllowSnippetConfiguration()
		defer disableSnippet()

		host := admissionTestHost

		annotations := map[string]string{
			"nginx.ingress.kubernetes.io/configuration-snippet": "something invalid",
		}
		firstIngress := framework.NewSingleIngress("first-ingress", "/", host, f.Namespace, framework.EchoService, 80, annotations)
		_, err := f.KubeClientSet.NetworkingV1().Ingresses(f.Namespace).Create(context.TODO(), firstIngress, metav1.CreateOptions{})
		assert.NotNil(ginkgo.GinkgoT(), err, "creating an ingress with invalid configuration should return an error")
	})
	*/

controller_test.go

		/* Deactivated to mitigate CVE-2025-1974
		// TODO: Implement sandboxing so this test can be done safely
		t.Run("When nginx test returns an error", func(t *testing.T) {
			nginx.command = testNginxTestCommand{
				t:        t,
				err:      fmt.Errorf("test error"),
				out:      []byte("this is the test command output"),
				expected: "_,test.example.com",
			}
			if nginx.CheckIngress(ing) == nil {
				t.Errorf("with a new ingress with an error, an error should be returned")
			}
		})
		*/

Vulnerable Function - CheckIngress

// CheckIngress returns an error in case the provided ingress, when added
// to the current configuration, generates an invalid configuration
func (n *NGINXController) CheckIngress(ing *networking.Ingress) error {
	startCheck := time.Now().UnixNano() / 1000000

	if ing == nil {
		// no ingress to add, no state change
		return nil
	}

	// Skip checks if the ingress is marked as deleted
	if !ing.DeletionTimestamp.IsZero() {
		return nil
	}

	if n.cfg.DeepInspector {
		if err := inspector.DeepInspect(ing); err != nil {
			return fmt.Errorf("invalid object: %w", err)
		}
	}

	// Do not attempt to validate an ingress that's not meant to be controlled by the current instance of the controller.
	if ingressClass, err := n.store.GetIngressClass(ing, n.cfg.IngressClassConfiguration); ingressClass == "" {
		klog.Warningf("ignoring ingress %v in %v based on annotation %v: %v", ing.Name, ing.ObjectMeta.Namespace, ingressClass, err)
		return nil
	}

	if n.cfg.Namespace != "" && ing.ObjectMeta.Namespace != n.cfg.Namespace {
		klog.Warningf("ignoring ingress %v in namespace %v different from the namespace watched %s", ing.Name, ing.ObjectMeta.Namespace, n.cfg.Namespace)
		return nil
	}

	if n.cfg.DisableCatchAll && ing.Spec.DefaultBackend != nil {
		return fmt.Errorf("this deployment is trying to create a catch-all ingress while DisableCatchAll flag is set to true. Remove '.spec.defaultBackend' or set DisableCatchAll flag to false")
	}
	startRender := time.Now().UnixNano() / 1000000
	cfg := n.store.GetBackendConfiguration()
	cfg.Resolver = n.resolver

	// Adds the pathType Validation
	if cfg.StrictValidatePathType {
		if err := inspector.ValidatePathType(ing); err != nil {
			return fmt.Errorf("ingress contains invalid paths: %w", err)
		}
	}

	var arrayBadWords []string

	if cfg.AnnotationValueWordBlocklist != "" {
		arrayBadWords = strings.Split(strings.TrimSpace(cfg.AnnotationValueWordBlocklist), ",")
	}

	for key, value := range ing.ObjectMeta.GetAnnotations() {
		if parser.AnnotationsPrefix != parser.DefaultAnnotationsPrefix {
			if strings.HasPrefix(key, fmt.Sprintf("%s/", parser.DefaultAnnotationsPrefix)) {
				return fmt.Errorf("this deployment has a custom annotation prefix defined. Use '%s' instead of '%s'", parser.AnnotationsPrefix, parser.DefaultAnnotationsPrefix)
			}
		}

		if strings.HasPrefix(key, fmt.Sprintf("%s/", parser.AnnotationsPrefix)) && len(arrayBadWords) != 0 {
			for _, forbiddenvalue := range arrayBadWords {
				if strings.Contains(value, strings.TrimSpace(forbiddenvalue)) {
					return fmt.Errorf("%s annotation contains invalid word %s", key, forbiddenvalue)
				}
			}
		}

		if !cfg.AllowSnippetAnnotations && strings.HasSuffix(key, "-snippet") {
			return fmt.Errorf("%s annotation cannot be used. Snippet directives are disabled by the Ingress administrator", key)
		}
	}

	k8s.SetDefaultNGINXPathType(ing)

	allIngresses := n.store.ListIngresses()

	filter := func(toCheck *ingress.Ingress) bool {
		return toCheck.ObjectMeta.Namespace == ing.ObjectMeta.Namespace &&
			toCheck.ObjectMeta.Name == ing.ObjectMeta.Name
	}
	ings := store.FilterIngresses(allIngresses, filter)
	parsed, err := annotations.NewAnnotationExtractor(n.store).Extract(ing)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}
	ings = append(ings, &ingress.Ingress{
		Ingress:           *ing,
		ParsedAnnotations: parsed,
	})
	startTest := time.Now().UnixNano() / 1000000
	_, servers, pcfg := n.getConfiguration(ings)

	err = checkOverlap(ing, servers)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}
	testedSize := len(ings)
	if n.cfg.DisableFullValidationTest {
		_, _, pcfg = n.getConfiguration(ings[len(ings)-1:])
		testedSize = 1
	}

	content, err := n.generateTemplate(cfg, *pcfg)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}

	err = n.testTemplate(content)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}
	n.metricCollector.IncCheckCount(ing.ObjectMeta.Namespace, ing.Name)
	endCheck := time.Now().UnixNano() / 1000000
	n.metricCollector.SetAdmissionMetrics(
		float64(testedSize),
		float64(endCheck-startTest)/1000,
		float64(len(ings)),
		float64(startTest-startRender)/1000,
		float64(len(content)),
		float64(endCheck-startCheck)/1000,
	)
	return nil
}

ingress-nginx/internal/ingress/controller/controller.go

content, err := n.generateTemplate(cfg, *pcfg)
	if err != nil {
		n.metricCollector.IncCheckErrorCount(ing.ObjectMeta.Namespace, ing.Name)
		return err
	}

err = n.testTemplate(content)

Calls testTemplate

// testTemplate checks if the NGINX configuration inside the byte array is valid
// running the command "nginx -t" using a temporal file.
func (n *NGINXController) testTemplate(cfg []byte) error {
	if len(cfg) == 0 {
		return fmt.Errorf("invalid NGINX configuration (empty)")
	}
	tmpfile, err := os.CreateTemp(filepath.Join(os.TempDir(), "nginx"), tempNginxPattern)
	if err != nil {
		return err
	}
	defer tmpfile.Close()
	err = os.WriteFile(tmpfile.Name(), cfg, file.ReadWriteByUser)
	if err != nil {
		return err
	}
	out, err := n.command.Test(tmpfile.Name())
	if err != nil {
		// this error is different from the rest because it must be clear why nginx is not working
		oe := fmt.Sprintf(`
-------------------------------------------------------------------------------
Error: %v
%v
-------------------------------------------------------------------------------
`, err, string(out))

		return errors.New(oe)
	}

	os.Remove(tmpfile.Name())
	return nil
}

Inputs cfg string (bytes)

func (n *NGINXController) testTemplate(cfg []byte) error {

Writes cfg to tmp dir

	tmpfile, err := os.CreateTemp(filepath.Join(os.TempDir(), "nginx"), tempNginxPattern)
	if err != nil {
		return err
	}
	defer tmpfile.Close()
	err = os.WriteFile(tmpfile.Name(), cfg, file.ReadWriteByUser)
	if err != nil {
		return err
	}

Validates tmp cfg file

out, err := n.command.Test(tmpfile.Name())

If successful, deletes files. Otherwise returns err

os.Remove(tmpfile.Name())

Test function executes nginx passing cfg file with -t.

// Test checks if config file is a syntax valid nginx configuration
func (nc NginxCommand) Test(cfg string) ([]byte, error) {
	//nolint:gosec // Ignore G204 error
	return exec.Command(nc.Binary, "-c", cfg, "-t").CombinedOutput()
}

https://github.com/kubernetes/ingress-nginx/blob/main/internal/ingress/controller/util.go#L144 - Root cause of the RCE

Since the attacker controls the contents of the cfg file, they can achieve unauthenticated RCE.

Vulnerable Admission Controller Server

// ServeHTTP implements http.Server method
func (acs *AdmissionControllerServer) ServeHTTP(w http.ResponseWriter, req *http.Request) {
	defer req.Body.Close()

	data, err := io.ReadAll(req.Body)
	if err != nil {
		klog.ErrorS(err, "Failed to read request body")
		w.WriteHeader(http.StatusBadRequest)
		return
	}

	codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
		Pretty: true,
	})

	obj, _, err := codec.Decode(data, nil, nil)
	if err != nil {
		klog.ErrorS(err, "Failed to decode request body")
		w.WriteHeader(http.StatusBadRequest)
		return
	}

	result, err := acs.AdmissionController.HandleAdmission(obj)
	if err != nil {
		klog.ErrorS(err, "failed to process webhook request")
		w.WriteHeader(http.StatusInternalServerError)
		return
	}

	if err := codec.Encode(result, w); err != nil {
		klog.ErrorS(err, "failed to encode response body")
		w.WriteHeader(http.StatusInternalServerError)
		return
	}
}

Calls HandleAdmission

// HandleAdmission populates the admission Response
// with Allowed=false if the Object is an ingress that would prevent nginx to reload the configuration
// with Allowed=true otherwise
func (ia *IngressAdmission) HandleAdmission(obj runtime.Object) (runtime.Object, error) {
	review, isV1 := obj.(*admissionv1.AdmissionReview)
	if !isV1 {
		return nil, fmt.Errorf("request is not of type AdmissionReview v1 or v1beta1")
	}

	if !apiequality.Semantic.DeepEqual(review.Request.Kind, ingressResource) {
		return nil, fmt.Errorf("rejecting admission review because the request does not contain an Ingress resource but %s with name %s in namespace %s",
			review.Request.Kind.String(), review.Request.Name, review.Request.Namespace)
	}

	status := &admissionv1.AdmissionResponse{}
	status.UID = review.Request.UID

	ingress := networking.Ingress{}

	codec := json.NewSerializerWithOptions(json.DefaultMetaFactory, scheme, scheme, json.SerializerOptions{
		Pretty: true,
	})
	_, _, err := codec.Decode(review.Request.Object.Raw, nil, &ingress)
	if err != nil {
		klog.ErrorS(err, "failed to decode ingress")
		status.Allowed = false
		status.Result = &metav1.Status{
			Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
			Message: err.Error(),
		}

		review.Response = status
		return review, nil
	}

	// Adds the warnings regardless of operation being allowed or not
	warning, err := ia.Checker.CheckWarning(&ingress)
	if err != nil {
		klog.ErrorS(err, "failed to get ingress warnings")
	}
	if len(warning) > 0 {
		status.Warnings = warning
	}

	if err := ia.Checker.CheckIngress(&ingress); err != nil {
		klog.ErrorS(err, "invalid ingress configuration", "ingress", fmt.Sprintf("%v/%v", review.Request.Namespace, review.Request.Name))
		status.Allowed = false
		status.Result = &metav1.Status{
			Status: metav1.StatusFailure, Code: http.StatusBadRequest, Reason: metav1.StatusReasonBadRequest,
			Message: err.Error(),
		}

		review.Response = status
		return review, nil
	}

	klog.InfoS("successfully validated configuration, accepting", "ingress", fmt.Sprintf("%v/%v", review.Request.Namespace, review.Request.Name))
	status.Allowed = true
	review.Response = status

	return review, nil
}

Calls CheckIngress on the received Ingress object

Therefore, an call CheckIngress on arbitrary Ingress objects if they have access to the admission controller server (webook)

Accessing the vulnerable admission controller server webhook

Name:             ingress-nginx-controller-6579f7cfb4-vzfjh
Namespace:        ingress-nginx
Priority:         0
Service Account:  ingress-nginx
Node:             minikube/192.168.49.2
Start Time:       Mon, 24 Mar 2025 18:20:31 -0500
Labels:           app.kubernetes.io/component=controller
                  app.kubernetes.io/instance=ingress-nginx
                  app.kubernetes.io/name=ingress-nginx
                  pod-template-hash=6579f7cfb4
Annotations:      <none>
Status:           Running
IP:               10.244.0.6
IPs:
  IP:           10.244.0.6
Controlled By:  ReplicaSet/ingress-nginx-controller-6579f7cfb4
Containers:
  controller:
    Container ID:  docker://0271a7ebeb16b1131e5c653279ebf0b6543e9e137ac544bdecf8e88e726fce10
    Image:         registry.k8s.io/ingress-nginx/controller:v1.11.3@sha256:d56f135b6462cfc476447cfe564b83a45e8bb7da2774963b00d12161112270b7
    Image ID:      docker-pullable://registry.k8s.io/ingress-nginx/controller@sha256:d56f135b6462cfc476447cfe564b83a45e8bb7da2774963b00d12161112270b7
    Ports:         80/TCP, 443/TCP, 8443/TCP
    Host Ports:    0/TCP, 0/TCP, 0/TCP
    Args:
      /nginx-ingress-controller
      --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
      --election-id=ingress-controller-leader
      --controller-class=k8s.io/ingress-nginx
      --ingress-class=nginx
      --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
      --default-ssl-certificate=ingress-nginx/ingress-certs
      --validating-webhook=:8443
      --validating-webhook-certificate=/usr/local/certificates/cert
      --validating-webhook-key=/usr/local/certificates/key
    State:          Running
      Started:      Mon, 24 Mar 2025 18:20:42 -0500
    Ready:          True
    Restart Count:  0
    Requests:
      cpu:      100m
      memory:   90Mi
    Liveness:   http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=5
    Readiness:  http-get http://:10254/healthz delay=10s timeout=1s period=10s #success=1 #failure=3
    Environment:
      POD_NAME:       ingress-nginx-controller-6579f7cfb4-vzfjh (v1:metadata.name)
      POD_NAMESPACE:  ingress-nginx (v1:metadata.namespace)
      LD_PRELOAD:     /usr/local/lib/libmimalloc.so
    Mounts:
      /usr/local/certificates/ from webhook-cert (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-gs9xj (ro)
Conditions:
  Type                        Status
  PodReadyToStartContainers   True 
  Initialized                 True 
  Ready                       True 
  ContainersReady             True 
  PodScheduled                True 
Volumes:
  webhook-cert:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  ingress-nginx-admission
    Optional:    false
  kube-api-access-gs9xj:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   Burstable
Node-Selectors:              kubernetes.io/os=linux
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type     Reason       Age                    From                      Message
  ----     ------       ----                   ----                      -------
  Normal   Scheduled    5m23s                  default-scheduler         Successfully assigned ingress-nginx/ingress-nginx-controller-6579f7cfb4-vzfjh to minikube
  Warning  FailedMount  5m22s (x3 over 5m23s)  kubelet                   MountVolume.SetUp failed for volume "webhook-cert" : secret "ingress-nginx-admission" not found
  Normal   Pulling      5m19s                  kubelet                   Pulling image "registry.k8s.io/ingress-nginx/controller:v1.11.3@sha256:d56f135b6462cfc476447cfe564b83a45e8bb7da2774963b00d12161112270b7"
  Normal   Pulled       5m12s                  kubelet                   Successfully pulled image "registry.k8s.io/ingress-nginx/controller:v1.11.3@sha256:d56f135b6462cfc476447cfe564b83a45e8bb7da2774963b00d12161112270b7" in 7.493s (7.493s including waiting). Image size: 292645598 bytes.
  Normal   Created      5m12s                  kubelet                   Created container controller
  Normal   Started      5m12s                  kubelet                   Started container controller
  Normal   RELOAD       5m10s                  nginx-ingress-controller  NGINX reload triggered due to a change in configuration

Unfortunately, it seems that this HTTP server may be internet accessible as the ingress-nginx docs say to open the port for it:

POCs

CVE-2025-1974: replicated

IngressNightmare Patch and Vulnerability Analysis

Jacob Sandum

GitHub Issues for each of the CVEs

CVE-2025-24513

Patch

CVE-2025-24514

Patch

CVE-2025-1097

CVE-2025-1098

CVE-2025-1974

Patch

Disabled Functionality

Vulnerable Function - CheckIngress

Accessing the vulnerable admission controller server webhook

POCs

Read more

Pwning your test engineers and their test cases with PIP dependency confusion

Public Research Coming Soon