Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: handle incorrect cluster RESTconfig without panic #20150

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

CefBoud
Copy link
Contributor

@CefBoud CefBoud commented Sep 30, 2024

Description

fixes #19496

The issue arises when a cluster has its secret configuration set with insecure as true but includes a non-empty caData. This PR resolves the issue by handling the error gracefully rather than causing a panic.

Testing Instruction

  1. Deploy a kind cluster. kind create cluster.
  2. Add the cluster to ArgoCD by creating the following secret ( yq is needed to run the command):
cat << EOF > /tmp/kind-cluster-secret.yaml
apiVersion: v1
kind: Secret
metadata:
  name: mycluster-kind
  labels:
    argocd.argoproj.io/secret-type: cluster
type: Opaque
stringData:
  name: kind
  server: $(kind get kubeconfig | yq eval '.clusters[0].cluster["server"]' -)
  config: |
    $(kind get kubeconfig | yq eval -j '
     {
          "tlsClientConfig": {
            "keyData": .users[0].user["client-key-data"],
            "certData": .users[0].user["client-certificate-data"],
            "insecure":  false,
            "caData": .clusters[0].cluster["certificate-authority-data"]
          }
      }' -)

EOF

In my case, the following is generated /tmp/kind-cluster-secret.yaml:

apiVersion: v1
kind: Secret
metadata:
  name: mycluster-kind
  labels:
    argocd.argoproj.io/secret-type: cluster
type: Opaque
stringData:
  name: kind
  server: https://127.0.0.1:37173
  config: |
    {
      "tlsClientConfig": {
        "keyData": "LS0tLS1CRUdJTiBSU0EgUFJJVkFURSBLRVktLS0tLQpNSUlFb3dJQkFBS0NBUUVBMTZmMnlTY1lqKzVmMERNRlJNalp6VW0xOW5rRmIyRHF0ZTdETXlhVE43TG5pbTJrCkVGTjF6NFd6NkhKcytPelhtNld6RGlxcng4NVo5LzJHWDJQZjJvQVVvclBJQU41TWtadEVWTWNoSEZodmNOWWwKZjd5L3Y2czJrZWVPQnpPOTgzVUNFWmM1bEFnYmltWDBnWVJLNjhndXhDVHNvU0JyalY2OGZhVHc2ai9UTG1kdgp4ZENmMHBrSXBRSm8rRkkwdEN1dkt2REh1eWU5c3doSm5oZitqdmlqcVJYZmR4bXIwdDd6c1FYZHZjZlJHSGNpCmw5dWNoUDZzRk5DOHJxODJhRWtPNmdtbkRvb1RSNld1Y3lNTGt6eVpEdTJpOGpucEJ6QUh1a1A5cDBld0YxODYKbG8zY21GbWZHMnJpVm5iSnI0TmJCdy9VbXE2UmQyc1FNQjdoWXdJREFRQUJBb0lCQVFDRnJ4ZHFwcE94VllHTApuTFVkYUt5cmQ1QVhFL3BzaXRXMHBpZlhJNGlMTkpvWDA5T3FVekpxekdVMEtRcW9YSmxqWVJwWUk5ZU02cndXCjROQlZmYUljM3R6cjV5cUtiME5qMTY1S0o0Njg5WTFQazFCM21OZjh4dlB6Q21tZXlWSnRBeEdmT2ZGMWRIa3UKRi9teitBQkJuVHh3VnJ0aFBxbXRYeGJWaEZQYndVV21kVzY1Q0w3Tk9qQXNiUjZ0SlkyL2hRaEtWanVla3ZCMApiT1htRWhlS1FrWVVnV3crSkNKRDE0a2J3ZDJiQXEvd1B0Zk95K3JMQXlHelFSNTA2QUxiTHRVeGg4Y2MxY2xhClNQNXAyZVk1SXNZdXdldHl2b3A0TFB4YkxsaTlRNnBOK0dWQ1NLaFAzUjZQTllsN290c3ZWZE1URnY3OGJxWkwKSXJzaUpkS0JBb0dCQU9vbEVFeDVKQW93aXQ2aUxvSWt0R0Era0x2RGVORndsbUZ6Z242YUJLWFdpK0JJU1d5OApFQnNIL1JWK1d4akg5WUlGdjlnaGMwR3gxcFZ3czNGL3B1MGxhd2M4NjJ1YXJHVU1ZTjJiRlRkNTNtMWh2M0pCCkdndHJ4OE1BbDhpdzNwZXJXbHplYkxiNXhqVUovQ1E2T1NLMUtORTgyTDc5ejltRkUvWkJPcGlEQW9HQkFPdkoKSEIvUmRkT1dvWWd6UTc2SXJkUnRhOUZSVW42TUhIMElMT1FkVWFnM2FWaUFjb0tXYjhKRmFGdlRwZzlaVGxCcQpiU0xIV0NaMFdMRmM0T0M2WFppR2E0dEkzcHdDMmV4OUhiZTFicWUwbytmd2gycWZaSGVyM3JTMllRRzFwSEhOCmxkQzc3NkIrUFovbnNRM1QrWFFaY0RaaENET2ZvZDRvMm5WUGIzMmhBb0dCQUllbUVWczhrM1NUeXVCUWRVRlEKb205Zno5bjMxUGNCa3gzQ0hjZmEwdmFBNVdoNmVJOVdKTDZFVWtzeGVQVlNZeG1BMnpRTHJRcFNUVThtaTg4bApkVC9PZmNNanVBQVBDL0UreXArWVNTSnFxRUlXYnlOeU0vU0o3S0U4cHJMT1JTYjVUdENleTd6MTE2N0NxaUVTCjJpY1JiU1JMYkFhNlpNQlFLTkJ4MmRDaEFuOGJvWFdxUk9HSjZoMEQyUDRYVmUwSTk4bXF2ZHpzOTM1ZktKWWEKSlZWc1lSSDNUUlllTjRMaFFVRDVnZFB6K0lNMDZVUGd6M3VGQmgxZXE0T3UwaWdsdDVyaXQ2ZGVvUGJvdjhDNwp4b2N4SC9vVk5CcXFaQU5pQXNJV2Ewank2bFphLzd5T3VYZGJWMi9oVFVWeVdXVGlOaGp3Um02dm5nVmFBMitXClQ3akJBb0dCQU5xSi94UldnbkZsTkZxUmpuZFZ5eTRiVk0wV3U2ZXVNRjBZRGJUcDdpZkRYRUY2REhBa1RKS0oKU1E5TFQ1NUZXZVJHWTRaSGVZTjZpV0lMbUgwZ01ITUxsVFFnSVRydDVIMDZ6cHFtelNSUGtYT2JXbytyb2EySQovQmlOeUZvRys3T0xzMGEvQXFncE1XR2RqdGNwTnFndmkzSHRXelZlSWY3VzJYNTBsVms1Ci0tLS0tRU5EIFJTQSBQUklWQVRFIEtFWS0tLS0tCg==",
        "certData": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURLVENDQWhHZ0F3SUJBZ0lJVDdPSk4rTDdVZ3N3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBNU1qRXlNVEEyTkRGYUZ3MHlOVEE1TWpFeU1URXhOREZhTUR3eApIekFkQmdOVkJBb1RGbXQxWW1WaFpHMDZZMngxYzNSbGNpMWhaRzFwYm5NeEdUQVhCZ05WQkFNVEVHdDFZbVZ5CmJtVjBaWE10WVdSdGFXNHdnZ0VpTUEwR0NTcUdTSWIzRFFFQkFRVUFBNElCRHdBd2dnRUtBb0lCQVFEWHAvYkoKSnhpUDdsL1FNd1ZFeU5uTlNiWDJlUVZ2WU9xMTdzTXpKcE0zc3VlS2JhUVFVM1hQaGJQb2NtejQ3TmVicGJNTwpLcXZIemxuMy9ZWmZZOS9hZ0JTaXM4Z0Eza3lSbTBSVXh5RWNXRzl3MWlWL3ZMKy9xemFSNTQ0SE03M3pkUUlSCmx6bVVDQnVLWmZTQmhFcnJ5QzdFSk95aElHdU5Ycng5cFBEcVA5TXVaMi9GMEovU21RaWxBbWo0VWpTMEs2OHEKOE1lN0o3MnpDRW1lRi82TytLT3BGZDkzR2F2UzN2T3hCZDI5eDlFWWR5S1gyNXlFL3F3VTBMeXVyelpvU1E3cQpDYWNPaWhOSHBhNXpJd3VUUEprTzdhTHlPZWtITUFlNlEvMm5SN0FYWHpxV2pkeVlXWjhiYXVKV2RzbXZnMXNICkQ5U2FycEYzYXhBd0h1RmpBZ01CQUFHalZqQlVNQTRHQTFVZER3RUIvd1FFQXdJRm9EQVRCZ05WSFNVRUREQUsKQmdnckJnRUZCUWNEQWpBTUJnTlZIUk1CQWY4RUFqQUFNQjhHQTFVZEl3UVlNQmFBRk53YVFpNmtEdnFPaTJjTAo1ZENKRnphdEtwZHBNQTBHQ1NxR1NJYjNEUUVCQ3dVQUE0SUJBUUJ5eWpLU1BGakUwTmJqY0Y3Vm9KSnNuTE42CmxOcjNlSU80NEU3eDJCNkh6VXFOMTFGN2JJRU5PWjJNVEFRSCtkMHg2eVdUZFBSazJ2dE05TjlrOVR0U2Qxa2oKdlh6bGVmbjBzR2pKU2JjczlpeG9GRjJZdUVVTk1TK1dqWTZHOVJtV2xDalJ1OFNHQ1NtK3RBejdtWWJnZEQ4bgpsbDlLZUF5dEtiWnJTWlJVYzQ2ZGJTYktldlFoVHdFd2RXTU5BeFcwUDRCUU83bis3K2ljSE9nRGs4VUEzaUkzCkdmazNBRzBOc21BTUV3Tndkek81UXNzYm9ZanRVLzZvSm92K1c0VGlVWU1qazM0OTFaZlRSKzJzQkxxNDMzOFEKSyt0eFBZQmlSdk5od05vQjhISnFHZmxSTFhCUTBoVGxvMURQcC9EWkJEUDR3bktGOTZrTTV3UDlLaW13Ci0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K",
        "insecure": false,
        "caData": "LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSURCVENDQWUyZ0F3SUJBZ0lJYjJTcmhBMzJjNzh3RFFZSktvWklodmNOQVFFTEJRQXdGVEVUTUJFR0ExVUUKQXhNS2EzVmlaWEp1WlhSbGN6QWVGdzB5TkRBNU1qRXlNVEEyTkRGYUZ3MHpOREE1TVRreU1URXhOREZhTUJVeApFekFSQmdOVkJBTVRDbXQxWW1WeWJtVjBaWE13Z2dFaU1BMEdDU3FHU0liM0RRRUJBUVVBQTRJQkR3QXdnZ0VLCkFvSUJBUURCeWpCeEl5NHlkVnRVUFhIbGNiYkYzSGwwRkNUTnE3UFBPb1hHODd3Zzh0RlExd1RicWg0Uit2MlUKUThJLyt1UmNRMllibGJiQzZnekY1NlhqUWJMRG9UOWwxbHJlVEl3emdCUG9UaTZmQmwvQjQ5dmFqdFQxbDNYbQpFSDNJTmhaME5FbTlZQXZZVmoyc2libWhIaDVrWnNDQUdOeW5JZFlUYkNIYlZRaEFtdUV2YzdDNG1qNlNCbFZPCkJJQkt2NGhyZ3c2dUhQMFJtOVhoR0dnbk5wQU1IYUtxaitoZ1lQcis5WFVWRW1pZjkyRjRnZTdvSlhWUC9nY3YKNllBd1d4K0NpN1RwczhZbGxFd3pCUVNiSHhURE45L3U5U2Vja0FlRHl0bmplK2Q3RjhqeFg5eFNabjZEc2tQRgpxNEova1V6MlN6ZW41SFRHNmRxYmQwTUdVVnVGQWdNQkFBR2pXVEJYTUE0R0ExVWREd0VCL3dRRUF3SUNwREFQCkJnTlZIUk1CQWY4RUJUQURBUUgvTUIwR0ExVWREZ1FXQkJUY0drSXVwQTc2am90bkMrWFFpUmMyclNxWGFUQVYKQmdOVkhSRUVEakFNZ2dwcmRXSmxjbTVsZEdWek1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQkFRQ2pUYjFJOUQyWApONFNJQjljc1FjK3dSRUV0NlhWYVhIOVUycW9NVFR3VFRrUW9mNzIrYk9jMG1UaHZnR21KTmZiaEVzWFJES1VkCk1PTVAwWEtZZ0lOMlNENE02dHE0c1J2WnJocTY2ck5vRlJFNUpmKzFxKzBxakhpdDA4OWFlMldWRHhzN1BGMGIKb1k3WW9MTXpvTGdSNjQ4NkdQdlRMYm8wcHVveDJIVHlMYjhWSUFwVStJU0tIcDBrT2pBTFJuSUhEbTM4RG1WVgpTZ0d3aGd1N284UkFPbzhTZit2QVJLTXhvZ2d6NWc4QkQzQzRteE9mcUNndVFUeitXc0JKTldQUkFYaHFWRFNQCm81Vk5iN0FsR2d5Y2VKS3k5LzVNczM0WVlPdkRDK1NDT2k2MWFPbXBTd1BHanIrSm5LTmI2anpSdjhDYlNSQXYKcXRnMnhRVERDWjRJCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K"
      }
    }

Create the secret by running kubectl apply -f /tmp/kind-cluster-secret.yaml.

  1. Create an ArgoCD Application and assign it to the created cluster. Everything should be working normally.
  2. Set insecure to true then update the secret:
sed -i  's/false/true/' /tmp/kind-cluster-secret.yaml&& kubectl apply -f /tmp/kind-cluster-secret.yaml
  1. The controller fails with the following:
Trace
19:35:00                controller | panic: Unable to apply K8s REST config defaults: specifying a root certificates file with the insecure flag is not allowed [recovered]
19:35:00                controller | 	panic: Unable to apply K8s REST config defaults: specifying a root certificates file with the insecure flag is not allowed
19:35:00                controller | goroutine 93 [running]:
19:35:00                controller | k8s.io/apimachinery/pkg/util/runtime.handleCrash({0x7375150, 0xa0cffc0}, {0x5c85840, 0xc0047161b0}, {0xa0cffc0, 0x0, 0x224b125?})
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:89 +0xee
19:35:00                controller | k8s.io/apimachinery/pkg/util/runtime.HandleCrash({0x0, 0x0, 0xc001a941c0?})
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/runtime/runtime.go:59 +0x108
19:35:00                controller | panic({0x5c85840?, 0xc0047161b0?})
19:35:00                controller | 	/usr/local/go/src/runtime/panic.go:785 +0x132
19:35:00                controller | github.com/argoproj/argo-cd/v2/pkg/apis/application/v1alpha1.(*Cluster).RESTConfig(0x645e2c0?)
19:35:00                controller | 	/home/cef/work/misc/K8S/argocd/argo-cd/pkg/apis/application/v1alpha1/types.go:3198 +0x79
19:35:00                controller | github.com/argoproj/argo-cd/v2/controller/cache.(*liveStateCache).handleModEvent(0xc000842b60, 0xc00083e1e0, 0xc00083e3c0)
19:35:00                controller | 	/home/cef/work/misc/K8S/argocd/argo-cd/controller/cache/cache.go:824 +0x165
19:35:00                controller | github.com/argoproj/argo-cd/v2/util/db.(*db).WatchClusters.func2(0xc0010b4c80, 0xc004b28000)
19:35:00                controller | 	/home/cef/work/misc/K8S/argocd/argo-cd/util/db/cluster.go:184 +0xda
19:35:00                controller | github.com/argoproj/argo-cd/v2/util/db.(*db).watchSecrets.func4({0x670afc0?, 0xc0010b4c80?}, {0x670afc0?, 0xc004b28000?})
19:35:00                controller | 	/home/cef/work/misc/K8S/argocd/argo-cd/util/db/secrets.go:134 +0x49
19:35:00                controller | k8s.io/client-go/tools/cache.ResourceEventHandlerFuncs.OnUpdate(...)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/tools/cache/controller.go:253
19:35:00                controller | k8s.io/client-go/tools/cache.(*processorListener).run.func1()
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/tools/cache/shared_informer.go:976 +0xea
19:35:00                controller | k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x30?)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:226 +0x33
19:35:00                controller | k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0xc0047c9f70, {0x7335440, 0xc001404f90}, 0x1, 0xc00140a690)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:227 +0xaf
19:35:00                controller | k8s.io/apimachinery/pkg/util/wait.JitterUntil(0xc001308770, 0x3b9aca00, 0x0, 0x1, 0xc00140a690)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:204 +0x7f
19:35:00                controller | k8s.io/apimachinery/pkg/util/wait.Until(...)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/backoff.go:161
19:35:00                controller | k8s.io/client-go/tools/cache.(*processorListener).run(0xc00144a3f0)
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/tools/cache/shared_informer.go:972 +0x5a
19:35:00                controller | k8s.io/apimachinery/pkg/util/wait.(*Group).Start.func1()
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:72 +0x4c
19:35:00                controller | created by k8s.io/apimachinery/pkg/util/wait.(*Group).Start in goroutine 292
19:35:00                controller | 	/home/cef/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:70 +0x73
19:35:00                controller | exit status 2
19:35:00                controller | Terminating controller

Checklist:

  • Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this does not need to be in the release notes.
  • The title of the PR states what changed and the related issues number (used for the release note).
  • The title of the PR conforms to the Toolchain Guide
  • I've included "Closes [ISSUE #]" or "Fixes [ISSUE #]" in the description to automatically close the associated issue.
  • I've updated both the CLI and UI to expose my feature, or I plan to submit a second PR with them.
  • Does this PR require documentation updates?
  • I've updated documentation as required by this PR.
  • I have signed off all my commits as required by DCO
  • I have written unit and/or e2e tests for my change. PRs without these are unlikely to be merged.
  • My build is green (troubleshooting builds).
  • My new feature complies with the feature status guidelines.
  • I have added a brief description of why this PR is necessary and/or what this PR solves.
  • Optional. My organization is added to USERS.md.
  • Optional. For bug fixes, I've indicated what older releases this fix should be cherry-picked into (this may or may not happen depending on risk/complexity).

Copy link

bunnyshell bot commented Sep 30, 2024

✅ Preview Environment deployed on Bunnyshell

Component Endpoints
argocd https://argocd-k5jfmq.bunnyenv.com/
argocd-ttyd https://argocd-web-cli-k5jfmq.bunnyenv.com/

See: Environment Details | Pipeline Logs

Available commands (reply to this comment):

  • 🔴 /bns:stop to stop the environment
  • 🚀 /bns:deploy to redeploy the environment
  • /bns:delete to remove the environment

Copy link

bunnyshell bot commented Sep 30, 2024

✅ Preview Environment created on Bunnyshell but will not be auto-deployed

See: Environment Details

Available commands (reply to this comment):

  • 🚀 /bns:deploy to deploy the environment

Copy link

codecov bot commented Sep 30, 2024

Codecov Report

Attention: Patch coverage is 36.92308% with 41 lines in your changes missing coverage. Please review.

Project coverage is 55.87%. Comparing base (afcc71b) to head (4091db1).
Report is 1 commits behind head on master.

Files with missing lines Patch % Lines
controller/sync.go 20.00% 10 Missing and 2 partials ⚠️
server/cluster/cluster.go 46.66% 6 Missing and 2 partials ⚠️
controller/cache/cache.go 50.00% 2 Missing and 2 partials ⚠️
pkg/apis/application/v1alpha1/types.go 60.00% 3 Missing and 1 partial ⚠️
server/application/terminal.go 0.00% 4 Missing ⚠️
cmd/argocd/commands/admin/cluster.go 0.00% 3 Missing ⚠️
controller/appcontroller.go 50.00% 1 Missing and 1 partial ⚠️
server/application/application.go 33.33% 1 Missing and 1 partial ⚠️
util/argo/argo.go 33.33% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #20150      +/-   ##
==========================================
- Coverage   55.89%   55.87%   -0.03%     
==========================================
  Files         321      321              
  Lines       44497    44540      +43     
==========================================
+ Hits        24871    24886      +15     
- Misses      17053    17076      +23     
- Partials     2573     2578       +5     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@CefBoud CefBoud force-pushed the fix/handle-cluster-RESTconfig-no-panic branch 2 times, most recently from 04ac06e to 87a08dc Compare October 1, 2024 01:05
@CefBoud CefBoud marked this pull request as ready for review October 1, 2024 02:03
@CefBoud CefBoud requested a review from a team as a code owner October 1, 2024 02:03
Copy link
Member

@reggie-k reggie-k left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR!
Could you add tests for the behavior of adding a cluster with both insecure and CA data?

@CefBoud CefBoud force-pushed the fix/handle-cluster-RESTconfig-no-panic branch from 87a08dc to db78f3f Compare October 1, 2024 11:59
@CefBoud CefBoud force-pushed the fix/handle-cluster-RESTconfig-no-panic branch from db78f3f to 4091db1 Compare October 1, 2024 12:01
@CefBoud CefBoud requested a review from reggie-k October 1, 2024 13:13
Copy link
Contributor

@todaywasawesome todaywasawesome left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Argocd Panics with invalid cluster secret
3 participants