# v3.2 to 3.3

## Breaking Changes

### ApplicationSet CRD exceeds the size limit for client-side apply

In this version, the ApplicationSet CRD exceeds the size limit for client-side apply, meaning that the previous upgrade methods will result in `The CustomResourceDefinition "applicationsets.argoproj.io" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes`

In order to avoid this issue, it is now required to upgrade Argo CD using Server Side Apply (SSA) along with `resolve conflicts`.
SSA does not store the ApplicationSet fields in `last-applied` annotation, thus not impacted by the annotation size limitation.
The `--force-conflicts` flag allows the apply operation to take ownership of fields that may have been previously managed by other tools (such as Helm or a previous `kubectl apply`). This is necessary for upgrades. Note that any custom modifications you've made to fields that are defined in the Argo CD manifests (like `affinity`, `env`, or `probes`) will be overwritten. However, fields not specified in the manifests (like `resources` limits/requests or `tolerations`) will be preserved.

#### Upgrading Argo CD which is managing itself 

When Argo CD is upgraded using an Argo CD Application, it is required to enable Server-Side Apply in the Application spec: 
``` yaml
    syncOptions:
    - ServerSideApply=true
```
With this configuration, Argo CD will add the `resolve conflicts` option automatically.   

#### Upgrading Argo CD manually using Kustomize or plain manifests

When Argo CD is upgraded manually using plain manifests or Kustomize overlays, it is required to upgrade it with `--server-side --force-conflicts`. For example, `kubectl apply -n argocd --server-side --force-conflicts -f manifests/install.yaml`

#### Upgrading Argo CD manually using Helm

Users upgrading Argo CD manually using `helm upgrade` are not impacted by this change, since Helm does not use client-side apply and does not result in creation of the `last-applied` annotation.

#### Users who previously upgraded to 3.3.0 or 3.3.1
In some cases, after upgrading to one of those versions and applying Server-Side Apply, the following error occured:   
`one or more synchronization tasks completed unsuccessfully, reason: Failed to perform client-side apply migration: failed to perform client-side apply migration on manager kubectl-client-side-apply: error when patching "/dev/shm/2047509016": CustomResourceDefinition.apiextensions.k8s.io "applicationsets.argoproj.io" is invalid: metadata.annotations: Too long: may not be more than 262144 bytes`.

Users that have configured the sync option `ClientSideApplyMigration=false` as a temporary remediation for the above error, should remove it after upgrading to `3.3.2`. Disabling `ClientSideApplyMigration` imposes a risk to encounter conflicts between K8s field managers in the future.

### Source Hydrator Now Tracks Hydration State Using Git Notes

Previously, Argo CD's Source Hydrator pushed a new hydrated commit for every DRY (source) commit, regardless of whether any manifest files (`manifest.yaml`) actually changed. This was necessary for the hydrator to track which DRY commit had last been hydrated: it embedded this information in the `hydrator.metadata` file's `drySha` field in each hydrated commit.

**Starting with v3.3, the Source Hydrator uses [git notes](https://git-scm.com/docs/git-notes) to record the state of the most recently hydrated DRY commit instead of creating a hydrated commit on every DRY commit.** The git note is stored in a dedicated namespace reserved for the Source Hydrator, making the state both reliable and isolated from other repository operations.

This design change improves repository cleanliness, reduces unnecessary commit noise, and lowers risk from excessive branch divergence due to frequent automated commits.

#### How It Works

- **On each hydration run:**
  1. The hydrator first pulls the existing git note in its namespace to check the last hydrated DRY commit SHA.
  2. If the note SHA matches the latest DRY SHA, the hydrator logs a debug message and skips hydration.
  3. If files such as `manifest.yaml` have *not* changed (even if DRY SHA is new), the hydrator skips committing manifests and only updates the git note to reflect the latest hydrated DRY SHA.
  4. If manifest files have changed, the hydrator commits the updated manifests and also updates the git note.

#### Migration Impact

- **Behavioral:**  
  - Users will no longer see a hydrated commit for every DRY commit. Only actual manifest changes will create a new commit; otherwise, the hydration state is tracked in the git note namespace.
- **Operational:**  
  - Applications and tools that depended on a hydrated commit for every DRY commit should now use the git note in the Source Hydrator namespace to determine the hydration state.
- **No action needed** for most users, but if you have automations dependent on hydrated commits as signals, please update them to consult the new git note.

#### Rationale and Benefits

- **Reduces repository clutter**: fewer unnecessary commits.
- **Improves performance** for teams with high-frequency DRY changes and rare manifest changes.
- **Decreases risk** of merge conflicts and branch bloat in automation scenarios.

### Removal of Application Path Cleaning During Hydration

- **Behavioral Change:**  
  In Argo CD versions prior to v3.3, the Source Hydrator would automatically clean (delete all files in) the application's configured path before writing out new manifest files during each hydration run. Starting with v3.3, this cleaning logic has been removed. The hydrator will now only overwrite or create files that correspond to the current manifest output; any additional files or obsolete data in the application path will remain untouched unless explicitly overwritten.
- **Operational Impact:**  
  If your repository or automation relied on automatic cleanup of stale or obsolete files in application paths, you must now handle this cleanup manually. This change avoids the risk of accidental deletion of unrelated files (for instance, if application paths overlap with directories containing other assets or when applications are restructured).
- **Recommended Action:**  
  Review your automation workflows and repository maintenance scripts to ensure that old or unwanted files in application paths are cleaned up if necessary. Consider implementing a periodic manual or automated cleanup procedure if your use case requires it.
- For more details on current behavior, see the [Source Hydrator user guide](../../user-guide/source-hydrator.md).

### Cluster version format change

**New behavior:**

3.3.3 now stores the cluster version in a more detailed format, `vMajor.Minor.Patch` compared to the previous format `Major.Minor`. 

This change is aligning how ArgoCD interprets K8s cluster version with how Helm `3.19.0` and above interprets it.    

This change makes it easier to compare versions and to support future features. It also allows for more accurate version comparisons and better compatibility with future Kubernetes releases. 


**Impact:**

Application Sets with Cluster Generators, that fetch clusters based on their Kubernetes version and use `argocd.argoproj.io/auto-label-cluster-info` on the cluster secret, need to be updated to use `argocd.argoproj.io/kubernetes-version` with the `vMajor.Minor.Patch` format instead of the previous `Major.Minor` format. 
More details [here](../applicationset/Generators-Cluster.md#fetch-clusters-based-on-their-k8s-version).

Additionally, API, UI and CLI commands that retrieve cluster information now return the version in the `vMajor.Minor.Patch` format.

The env variable $KUBE_VERSION that is used with Argo CD CMP Plugins remains unchanged and returns the version in `Major.Minor.Patch` format, so CMP Plugins are not impacted.

### Anonymous call to Settings API returns fewer fields

The Settings API now returns less information when accessed anonymously.
It no longer returns the `resourceOverrides` field which is considered sensitive information.

### New ENV Variable to control K8s Server Side Timeout of API Requests

The new environment variable `ARGOCD_K8S_SERVER_SIDE_TIMEOUT` can be used to control the K8s server side timeout of API requests.
In 3.2 and before this change, the K8s server side timeout was controlled by `ARGOCD_K8S_TCP_TIMEOUT` 
which is also used to control the TCP timeout when communicating with the K8s API server. 
From now onwards, the Kubernetes server-side timeout is controlled by a separate environment variable.

### Deprecated --self-heal-backoff-cooldown-seconds flag 

The `--self-heal-backoff-cooldown-seconds` flag of the `argocd-application-controller` has been deprecated and will be
removed in a future release.

## Helm Upgraded to 3.19.4

Argo CD v3.3 upgrades the bundled Helm version to 3.19.4. This Helm release interprets K8s version in a semantic version format of `vMajor.Minor.Patch`, instead of the previous `vMajor.Minor` format. This led to a breaking change in Argo CD described [above](#cluster-version-format-change).

## Kustomize Upgraded to 5.8.1

Argo CD v3.3 upgrades the bundled Kustomize version from v5.7.0 to v5.8.1. According to the
[5.7.1](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.7.1)
and [5.8.1](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.8.1) release notes, there are no breaking changes.

However, note that Kustomize 5.7.1 introduces code to replace the `shlex` library used for parsing arguments in exec plugins.
If any existing manifests become corrupted, please follow the instructions in the
[5.7.1 release notes](https://github.com/kubernetes-sigs/kustomize/releases/tag/kustomize%2Fv5.7.1).

Kustomize 5.8.0 also resolves an issue where namespaces were not properly propagated to Helm charts.
If you rely on Helm charts within kustomization files, please review the details in the associated fix:
[kubernetes-sigs/kustomize#5940](https://github.com/kubernetes-sigs/kustomize/pull/5940).

## Added Healthchecks

* [ceph.rook.io/CephCluster](https://github.com/argoproj/argo-cd/commit/9ee5cca38b84524c701b75f1cea3611b9e857dbe)
* [ceph.rook.io/CephObjectStore](https://github.com/argoproj/argo-cd/commit/9ee5cca38b84524c701b75f1cea3611b9e857dbe)
* [objectbucket.io/ObjectBucketClaim](https://github.com/argoproj/argo-cd/commit/9ee5cca38b84524c701b75f1cea3611b9e857dbe)
* [keda.sh/ScaledJob](https://github.com/argoproj/argo-cd/commit/e58bdf2f87b5b60a05fde0b7837779061b170c08)
* [services.cloud.sap.com/ServiceBinding](https://github.com/argoproj/argo-cd/commit/51c9add05d9bc8f8fafc1631968eb853db53a904)
* [services.cloud.sap.com/ServiceInstance](https://github.com/argoproj/argo-cd/commit/51c9add05d9bc8f8fafc1631968eb853db53a904)
* [\_.cnrm.cloud.google.com/\_](https://github.com/argoproj/argo-cd/commit/30abebda3d930d93065eec8864aac7e0d56ae119)

## More detailed cluster version

3.3.3 now stores the cluster version in a more detailed format, Major.Minor.Patch compared to the previous format Major.Minor.
This change is to make it easier to compare versions and to support future features.
This change also allows for more accurate version comparisons and better compatibility with future Kubernetes releases.

Users will notice it in the UI and the CLI commands that retrieve cluster information.