Skip to content

avoid headless - #291

Closed
Andrei Kvapil (kvaps) wants to merge 208 commits into
mainfrom
avoid-headless
Closed

Andrei Kvapil (kvaps) wants to merge 208 commits into
mainfrom
avoid-headless

Conversation

@kvaps

@kvaps Andrei Kvapil (kvaps) commented Jun 27, 2025 •

Copy link
Copy Markdown
Member

This change references to upstream issue clastix/kamaji#856

It's better to use ordinary service instead accessing etcd via multiple endpoints from kubernetes-api due to DNS resloving issue, which might happen when one replica have no IP address assigned.

Andrei Kvapil (kvaps) and others added 30 commits March 7, 2024 16:21
Signed-off-by: Andrei Kvapil <[email protected]>
Signed-off-by: Andrei Kvapil <[email protected]>
Initializing the project
Signed-off-by: Andrei Kvapil <[email protected]>
Update kubebuilder to v3.14.0
Generated and wrote the most basic EtcdCluster custom resource.
See #9
Fixes from comments #15 
- Fix API group name for custom resources
- remove unnecessary comment lines
1. Union argument layers and copy source code.
2. Changed layers order final image for grow cache hit
fixes: #10

Signed-off-by: Andrei Kvapil <[email protected]>
- Added basic validating and defaulting webhooks
- Added simple test for defaulting webhook
Controller now can successfully bootstrap etcd cluster :)

Controller creates Service, ConfigMap and StatefulSet based on
EtcdCluster CR
Only EmptyDir storage is supported
Initialized status is being set and updated
Proof that it works:
https://github.com/hiddenmarten/etcd-operator/pull/1

After that just changed the name of the label, to the easier one.
Update cluster Ready status according to StatefulSet status and update
ConfigMap cluster state to `existing` after first time STS is ready
fixes #24
fixes #23 
fixes #30

---------

Co-authored-by: Kirill Ilin <[email protected]>
Co-authored-by: Kirill Garbar <[email protected]>
The auxiliary GH action to setup release drafts and update change log
accordingly. Configured with autolabeler, so the PRs will be labeled
according to the PR branch name and/or files changed:

| Branch name  | Files changed | Resulting label |
| ------------- | ------------- | ------------- |
| deps/*  |  | `dependencies`  |
| chore/*  |  | `chore`  |
| docs/* | **/*.md | `documentation` |
| enh/* |  | `enhancement` |
| feature/* |  | `feature` |
| fix/* |  | `bugfix` |
| bugfix/* |  | `bugfix` |
|  | /api/* | `api-change` |
|  | /internal/* | `controllers` |

Based on labels release draft will be updated with 'Features', 'Bug
Fixes' and 'Maintenance' categories which will include corresponding
PR's title.
It's also possible to manage release draft's version by applying
`major`, `minor` and `patch` labels on PR. By default the version will
be the next patch version after the last release, v0.0.1 if none.

---------

Signed-off-by: Artem Bortnikov <[email protected]>
Explicit RW permissions for content and pull requests required, due to
repository settings, which grants to workflows only read permissions in
the repository for the contents and packages scopes.

- RW for content is required for release draft creation
(release-drafter).
- RW for pull-requests is required for adding labels to PRs
(auto-labeler).

Signed-off-by: Artem Bortnikov <[email protected]>
in order to access cluster from clients using one IP we decided to use
ClusterIP Service
Created basic docs site and filled content for main pages
This is important when scaling cluster up for this operation to be zero down time
fixes #25
In case error happens during status update request, reconcile will not
be requeued. So, this call should be moved ouside `defer` function
fixes #36
Just an idea to create `CODEOWNERS` to automatically assign reviewers
when changing specific part of code.
Is there anybody who would like to keep informed about updates?

Aproove from codeowners is required to merge every specific change.
The test of etcd cluster with TLS and auth config fails frequently,
because the client certificate does not get created, before the etcd
cluster is up and running. This PR adds a step to wait for the client
cert to become ready, before executing the rest of the tests.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Reduced RSA private key sizes for various certificates to enhance
security.
- Added a new end-to-end test case to ensure readiness of TLS client
certificates before proceeding with cluster operations.

- **Bug Fixes**
- Improved test reliability by ensuring TLS certificates are ready
before stateful set readiness checks.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Chores**
- Updated ownership assignments for various directories and file types
to include new code owners.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Artem Bortnikov <[email protected]>
this PR fixes the errors:

```
E0924 11:31:59.069472       1 reflector.go:150] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:232: Failed to watch *v1.StorageClass: unknown (get storageclasses.storage.k8s.io)
E0924 11:32:57.379956       1 reflector.go:150] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:232: Failed to watch *v1.StorageClass: unknown (get storageclasses.storage.k8s.io)
E0924 11:33:50.776342       1 reflector.go:150] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:232: Failed to watch *v1.StorageClass: unknown (get storageclasses.storage.k8s.io)
E0924 11:34:21.872401       1 reflector.go:150] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:232: Failed to watch *v1.StorageClass: unknown (get storageclasses.storage.k8s.io)
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Enhanced role permissions to include the ability to monitor changes to
resources in real-time by adding the `watch` verb.
- **Bug Fixes**
- Improved resource management capabilities for the
`EtcdClusterReconciler` by updating permissions to allow watching of
storage classes.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
# Motivation

I started some "R'n'D" (scare quotes intended) for implementing scale
up, scale down, self-healing and so on and quickly realized, that the
coding of the member add/member remove and similar steps is the more
trivial part of the undertaking. The difficult part is coming up with a
working algorithm that can correctly deduce the cluster's state and
execute the necessary actions at the right time.

To better reason about the controller's algorithm now, and to better
develop it going forward, I feel it is important to have good
documentation of the current design and the intended next steps, so I
started with trying to document the current state of the code.

# Results

This document contains a mermaid flowchart that outlines the
reconciliation loop. It is better viewed in [rendered
form](https://github.com/aenix-io/etcd-operator/blob/docs/design/docs/DESIGN.md).

Going forward, I envision this document to have at least three purposes:
* Let the developers spot flaws and prompt them to open issues.
* Act as a more detailed form of documentation for advanced users.
* Be a blueprint for implementing anything non-trivial.

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Documentation**
- Updated the design document for the `EtcdCluster` custom resources
with a detailed flowchart illustrating the reconciliation process and
lifecycle management within a Kubernetes environment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Hidden Marten <[email protected]>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Subj. Without such import, there is auth error:

```
Error setting up etcd client: error creating Kubernetes client: no Auth Provider found for name "oidc"
```

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **New Features**
- Introduced commands for managing etcd pods, including status checks,
defragmentation, compaction, alarm management, and member management.
  
- **Bug Fixes**
- Standardized error handling for improved user feedback during command
execution.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
* We create several etcd clients in the reconciliation loop, but never
  had a defer .Close() for them.
* We erroneously try to configure these clients with etcd-0 instead of
  etcd-0.etcd-headless.namespace as the url.

Now fixed.

Signed-off-by: Timofei Larkin <[email protected]>
Signed-off-by: Timofei Larkin <[email protected]>
* Several `defer c.Close()` added to clean up etcd clients after reconciliation.
* DNS names targeting etcd cluster members fixed to resolve properly.
* TLS capability added to etcd clients created during reconciliation.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

- **Documentation**
- Added Kirill Klinchenkov and Timofei Larkin as new maintainers in the
maintainers list.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
@coderabbitai

coderabbitai Bot commented Jun 27, 2025 •

Copy link
Copy Markdown
Contributor

Walkthrough

The etcdcluster manifest was updated to simplify the DataStore endpoints from multiple pod-specific DNS entries to a single service DNS entry. Additionally, the DNS names in the etcd-server and etcd-peer Certificate specifications were expanded to include three more service-related DNS names.

Changes

File(s) Change Summary
examples/manifests/etcdcluster-kamaji.yaml Simplified DataStore endpoints to a single service DNS; added three new DNS names to Certificate specs.

Poem

In the cluster where data flows free,
Endpoints unite, as simple as can be.
Certificates grow with new DNS delight,
Etcd's connections now shining bright.
With every hop and every name,
Kamaji’s cluster won a simpler game!
🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 856fede and dd4c9ac.

📒 Files selected for processing (1)
  • examples/manifests/etcdcluster-kamaji.yaml (3 hunks)
⏰ Context from checks skipped due to timeout of 90000ms (7)
  • GitHub Check: pre-commit
  • GitHub Check: test-e2e on k8s penultimate version
  • GitHub Check: test-e2e on k8s latest version
  • GitHub Check: test on k8s latest version
  • GitHub Check: test on k8s penultimate version
  • GitHub Check: test-e2e on k8s previous version
  • GitHub Check: test on k8s previous version
🔇 Additional comments (3)
examples/manifests/etcdcluster-kamaji.yaml (3)

140-143: Service DNS SANs are a welcome addition

Adding the three service-level names ensures clients using the new service endpoint pass TLS verification.
No issues spotted.


178-181: Mirror change for peer certificate looks good

The same SANs on the peer cert keep intra-cluster traffic valid when connecting via the service.


23-25: ```shell
#!/bin/bash

Locate and display the first 50 lines of the example manifest to verify its kind and context

file=$(fd etcdcluster-kamaji.yaml | head -n1)
echo "Found file: $file"
head -n 50 "$file"


</details>

</blockquote></details>

</details>
<!-- internal state start -->


<!-- DwQgtGAEAqAWCWBnSTIEMB26CuAXA9mAOYCmGJATmriQCaQDG+Ats2bgFyQAOFk+AIwBWJBrngA3EsgEBPRvlqU0AgfFwA6NPEgQAfACgjoCEYDEZyAAUASpETZWaCrKNwSPbABsvkCiQBHbGlcSHFcLzpIACI0CXx4elgSNFpIxEQAGmj7bAFmdRp6OUhsREpIAGsJNG5kNGRbSAxHAQqAJgBOAEZMngp8bnxy5FwAd3xIZm0sBlhMUkQNSAAxeApEXD71dB98MeRmfH8w+DZIADNjsOTIElwGegYvMpo+AntcZ1Dsbj7eaSOFReeQfM68fBST4qeBedTya7+OEwuG4eS0bAUeAYIgoDDqeBoOEAL2o8HwGGWAGVRBTaNtQnEErRkGVsbjmN5xNxIncMLQhtjcMgrnwANJoaZCeB9MYIOZTNDyRDguEXUG3JgYchiclYa5ai7wIiYskU/r4BjSJYwW5NVBxbReYEeUV+EgSeAkMboUKwXC4OocAD0waI6lgeQ0TGYwbQZHgAA8wOTg/dHmBBsoCBRg9xvF5g11uhojPpjOAoGR6PgLjgCMQyNmojG2BhOP1+MJROIpDJ5EwlFRVOotDpyyYoHBUA6sGg8IRSOQqEUFKx2FwqD6HE4XJASoPlCPNNpdGBDBXTAYSInJTzpMHpviLiFEGmHrRnq9KGBKpK0NKGiyJKXgcAY0QQQYFiQAAggAko2y7UFEO7THutaMPMOLSG4tzVoK7bIHCmx4jcHgACLUGgVI5h4iDcKIkBjA09iqvARpRBcAzMDc/geNitCSIk2BEjwiiQORAByVJ8rgWLSExEZiRQwphJMaCsTivLlBQnpWhJ0myXuYwRvgeDoMpmiwbQgniBSRIgn0uC3FJMkYJKCnOFQsjINiZF3B+YA6VIfCYPQ6a0GADEVAAwpQ4hGgwyH2AxDDIGMlAeDeND8lEJSpIJOK8SQHjkD67DyYgXDRBF0R9DVH4aH+UrwEFPk0MwGiIBIDB1eg/IxBFTX/tKbWbCQnXdQw0YvONFAaF4lpEtEyzuOUmELAp2DPhqWX8gRoT+C+/gYPpYV3Im3Dnc5FGGe5bDIEwIV5btMAADIyVaKnsfASU0EsRjmJYMFeG8ZoYKMkw3ZASjPM44PIBhN5DCpUTXPmAhwgwskEjhBiSRSJClhB0Rluel5VgNGHzg2S7Nk8LBth2W65Lu8gHooR5qCe47k5Oa4FLgAD6iSIEL/iet6dBC5s3xnhe/M2QALAwnRoAwAAcADsAi0AIACsFxoFrFz6yQADMnRq90WtK0rAAM3TtKonQkAAbDbWu0Br8sUwL6giyy4sel6GW0EL1by0AA=== -->

<!-- internal state end -->
<!-- tips_start -->

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

<details>
<summary>❤️ Share</summary>

- [X](https://twitter.com/intent/tweet?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A&url=https%3A//coderabbit.ai)
- [Mastodon](https://mastodon.social/share?text=I%20just%20used%20%40coderabbitai%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20the%20proprietary%20code.%20Check%20it%20out%3A%20https%3A%2F%2Fcoderabbit.ai)
- [Reddit](https://www.reddit.com/submit?title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&text=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code.%20Check%20it%20out%3A%20https%3A//coderabbit.ai)
- [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https%3A%2F%2Fcoderabbit.ai&mini=true&title=Great%20tool%20for%20code%20review%20-%20CodeRabbit&summary=I%20just%20used%20CodeRabbit%20for%20my%20code%20review%2C%20and%20it%27s%20fantastic%21%20It%27s%20free%20for%20OSS%20and%20offers%20a%20free%20trial%20for%20proprietary%20code)

</details>

<details>
<summary>🪧 Tips</summary>

### Chat

There are 3 ways to chat with [CodeRabbit](https://coderabbit.ai?utm_source=oss&utm_medium=github&utm_campaign=aenix-io/etcd-operator&utm_content=291):

- Review comments: Directly reply to a review comment made by CodeRabbit. Example:
  - `I pushed a fix in commit <commit_id>, please review it.`
  - `Explain this complex logic.`
  - `Open a follow-up GitHub issue for this discussion.`
- Files and specific lines of code (under the "Files changed" tab): Tag `@coderabbitai` in a new review comment at the desired location with your query. Examples:
  - `@coderabbitai explain this code block.`
  -	`@coderabbitai modularize this function.`
- PR comments: Tag `@coderabbitai` in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
  - `@coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.`
  - `@coderabbitai read src/utils.ts and explain its main purpose.`
  - `@coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.`
  - `@coderabbitai help me debug CodeRabbit configuration file.`

### Support

Need help? Create a ticket on our [support page](https://www.coderabbit.ai/contact-us/support) for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

### CodeRabbit Commands (Invoked using PR comments)

- `@coderabbitai pause` to pause the reviews on a PR.
- `@coderabbitai resume` to resume the paused reviews.
- `@coderabbitai review` to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
- `@coderabbitai full review` to do a full review from scratch and review all the files again.
- `@coderabbitai summary` to regenerate the summary of the PR.
- `@coderabbitai generate docstrings` to [generate docstrings](https://docs.coderabbit.ai/finishing-touches/docstrings) for this PR.
- `@coderabbitai generate sequence diagram` to generate a sequence diagram of the changes in this PR.
- `@coderabbitai resolve` resolve all the CodeRabbit review comments.
- `@coderabbitai configuration` to show the current CodeRabbit configuration for the repository.
- `@coderabbitai help` to get help.

### Other keywords and placeholders

- Add `@coderabbitai ignore` anywhere in the PR description to prevent this PR from being reviewed.
- Add `@coderabbitai summary` to generate the high-level summary at a specific location in the PR description.
- Add `@coderabbitai` anywhere in the PR title to generate the title automatically.

### CodeRabbit Configuration File (`.coderabbit.yaml`)

- You can programmatically configure CodeRabbit by adding a `.coderabbit.yaml` file to the root of your repository.
- Please see the [configuration documentation](https://docs.coderabbit.ai/guides/configure-coderabbit) for more information.
- If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: `# yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json`

### Documentation and Community

- Visit our [Documentation](https://docs.coderabbit.ai) for detailed information on how to use CodeRabbit.
- Join our [Discord Community](http://discord.gg/coderabbit) to get help, request features, and share feedback.
- Follow us on [X/Twitter](https://twitter.com/coderabbitai) for updates and announcements.

</details>

<!-- tips_end -->

Andrei Kvapil (kvaps) added a commit to cozystack/cozystack that referenced this pull request Jul 2, 2025
…1128)

ref to #1127,
clastix/kamaji#856 and
cozystack/etcd-operator#291

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **New Features**
  * Updated etcd chart to version 2.9.0.
* **Improvements**
* Simplified etcd endpoint configuration to use a single static
endpoint.
* Expanded TLS certificate DNS names to include additional service
addresses.
  * Streamlined resource configuration for etcd deployment.
* **Chores**
  * Updated version mapping for etcd package.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.