Deploy and Operate Elasticsearch
Elasticsearch is a distributed search
and analytics engine. Edka deploys it through the Elastic Cloud on Kubernetes
(ECK) operator into the elastic-system namespace of your cluster, with a
persistent volume per node, operator-issued TLS on the REST API, and an
optional Kibana instance.
Edka does not install a Helm chart for this app. It applies Elasticsearch and Kibana custom resources and lets the operator reconcile the underlying StatefulSet, Services, and certificates.
Architecture
Section titled “Architecture”| Component | Purpose | How Edka connects it |
|---|---|---|
| Elastic Operator | Reconciles the Elasticsearch and Kibana resources, certificates, and rolling changes | Installed as a required add-on before the app |
| cert-manager | Certificate lifecycle automation in the cluster | Installed as a required add-on before the app |
| Elasticsearch | One default nodeSet carrying the master, data, ingest, ml, remote_cluster_client, and transform roles | Elasticsearch resource named elasticsearch in elastic-system |
| Data volumes | Index and shard data, one volume per node | PersistentVolumeClaim per node from the selected storage class |
| Kibana | Web console and dashboards for the cluster | Kibana resource referencing the elasticsearch cluster, optional |
| Envoy Gateway | Public HTTPS access to Kibana | Optional: select a traffic class and a Kibana hostname |
Prerequisites
Section titled “Prerequisites”Before installing Elasticsearch, prepare:
- an Edka cluster with enough capacity for the node count, per-node memory, and per-node CPU you plan to select
- a storage class for the data volumes; the default is
hcloud-volumes - for Kibana exposure, an Envoy Gateway traffic class and a DNS hostname that resolves through it
Elasticsearch declares cert-manager and the Elastic Operator as required add-ons. Edka queues them during installation, so you do not have to install them first. See Add-ons, Elastic Operator, and cert-manager.
Install Elasticsearch
Section titled “Install Elasticsearch”- Open your cluster and select Apps.
- Find Elasticsearch and select Install.
- In Elasticsearch, set Version, Node Count, Memory Limit per Node, CPU Limit per Node, and Java Heap Size.
- In Kibana, keep or clear Enable Kibana. With Kibana enabled, set Elastic User Password, Kibana Memory Limit, and Kibana CPU Limit.
- In Storage, choose Storage Size per Node and Storage Class.
- In Access, decide whether to enable Expose Kibana via Gateway and, if so, choose the Traffic Class, the Kibana Hostname, and the TLS certificate options.
- Select Install App and follow the operation progress until the cluster is ready.
Version and Storage Class are fixed at install time and cannot be changed from the app configuration form afterwards. Node Count, the resource limits, Java Heap Size, Storage Size per Node, the Kibana settings, and the access settings stay editable.
All resources use fixed names in the elastic-system namespace, so plan for a
single Elasticsearch installation per cluster.
Choose a version
Section titled “Choose a version”Version offers 9.1.5 and 8.19.5. The same version applies to Kibana, so
both stay on one release line.
Size memory, CPU, and the JVM heap
Section titled “Size memory, CPU, and the JVM heap”Memory Limit per Node ranges from 2Gi to 128Gi and defaults to 2Gi.
CPU Limit per Node ranges from 500m to 48000m and defaults to 1000m.
Java Heap Size defaults to 1g and is applied as -Xms and -Xmx through
ES_JAVA_OPTS. Set it to roughly half the memory limit, for example 4g for an
8Gi limit, so the remaining container memory stays available for the
filesystem cache.
Each Elasticsearch container requests the same memory as its limit and 100m
CPU. Kibana uses the same pattern against its own limits.
Elastic User Password
Section titled “Elastic User Password”| Secret | Purpose | Handling |
|---|---|---|
| Elastic User Password | Password of the built-in elastic user, used to sign in to Kibana | Generated at install when Kibana is enabled and stored in the elasticsearch-es-elastic-user Secret in elastic-system |
The field appears only when Enable Kibana is on. Use the generate control to produce a 16-character value, or enter your own.
On a configuration update, leaving Elastic User Password empty keeps the
current password: Edka reads the existing elasticsearch-es-elastic-user Secret
and reuses its elastic value. Entering a new value replaces the password.
With Kibana disabled, the template does not create that Secret and the password
of the elastic user is left to the operator.
To read the password after installation, open the elasticsearch-es-elastic-user
Secret in Cluster Explorer. Secret values are
protected reads and require recent passkey verification when your organization
enforces it.
Cluster Layout and Endpoint
Section titled “Cluster Layout and Endpoint”The Elasticsearch resource defines one nodeSet named default with Node Count
pods. Each pod carries all node roles: master, data, ingest, ml,
remote_cluster_client, and transform. Node Count offers 1, 3, 5, 7, or 9
nodes.
Every pod runs a privileged init container that sets
vm.max_map_count=262144 before Elasticsearch starts, and uses the elastic
ServiceAccount in elastic-system.
Edka publishes the REST API as an internal service endpoint:
https://elasticsearch-es-http.elastic-system.svc.cluster.local:9200The operator issues the certificate for that endpoint, so in-cluster clients that do not trust the operator CA need the CA certificate from the cluster or an explicit trust configuration.
Kibana
Section titled “Kibana”Enable Kibana is on by default and deploys a single Kibana instance wired to
the elasticsearch cluster through elasticsearchRef. Its Service is
kibana-kb-http on port 5601, of type ClusterIP, with the self-signed
certificate disabled on the Kibana HTTP layer.
Kibana ships with Fleet preconfigured but inactive: xpack.fleet.agents.enabled
is false, while the system, elastic_agent, fleet_server, and kubernetes
packages and the Fleet Server and Elastic Agent policies are already defined.
Kibana resources default to 1Gi memory and 1000m CPU. Kibana Memory
Limit offers 1Gi, 2Gi, or 4Gi, and Kibana CPU Limit offers 500m
through 4000m.
Disable Kibana if you only need the REST API. Note that the Elastic User Password field and the Kibana resource fields are hidden while it is off.
Expose Kibana
Section titled “Expose Kibana”Expose Kibana via Gateway is off by default, which keeps Kibana reachable only inside the cluster. Enabling it adds:
- Traffic Class: the Gateway traffic class available in your cluster
- Kibana Hostname: the public hostname, for example
kibana.example.com - Use Cluster Issuer to request TLS Certificates and Cluster Issuer
Name, which defaults to
letsencrypt-http
With an Envoy Gateway traffic class, Edka publishes Kibana as an HTTPRoute attached to that Gateway, and TLS terminates at the Gateway listener. See Gateway API and Domains and TLS.
The hostname is also written into Kibana’s server.publicBaseUrl, so point DNS
at the Gateway before or shortly after installation. Once Kibana and exposure
are both enabled, Edka shows a Kibana Dashboard endpoint at
https://<your-hostname>.
Storage
Section titled “Storage”Storage uses one PersistentVolumeClaim per node, created from the nodeSet’s
volume claim template with the ReadWriteOnce access mode.
| Option | Default | Notes |
|---|---|---|
| Storage Size per Node | 50Gi | 10Gi to 1000Gi; stays editable after installation |
| Storage Class | hcloud-volumes | Selected from the cluster’s storage classes, fixed at install |
Host path storage is not available through the app configuration. Growing Storage Size per Node later requires a storage class that allows volume expansion.
Version Updates
Section titled “Version Updates”This app has no chart auto-update policy. The Elasticsearch and Kibana versions are set at install time from the Version options and are not editable in the configuration form afterwards. The Elastic Operator and cert-manager are managed as cluster add-ons, separately from the app.
Troubleshooting
Section titled “Troubleshooting”| Symptom | Check |
|---|---|
| Managed secret “elasticsearch-es-elastic-user” was not found | The elastic user Secret is missing in elastic-system; enter an explicit Elastic User Password on the update |
| Managed secret “elasticsearch-es-elastic-user” is missing the “elastic” value | The Secret exists without its elastic key; enter an explicit Elastic User Password on the update |
| Pods stay Pending | Verify the storage class can provision one volume per node and that the node pool has the requested memory and CPU |
| Nodes restart under load | Confirm Java Heap Size is roughly half of Memory Limit per Node |
| Kibana stays unready | Confirm the Elasticsearch cluster is ready and review the Kibana Logs for authentication errors |
Kibana sign-in with elastic fails | Read the current password from the elasticsearch-es-elastic-user Secret, or set a new one in the app configuration |
| The Kibana hostname does not resolve | Point DNS at the selected Gateway traffic class and confirm Expose Kibana via Gateway is enabled |