diff --git a/content/docs/self-hosted/installation/distributed/k8s-helm-enterprise.mdx b/content/docs/self-hosted/installation/distributed/k8s-helm-enterprise.mdx index 20160ca..57a8dd1 100644 --- a/content/docs/self-hosted/installation/distributed/k8s-helm-enterprise.mdx +++ b/content/docs/self-hosted/installation/distributed/k8s-helm-enterprise.mdx @@ -2,18 +2,19 @@ title: Kubernetes (Enterprise) description: Install Parseable Enterprise in distributed mode on Kubernetes with Helm. --- +import { Step, Steps } from 'fumadocs-ui/components/steps'; -This guide installs Parseable Enterprise in distributed mode on Kubernetes. The installation creates ingestor, querier, and Prism nodes. +This guide installs Parseable Enterprise in distributed mode on Kubernetes. The installation creates ingestor, querier, and Prism pods. To install Parseable OSS, use the [distributed OSS Kubernetes guide](/docs/self-hosted/installation/distributed/k8s-helm-oss). -## 1. Prerequisites +## Prerequisites Before you begin, make sure you have: -- `kubectl` configured for your Kubernetes cluster +- `kubectl` installed and configured for the Kubernetes cluster where you want to install Parseable - Helm 3 or later - A Kubernetes storage class - An existing, empty object-store bucket with access credentials @@ -29,13 +30,26 @@ kubectl get nodes kubectl get storageclass ``` -## 2. Create the namespace +The `kubectl config current-context` output should point to the cluster where +you want to install Parseable. Use one of the storage class names returned by +`kubectl get storageclass` in the values file below. + + + + +## 1. Create the namespace ```bash kubectl create namespace parseable ``` -## 3. Choose an object store +If you are testing in a shared cluster, use a separate namespace for your test +installation and replace `parseable` in the commands below with that namespace. + + + + +## 2. Choose an object store Choose exactly one object store below. Replace every `<...>` placeholder in the selected option before running the command. @@ -84,7 +98,7 @@ EOF that match your MinIO deployment. -Use `s3-store` in [`enterprise-values.yaml`](#7-create-enterprise-valuesyaml). +Use `s3-store` in [`enterprise-values.yaml`](#6-create-enterprise-valuesyaml). ### Option B: Amazon S3 @@ -109,7 +123,7 @@ EOF [S3 IRSA guide](/docs/self-hosted/storage-targets/aws-s3#kubernetes-with-irsa). -Use `s3-store` in [`enterprise-values.yaml`](#7-create-enterprise-valuesyaml). +Use `s3-store` in [`enterprise-values.yaml`](#6-create-enterprise-valuesyaml). ### Option C: Google Cloud Storage @@ -131,7 +145,7 @@ Keep the service-account JSON file ready. You will create its Kubernetes Secret The service-account JSON file is a credential. Store it securely and never commit it to source control. -Use `gcs-store` in [`enterprise-values.yaml`](#7-create-enterprise-valuesyaml). +Use `gcs-store` in [`enterprise-values.yaml`](#6-create-enterprise-valuesyaml). ### Option D: Azure Blob Storage @@ -149,9 +163,12 @@ azr.url= EOF ``` -Use `blob-store` in [`enterprise-values.yaml`](#7-create-enterprise-valuesyaml). +Use `blob-store` in [`enterprise-values.yaml`](#6-create-enterprise-valuesyaml). + + + -## 4. Create the Kubernetes Secret +## 3. Create the Kubernetes Secret After creating the file for your selected object store, review it and create the Kubernetes Secret: @@ -179,7 +196,10 @@ After confirming that the Kubernetes Secret exists, delete the local plaintext f rm parseable-env-secret ``` -## 5. Create Enterprise Secrets + + + +## 4. Create Enterprise Secrets Create the license Secret. Replace `` with the directory containing both license files: @@ -190,7 +210,7 @@ kubectl create secret generic parseable-license \ --from-file=parseable_license.sig='/parseable_license.sig' ``` -Create a random 16-character shared Secret for communication between Enterprise nodes: +Create a random 16-character shared Secret for communication between Enterprise pods: ```bash kubectl create secret generic parseable-cluster-secret \ @@ -198,14 +218,20 @@ kubectl create secret generic parseable-cluster-secret \ --from-literal=cluster-secret="$(openssl rand -hex 8)" ``` -## 6. Add the Parseable Helm repository + + + +## 5. Add the Parseable Helm repository ```bash helm repo add parseable https://charts.parseable.com helm repo update parseable ``` -## 7. Create `enterprise-values.yaml` + + + +## 6. Create `enterprise-values.yaml` Create a file named `enterprise-values.yaml`: @@ -276,7 +302,7 @@ parseable: enabled: true storageClass: accessMode: ReadWriteOnce - size: 100Gi + size: 50Gi prism: replicas: 1 @@ -301,7 +327,29 @@ Before continuing: 3. Replace both `` placeholders with a storage class returned by `kubectl get storageclass`. 4. Adjust replicas, CPU, memory, and volume sizes for your workload. -## 8. Install Parseable Enterprise +You can render the chart before installing it: + +```bash +helm template parseable parseable/parseable \ + --namespace parseable \ + --values ./enterprise-values.yaml +``` + +This catches invalid storage combinations before anything is created in the +cluster. + +### Storage behavior + +- Enterprise distributed mode requires S3, GCS, or Azure Blob Storage. It does not support `local-store`. +- Ingestor `staging.enabled: true` creates one staging PVC per ingestor. `false` uses `emptyDir`, so pending staged data is lost if the Pod is replaced. +- Querier `hotTier.enabled: true` creates one hot-tier PVC per querier. `false` uses `emptyDir`. Hot tier is a cache and can rebuild from the object store. +- Choose both settings during installation. To change one later, recreate the affected StatefulSet. +- Kubernetes rejects changing a volume claim template's `size` on an existing StatefulSet. If the StorageClass supports volume expansion, resize each generated PVC directly. + + + + +## 7. Install Parseable Enterprise ```bash helm upgrade --install parseable parseable/parseable \ @@ -311,16 +359,24 @@ helm upgrade --install parseable parseable/parseable \ --timeout 10m ``` -## 9. Verify the installation + + + +## 8. Verify the installation ```bash helm status parseable --namespace parseable kubectl get pods,pvc,services,statefulsets --namespace parseable ``` -The ingestor, querier, and Prism pods should be `Running`, and all PVCs should be `Bound`. +The ingestor, querier, and Prism pods should be `Running`, and all PVCs should +be `Bound`. With the sample values above, one staging PVC is created for each +ingestor replica and one hot-tier PVC is created for each querier replica. + + + -## 10. Access Parseable +## 9. Access Parseable Forward the Prism service: @@ -338,7 +394,10 @@ Send ingestion traffic inside the cluster to: http://parseable-ingestor-service.parseable.svc.cluster.local ``` -## 11. Troubleshooting + + + +## 10. Troubleshooting Start with Prism, then the querier, and then the ingestors: @@ -354,3 +413,6 @@ kubectl logs statefulset/parseable-ingestor --namespace parseable - `CreateContainerConfigError` usually indicates a missing Secret or Secret key. - Object-store authentication errors usually indicate incorrect credentials, permissions, endpoint, bucket, or container values. - Use an empty bucket or container for a new Parseable cluster. Metadata from another Parseable cluster can cause credential or deployment identity conflicts. + + + diff --git a/content/docs/self-hosted/installation/distributed/k8s-helm-oss.mdx b/content/docs/self-hosted/installation/distributed/k8s-helm-oss.mdx index 120677b..cc0c3ea 100644 --- a/content/docs/self-hosted/installation/distributed/k8s-helm-oss.mdx +++ b/content/docs/self-hosted/installation/distributed/k8s-helm-oss.mdx @@ -7,16 +7,17 @@ redirect_from: - /installation/kubernetes-helm - /installation/setup-parseable-on-kubernetes-via-helm --- +import { Step, Steps } from 'fumadocs-ui/components/steps'; -This guide installs Parseable OSS in distributed mode on Kubernetes. The installation creates separate ingestor and querier nodes. +This guide installs Parseable OSS in distributed mode on Kubernetes. The installation creates separate ingestor and querier pods. To install Parseable Enterprise, use the [distributed Enterprise Kubernetes guide](/docs/self-hosted/installation/distributed/k8s-helm-enterprise). -## 1. Prerequisites +## Prerequisites Before you begin, make sure you have: -- `kubectl` configured for your Kubernetes cluster +- `kubectl` installed and configured for the Kubernetes cluster where you want to install Parseable - Helm 3 or later - A Kubernetes storage class - An existing, empty object-store bucket or container @@ -30,13 +31,26 @@ kubectl get nodes kubectl get storageclass ``` -## 2. Create the namespace +The `kubectl config current-context` output should point to the cluster where +you want to install Parseable. Use one of the storage class names returned by +`kubectl get storageclass` in the values file below. + + + + +## 1. Create the namespace ```bash kubectl create namespace parseable ``` -## 3. Choose an object store +If you are testing in a shared cluster, use a separate namespace for your test +installation and replace `parseable` in the commands below with that namespace. + + + + +## 2. Choose an object store Choose exactly one object store below. Replace every `<...>` placeholder in the selected option before running the command. @@ -85,7 +99,7 @@ EOF that match your MinIO deployment. -Use `s3-store` in [`oss-values.yaml`](#6-create-oss-valuesyaml). +Use `s3-store` in [`oss-values.yaml`](#5-create-oss-valuesyaml). ### Option B: Amazon S3 @@ -110,7 +124,7 @@ EOF [S3 IRSA guide](/docs/self-hosted/storage-targets/aws-s3#kubernetes-with-irsa). -Use `s3-store` in [`oss-values.yaml`](#6-create-oss-valuesyaml). +Use `s3-store` in [`oss-values.yaml`](#5-create-oss-valuesyaml). ### Option C: Google Cloud Storage @@ -132,7 +146,7 @@ Keep the service-account JSON file ready. You will create its Kubernetes Secret The service-account JSON file is a credential. Store it securely and never commit it to source control. -Use `gcs-store` in [`oss-values.yaml`](#6-create-oss-valuesyaml). +Use `gcs-store` in [`oss-values.yaml`](#5-create-oss-valuesyaml). ### Option D: Azure Blob Storage @@ -150,9 +164,12 @@ azr.url= EOF ``` -Use `blob-store` in [`oss-values.yaml`](#6-create-oss-valuesyaml). +Use `blob-store` in [`oss-values.yaml`](#5-create-oss-valuesyaml). + + + -## 4. Create the Kubernetes Secret +## 3. Create the Kubernetes Secret After creating the file for your selected object store, review it and create the Kubernetes Secret: @@ -180,14 +197,20 @@ After confirming that the Kubernetes Secret exists, delete the local plaintext f rm parseable-env-secret ``` -## 5. Add the Parseable Helm repository + + + +## 4. Add the Parseable Helm repository ```bash helm repo add parseable https://charts.parseable.com helm repo update parseable ``` -## 6. Create `oss-values.yaml` + + + +## 5. Create `oss-values.yaml` Create a file named `oss-values.yaml`: @@ -254,7 +277,29 @@ Before continuing: 3. Replace `` with a storage class returned by `kubectl get storageclass`. 4. Adjust replicas, CPU, memory, and the staging volume size for your workload. -## 7. Install Parseable +You can render the chart before installing it: + +```bash +helm template parseable parseable/parseable \ + --namespace parseable \ + --values ./oss-values.yaml +``` + +This catches invalid storage combinations before anything is created in the +cluster. + +### Storage behavior + +- Distributed mode requires S3, GCS, or Azure Blob Storage. It does not support `local-store`. +- `staging.enabled: true` creates one staging PVC for each ingestor. +- `staging.enabled: false` uses `emptyDir`. Pending staged data is lost if the ingestor Pod is replaced. +- Choose the staging setting during installation. To change it later, recreate the ingestor StatefulSet. +- Kubernetes rejects changing the staging volume claim template's `size` on an existing StatefulSet. If the StorageClass supports volume expansion, resize each generated PVC directly. + + + + +## 6. Install Parseable ```bash helm upgrade --install parseable parseable/parseable \ @@ -264,16 +309,24 @@ helm upgrade --install parseable parseable/parseable \ --timeout 10m ``` -## 8. Verify the installation + + + +## 7. Verify the installation ```bash helm status parseable --namespace parseable kubectl get pods,pvc,services,statefulsets --namespace parseable ``` -All ingestor and querier pods should be `Running`, and all PVCs should be `Bound`. +All ingestor and querier pods should be `Running`, and all PVCs should be +`Bound`. With the sample values above, one staging PVC is created for each +ingestor replica. + + + -## 9. Access Parseable +## 8. Access Parseable Forward the querier service: @@ -291,7 +344,10 @@ Send ingestion traffic inside the cluster to: http://parseable-ingestor-service.parseable.svc.cluster.local ``` -## 10. Troubleshooting + + + +## 9. Troubleshooting ```bash kubectl get events --namespace parseable --sort-by=.lastTimestamp @@ -303,3 +359,6 @@ kubectl logs statefulset/parseable-ingestor --namespace parseable - `CreateContainerConfigError` usually indicates a missing Secret or Secret key. - Object-store authentication errors usually indicate incorrect credentials, permissions, endpoint, bucket, or container values. - Use an empty bucket or container for a new Parseable cluster. Metadata from another Parseable cluster can cause credential or deployment identity conflicts. + + + diff --git a/content/docs/self-hosted/installation/standalone/k8s-oss.mdx b/content/docs/self-hosted/installation/standalone/k8s-oss.mdx index 7869526..cd9c635 100644 --- a/content/docs/self-hosted/installation/standalone/k8s-oss.mdx +++ b/content/docs/self-hosted/installation/standalone/k8s-oss.mdx @@ -4,6 +4,7 @@ description: Install Parseable OSS in standalone mode on Kubernetes with Helm. redirect_from: - /installation/standalone/k8s --- +import { Step, Steps } from 'fumadocs-ui/components/steps'; This guide installs Parseable OSS as a single standalone pod on Kubernetes. You can use local storage, MinIO, Amazon S3, Google Cloud Storage, or Azure Blob Storage. @@ -12,11 +13,11 @@ This guide installs Parseable OSS as a single standalone pod on Kubernetes. You use the [distributed Enterprise Kubernetes guide](/docs/self-hosted/installation/distributed/k8s-helm-enterprise). -## 1. Prerequisites +## Prerequisites Before you begin, make sure you have: -- `kubectl` configured for your Kubernetes cluster +- `kubectl` installed and configured for the Kubernetes cluster where you want to install Parseable - Helm 3 or later - A Kubernetes storage class - An existing, empty bucket or container with access credentials if you use an object store @@ -29,13 +30,26 @@ kubectl get nodes kubectl get storageclass ``` -## 2. Create the namespace +The `kubectl config current-context` output should point to the cluster where +you want to install Parseable. Use one of the storage class names returned by +`kubectl get storageclass` in the values file below. + + + + +## 1. Create the namespace ```bash kubectl create namespace parseable ``` -## 3. Choose a storage option +If you are testing in a shared cluster, use a separate namespace for your test +installation and replace `parseable` in the commands below with that namespace. + + + + +## 2. Choose a storage option Choose exactly one storage option below. Replace every `<...>` placeholder in the selected option before creating the Kubernetes Secret. @@ -53,7 +67,7 @@ password= EOF ``` -Use `local-store` in [`standalone-values.yaml`](#6-create-standalone-valuesyaml). +Use `local-store` in [`standalone-values.yaml`](#5-create-standalone-valuesyaml). ### Option B: MinIO @@ -97,7 +111,7 @@ EOF administrator credentials before using this setup outside a test cluster. -Use `s3-store` in [`standalone-values.yaml`](#6-create-standalone-valuesyaml). +Use `s3-store` in [`standalone-values.yaml`](#5-create-standalone-valuesyaml). ### Option C: Amazon S3 @@ -122,7 +136,7 @@ EOF [S3 IRSA guide](/docs/self-hosted/storage-targets/aws-s3#kubernetes-with-irsa). -Use `s3-store` in [`standalone-values.yaml`](#6-create-standalone-valuesyaml). +Use `s3-store` in [`standalone-values.yaml`](#5-create-standalone-valuesyaml). ### Option D: Google Cloud Storage @@ -145,7 +159,7 @@ Keep the service-account JSON file ready. You will create its Kubernetes Secret commit it to source control. -Use `gcs-store` in [`standalone-values.yaml`](#6-create-standalone-valuesyaml). +Use `gcs-store` in [`standalone-values.yaml`](#5-create-standalone-valuesyaml). ### Option E: Azure Blob Storage @@ -163,9 +177,12 @@ azr.url= EOF ``` -Use `blob-store` in [`standalone-values.yaml`](#6-create-standalone-valuesyaml). +Use `blob-store` in [`standalone-values.yaml`](#5-create-standalone-valuesyaml). + + + -## 4. Create the Kubernetes Secret +## 3. Create the Kubernetes Secret Review the file created for your selected storage option, then create the Secret: @@ -200,14 +217,20 @@ Delete the local plaintext configuration file after creating the Secret: rm parseable-env-secret ``` -## 5. Add the Parseable Helm repository + + + +## 4. Add the Parseable Helm repository ```bash helm repo add parseable https://charts.parseable.com helm repo update parseable ``` -## 6. Create `standalone-values.yaml` + + + +## 5. Create `standalone-values.yaml` Create a file named `standalone-values.yaml`: @@ -251,7 +274,6 @@ parseable: accessMode: ReadWriteOnce size: 5Gi data: - # Keep enabled for local-store. Set to false for object stores. enabled: true storageClass: accessMode: ReadWriteOnce @@ -264,14 +286,43 @@ the [OIDC guide](/docs/user-guide/openid). Before continuing: 1. Set `parseable.store.type` for the storage option selected in step 3. -2. Keep `data.enabled: true` for `local-store`; set it to `false` for an object store. -3. If you selected Google Cloud Storage, uncomment the `gcsCredentials` block. -4. Replace both `` placeholders with a storage class returned by `kubectl get storageclass`. -5. Adjust volume sizes for your workload. +2. If you selected Google Cloud Storage, uncomment the `gcsCredentials` block. +3. Replace each enabled volume's `` with a storage class returned by `kubectl get storageclass`. +4. Adjust volume sizes for your workload. + +You can render the chart before installing it: + +```bash +helm template parseable parseable/parseable \ + --namespace parseable \ + --values ./standalone-values.yaml +``` -The staging volume temporarily buffers incoming data. The data volume stores permanent data only when `local-store` is selected; object-store deployments keep permanent data in the selected bucket or container. +This catches invalid storage combinations before anything is created in the +cluster. -## 7. Install Parseable +### Storage behavior + +- `staging.enabled: true` creates a staging PVC. `false` uses `emptyDir`, so pending staged data is lost if the Pod is replaced. +- With `local-store`, `data.enabled: true` creates a data PVC. `false` uses `emptyDir`, so stored data is lost if the Pod is replaced. +- If local-store data is disabled, staging must also be disabled. The chart rejects persistent staging with temporary local data. +- With S3, GCS, or Azure Blob Storage, permanent data is stored in the object store. The chart does not create or mount a data volume. +- Choose between PVC and `emptyDir` during installation. Changing that choice later requires recreating the StatefulSet. +- Kubernetes rejects changing a volume claim template's `size` on an existing StatefulSet. If the StorageClass supports volume expansion, resize the generated PVC directly. + + + **Upgrading to chart 3.1.0** + + New installations are not affected. If standalone Parseable was installed + with chart 3.0.2 or earlier, do not upgrade it directly. Create a separate + installation with chart 3.1.0, migrate and verify the data, and then switch + traffic to the new installation. + + + + + +## 6. Install Parseable ```bash helm upgrade --install parseable parseable/parseable \ @@ -281,16 +332,27 @@ helm upgrade --install parseable parseable/parseable \ --timeout 10m ``` -## 8. Verify the installation + + + +## 7. Verify the installation ```bash helm status parseable --namespace parseable -kubectl get pods,pvc,services,deployments --namespace parseable +kubectl get pods,pvc,services,statefulsets --namespace parseable ``` -The Parseable pod should be `Running`, and every enabled PVC should be `Bound`. +The Parseable pod should be `Running`, and every created PVC should be `Bound`. +With the release name `parseable`, a standalone `local-store` install with both +staging and data persistence enabled creates two PVCs: + +- `stage-volume-parseable-standalone-0` +- `data-volume-parseable-standalone-0` + + + -## 9. Access Parseable +## 8. Access Parseable ```bash kubectl port-forward service/parseable-standalone-service 8000:80 --namespace parseable @@ -298,15 +360,21 @@ kubectl port-forward service/parseable-standalone-service 8000:80 --namespace pa Open `http://localhost:8000` and sign in with the username and password stored in `parseable-env-secret`. -## 10. Troubleshooting + + + +## 9. Troubleshooting ```bash kubectl get events --namespace parseable --sort-by=.lastTimestamp kubectl describe pod --namespace parseable -l app.kubernetes.io/instance=parseable -kubectl logs deployment/parseable-standalone --namespace parseable --tail=200 +kubectl logs statefulset/parseable-standalone --namespace parseable --tail=200 ``` - `Pending` PVCs usually indicate an invalid storage class or unavailable volume provisioner. - `CreateContainerConfigError` usually indicates a missing Secret or Secret key. - Object-store authentication errors usually indicate incorrect credentials, permissions, endpoint, bucket, or container values. - Use an empty bucket or container for a new Parseable installation. + + +