Skip to main content
Version: 3.6 (unsupported)

Collecting logs from Scalar products on a Kubernetes cluster

This document explains how to deploy Grafana Loki and Promtail on Kubernetes with Helm. After following this document, you can collect logs of Scalar products on your Kubernetes environment.

If you use a managed Kubernetes cluster and you want to use the cloud service features for monitoring and logging, please refer to the following document.

Prerequisites​

Add the grafana helm repository​

This document uses Helm for the deployment of Prometheus Operator.

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update

Prepare a custom values file​

Please get the sample file scalar-loki-stack-custom-values.yaml for loki-stack. For the logging of Scalar products, this sample file's configuration is recommended.

In the production environment, it is recommended to add labels to the worker node for Scalar products as follows.

Since the promtail pods deployed in this document collect only Scalar product logs, it is sufficient to deploy promtail pods only on the worker node where Scalar products are running. So, you should set nodeSelector in the custom values file (scalar-loki-stack-custom-values.yaml) as follows if you add labels to your Kubernetes worker node.

  • ScalarDB Cluster Example

    promtail:
      nodeSelector:
        scalar-labs.com/dedicated-node: scalardb-cluster

  • (Deprecated) ScalarDB Server Example

    promtail:
      nodeSelector:
        scalar-labs.com/dedicated-node: scalardb

  • ScalarDL Ledger Example

    promtail:
      nodeSelector:
        scalar-labs.com/dedicated-node: scalardl-ledger

  • ScalarDL Auditor Example

    promtail:
      nodeSelector:
        scalar-labs.com/dedicated-node: scalardl-auditor

In the production environment, it is recommended to add taints to the worker node for Scalar products as follows.

Since promtail pods are deployed as DaemonSet, you must set tolerations in the custom values file (scalar-loki-stack-custom-values.yaml) as follows if you add taints to your Kubernetes worker node.

  • ScalarDB Cluster Example

    promtail:
      tolerations:
        - effect: NoSchedule
          key: scalar-labs.com/dedicated-node
          operator: Equal
          value: scalardb-cluster

  • (Deprecated) ScalarDB Server Example

    promtail:
      tolerations:
        - effect: NoSchedule
          key: scalar-labs.com/dedicated-node
          operator: Equal
          value: scalardb

  • ScalarDL Ledger Example

    promtail:
      tolerations:
        - effect: NoSchedule
          key: scalar-labs.com/dedicated-node
          operator: Equal
          value: scalardl-ledger

  • ScalarDL Auditor Example

    promtail:
      tolerations:
        - effect: NoSchedule
          key: scalar-labs.com/dedicated-node
          operator: Equal
          value: scalardl-auditor

Deploy Loki and Promtail​

It is recommended to deploy Loki and Promtail on the same namespace monitoring as Prometheus and Grafana. You have already created the monitoring namespace in the document Monitoring Scalar products on the Kubernetes cluster.

helm install scalar-logging-loki grafana/loki-stack -n monitoring -f scalar-loki-stack-custom-values.yaml

Check if Loki and Promtail are deployed​

If the Loki and Promtail pods are deployed properly, you can see the STATUS is Running using the kubectl get pod -n monitoring command. Since promtail pods are deployed as DaemonSet, the number of promtail pods depends on the number of Kubernetes nodes. In the following example, there are three worker nodes for Scalar products in the Kubernetes cluster.

$ kubectl get pod -n monitoring
NAME                                 READY   STATUS    RESTARTS   AGE
scalar-logging-loki-0                1/1     Running   0          35m
scalar-logging-loki-promtail-2fnzn   1/1     Running   0          32m
scalar-logging-loki-promtail-2pwkx   1/1     Running   0          30m
scalar-logging-loki-promtail-gfx44   1/1     Running   0          32m

View log in Grafana dashboard​

You can see the collected logs in the Grafana dashboard as follows.

  1. Access the Grafana dashboard
  2. Go to the Explore page
  3. Select Loki from the top left pull-down
  4. Set conditions to query logs
  5. Select the Run query button at the top right

Please refer to the Monitoring Scalar products on the Kubernetes cluster for more details on how to access the Grafana dashboard.