u/csantve

Overview

Writes

Working dashboard

I don't know what happened, out of nowhere mimir dashboard data is missing. Not only the overview but other mimir dashboards as well. I'm using the helm chart mimir-distributed. The values are just limits and nothing special. These are my monitoring values though.

metaMonitoring:
  dashboards:
    enabled: true
    labels:
      grafana_dashboard: "1"
  serviceMonitor:
    enabled: true
    interval: 15s

Could I be missing something? Any help would be appreciated.

Greetings. I just started setting up the LGTM stack on my k8s cluster using alloy and I'm using the mimir-distributed helm chart for HA, but the small sample values are for 1M+ series and request a lot of memory (60GB+) so I tried reducing the overall requests. I wonder if I'm missing anything else or if something will break eventually. These are my helm values. I haven't touched any of mimir's parameters, only set the storage backend to S3.

alertmanager:
  persistentVolume:
    enabled: true
  replicas: 2
  resources:
    limits:
      memory: 256Mi
    requests:
      cpu: 50m
      memory: 128Mi
  statefulSet:
    enabled: true

compactor:
  persistentVolume:
    size: 5Gi
  resources:
    limits:
      memory: 1Gi
    requests:
      cpu: 100m
      memory: 512Mi

distributor:
  replicas: 2
  resources:
    limits:
      memory: 256Mi
    requests:
      cpu: 100m
      memory: 128Mi

ingester:
  persistentVolume:
    size: 10Gi
  replicas: 3
  resources:
    limits:
      memory: 1Gi
    requests:
      cpu: 200m
      memory: 512Mi
  topologySpreadConstraints: {}
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/component
                operator: In
                values:
                  - ingester
          topologyKey: "kubernetes.io/hostname"

  zoneAwareReplication:
    topologyKey: "kubernetes.io/hostname"

chunks-cache:
  enabled: false

index-cache:
  enabled: false

metadata-cache:
  enabled: false

results-cache:
  enabled: false

minio:
  enabled: false

overrides_exporter:
  replicas: 1
  resources:
    limits:
      memory: 128Mi
    requests:
      cpu: 50m
      memory: 64Mi

querier:
  replicas: 2
  resources:
    limits:
      memory: 512Mi
    requests:
      cpu: 100m
      memory: 256Mi

query_frontend:
  replicas: 2
  resources:
    limits:
      memory: 256Mi
    requests:
      cpu: 100m
      memory: 128Mi

ruler:
  replicas: 2
  resources:
    limits:
      memory: 512Mi
    requests:
      cpu: 100m
      memory: 256Mi

store_gateway:
  persistentVolume:
    size: 10Gi
  replicas: 3
  resources:
    limits:
      memory: 512Mi
    requests:
      cpu: 100m
      memory: 256Mi
  topologySpreadConstraints: {}
  affinity:
    podAntiAffinity:
      requiredDuringSchedulingIgnoredDuringExecution:
        - labelSelector:
            matchExpressions:
              - key: app.kubernetes.io/component
                operator: In
                values:
                  - store-gateway
          topologyKey: "kubernetes.io/hostname"
  zoneAwareReplication:
    topologyKey: "kubernetes.io/hostname"

gateway:
  replicas: 1
  resources:
    limits:
      memory: 731Mi
    requests:
      cpu: 1
      memory: 512Mi

I think I should also increase kafka's replicas as it is a single point of failure currently, or disable it altogether and just use grpc.

Help would be appreciated.

I have a few ARM VPSs from netcup and they've been running really nice and fast. But I know they oversell (is that the correct term?) their VPSs as I've been seeing some cpu steal, so I've been wondering if they oversell too much.

Keep in mind I like them and their prices and would get ARM root servers if they sold them, but I've been wondering that.