Monitor

The Monitor Resource is an integral part of DataOS’ Observability system, designed to trigger incidents based on specific events or metrics. By leveraging the Monitor Resource alongside the Pager Resource for sending alerts, DataOS users can achieve comprehensive observability and proactive incident management, ensuring high system reliability and performance. To understand the key concepts associated with a Monitor, refer to the following link: Core Concepts.

Structure of Monitor manifest

SyntaxCode

monitor_manifest_structure.yml

# Resource meta section
name: ${{certificateexpirymonitornew}} # Resource name
version: v1alpha
type: monitor
tags:
  - ${{dataos:type:resource}} # Tags
description: ${{SSL certificate is about to expire less then 24 hrs}} # Resource description
layer: user
runAsUser: ${{iamgroot}} # User ID of User (or use case assignee)
monitor:

# Monitor-specific section
  schedule: ${{'*/2 * * * *'}} # Monitor schedule
  properties:
    ${{alpha: beta}}
  incident: # mandatory
    asset: ${{output_1}}
    column: ${{column_2}}
    name: ${{CertificateExpirydata}}
    severity: ${{high}}
    incidentType: ${{field_profiling}}

# Equation monitor specification
  type: equation_monitor # mandatory
  equation: 
    # LHS
    leftExpression: 
      queryCoefficient: ${{1}} # mandatory
      queryConstant: ${{0}} # mandatory
      query: # mandatory
        type: ${{prom}} # mandatory
        cluster: ${{thanos}}
        description: ${{query description}}
        dsnSecretRef:
          name: ${{secret}} # mandatory
          workspace: ${{sandbox}}
          key: ${{username}}
          keys:
            - ${{username}}
            - ${{password}}
          allkeys: ${{true}}
          consumptionType: ${{string}}
        ql: certmanager_certificate_expiration_timestamp_seconds{container="cert-manager-controller", endpoint="9402", exported_namespace="caretaker", instance="10.212.4.9:9402", issuer_group="cert-manager.io", issuer_kind="ClusterIssuer", issuer_name="ca", job="cert-manager-ds", name="caretaker-api-cert", namespace="cert-manager", pod="cert-manager-ds-7d8cc489dd-d46sb", service="cert-manager-ds"} - time() # mandatory
        comparisonColumn:  
          name: ${{column1}}
          dataType: ${{string}}

    # RHS
    rightExpression: # mandatory
      queryCoefficient: ${{1}} # mandatory
      queryConstant: ${{7766092}} # mandatory
      query: # mandatory
        type: ${{trino}} # mandatory
        cluster: ${{themis}} # mandatory
        dsn: ${{integer}} 
        dsnSecretRef: 
          name: ${{secret}} # mandatory
          workspace: ${{sandbox}}
          key: ${{username}}
          keys:
            - ${{username}}
            - ${{password}}
          allkeys: ${{true}}
          consumptionType: ${{string}}       
        ql: ${{SELECT metric_value FROM icebase.soda.soda_check_metrics_01 WHERE metric_name = 'missing_count' ORDER BY timestamp DESC LIMIT 1;}} # mandatory
        comparisonColumn: 
          name: ${{column1}} # mandatory
          dataType: ${{integer}} # mandatory      
    # Operator
    operator: ${{less_than}}

First Steps

Monitor Resource in DataOS can be created by applying the manifest file using the DataOS CLI. To learn more about this process, navigate to the link: First steps.

Configuration

Monitors can be configured to autoscale and match varying workload demands, reference pre-defined Secrets and Volumes, and more. The specific configurations may vary depending on the use case. For a detailed breakdown of the configuration options and attributes, please refer to the documentation: Attributes of Monitor manifest.