Skip to content

Monitor the Logs of a Worker

This section involves the steps to observe the logs of the Worker on different endpoints such as DataOS CLI, Metis UI, and Operations App.

DataOS CLI

To monitor the logs of a Worker using DataOS CLI, follow the steps below:

  1. On DataOS CLI, execute the following command by replacing the placeholders with the actual values. 

    dataos-ctl log -t worker -w ${{workspace-name}} -n ${{workflow-name}}

    Example Usage:

    dataos-ctl log -t worker -w public -n textile-insights-worker
INFO[0000] 📃 log(public)...                             
INFO[0001] 📃 log(public)...complete                     

              NODE NAME             │     CONTAINER NAME      │ ERROR  
────────────────────────────────────┼─────────────────────────┼────────
  textile-insights-worker-gqjk-ss-0 │ textile-insights-worker │        

-------------------LOGS-------------------
🚀 Lens2 (0.35.60-20) => DataOS:dataos-training.dataos.app
DEBUG: Configured time-zones: [
  "America/Toronto",
  "America/Vancouver",
  "UTC"
]
DEBUG: source: {
  "type": "flash",
  "dialect": "duckdb",
  "connection": "flash",
  "meta": {
    "userId": "abhishekgupta",
    "host": "textile-flash.public.svc.cluster.local",
    "port": 5433
  }
}
DEBUG: 🧑‍🤝‍🧑 /etc/dataos/work/dp-automation/demo_1/textile-analytics/data-product/consumer-aligned/build/semantic-model-view/model/user_groups.yml => [
  {
    "name": "default",
    "description": "Administrators with full access to textile analytics data product",
    "includes": "*",
    "api_scopes": [
      "meta",
      "data",
      "graphql",
      "jobs",
      "source"
    ],
    "allow_private_members_access": false
  }
]

Metis UI

To monitor the logs of a Worker on the Metis Catalog UI, follow the steps below:

  1. Open the Metis Catalog.

    Open the Metis Catalog

  2. Search for the Worker by name.

    Search the Worker in Metis

  3. Click on the Worker that needs to be monitored and navigate to the ‘Runtime’ section.

    Worker runtime in Metis

  4. Click on any pod name for which you want to monitor the logs, and navigate to the ‘Pod Logs’ section.

    Pod Logs in Metis

Operations App

To monitor the logs of a Worker on the Operations App, follow the steps below:

  1. Open the Operations app.

    Open the Operations app

  2. Navigate to User Space → Resources → Worker and search for the Worker by name.

    Operations > Resources > Worker

  3. Click on the Worker that needs to be monitored and navigate to the ‘Resource Runtime’ section.

    Worker resource runtime in Operations

  4. Click on any runtime node for which you want to monitor the logs, and navigate to the ‘Runtime Node Logs’ section. Here, users can monitor the init container and main container logs.

    Runtime Node Logs in Operations