Monitor the Logs of a Workflow

This section involves the steps to observe the logs of the Workflow on different endpoints such as DataOS CLI, Metis UI, and Operations App.

DataOS CLI

To monitor the logs of a Workflow using DataOS CLI, follow the steps below:

1. On DataOS CLI, execute the following command by replacing the placeholders with the actual values.

   dataos-ctl log -t workflow -w ${{workspace-name}} -n ${{workflow-name}}
   

   Example Usage:

   dataos-ctl log -t workflow -w public -n t-sub
   
     NODE NAME │ CONTAINER NAME │ ERROR  
   ────────────┼────────────────┼────────
     t-sub-rd  │ main           │        
   
   -------------------LOGS-------------------
   time="2025-06-12T07:24:28.680Z" level=info msg="Starting Workflow Executor" version=v3.5.11
   time="2025-06-12T07:24:28.683Z" level=info msg="Using executor retry strategy" Duration=1s Factor=1.6 Jitter=0.5 Steps=5
   time="2025-06-12T07:24:28.683Z" level=info msg="Executor initialized" deadline="0001-01-01 00:00:00 +0000 UTC" includeScriptOutput=false namespace=public podName=t-sub-jvsg-kym-1635177909 templateName= version="&Version{Version:v3.5.11,BuildDate:2024-09-20T14:09:00Z,GitCommit:25bbb71cced32b671f9ad35f0ffd1f0ddb8226ee,GitTag:v3.5.11,GitTreeState:clean,GoVersion:go1.21.13,Compiler:gc,Platform:linux/amd64,}"
   time="2025-06-12T07:24:28.706Z" level=info msg="Loading manifest to /tmp/manifest.yaml"
   time="2025-06-12T07:24:28.706Z" level=info msg="kubectl delete --ignore-not-found -f /tmp/manifest.yaml -o name"
   time="2025-06-12T07:24:29.653Z" level=info msg="sub-process exited" argo=true error="<nil>"
   

Metis UI

To monitor the logs of a Workflow on the Metis Catalog UI, follow the steps below:

1. Open the Metis Catalog.

   

   Open the Metis Catalog

2. Search for the Workflow by name.

   

   Search the Workflow in Metis

3. Click on the Workflow that needs to be monitored and navigate to the ‘Run History’ section.

   

   Workflow run history

4. Click on any run name for which you want to monitor the logs, and navigate to the ‘Physical Plan’ section.

   

   Physical plan

5. Under the ‘Physical Plan’ section, click on the pod name.

   

   Select pod

6. Navigate to the pod logs section, where you can access the init and main logs.

   

   Pod logs in Metis

Operations App

To monitor the logs of a Workflow on the Operations App, follow the steps below:

1. Open the Operations app.

   

   Open the Operations app

2. Navigate to User Space → Resources → Workflow and search for the Workflow by name.

   

   Operations > Resources > Workflow

3. Click on the Workflow that needs to be monitored and navigate to the ‘Resource Runtime’ section.

   

   Workflow resource runtime

4. Click on any runtime node for which you want to monitor the logs, and navigate to the ‘Runtime Node Logs’ section. Here, users can monitor the init container and main container logs.

   

   Runtime Node Logs