Skip to content

How to iterate the Data Product?

After successfully deploying the Data Product, it’s time to observe the results if the Data Product incorporates with your use-case or not, If the Data Product does not match the goal then you can iterate the process until your goal is achieved.

Iterate the Data Product

Iterating a Data Product involves refining and enhancing it based on feedback, performance metrics, and evolving business requirements. If you need to iterate your Data Product based on feedback from data consumers, follow these steps:

  • Monitor Service Level Objectives (SLOs): Track Service Level Objectives (SLOs) and receive notifications through your preferred tool, such as email or messaging apps. Additionally, get alerts for incidents like pipeline failures to address issues promptly. Notifications are generated by Pager and will inform you of any problems, enabling you to resolve them quickly. Below is an example of an alert you might receive in a Teams channel when a workflow fails:
  • Implement changes: Update the Data Product according to the issues arises. This might involve modifying data sources, transformation logic, data models, and workflows. To update the Data Product simply make the changes in the resources that require updation and re-apply the bundle resource using the below command:

dataos-ctl apply -f /home/office/bundle.yaml
Output:

INFO[0000] 🛠 apply...                                   
INFO[0000] 🔧 applying data-product-bundle:v1:bundle...   
INFO[0001] 🔧 applying data-product-bundle:v1:bundle...updated 
INFO[0001] 🛠 apply...complete

  • Access in business-friendly ways: After successfull deployment of a Data Product, it can be accessible through SQL interfaces (Workbench), dashboards (Superset), a semantic layer, APIs, or a Data Product Hub for user convenience. To access the Data Product via Workbench you can go to Workbench , select the Cluster, schema, and output dataset and you are good to go.

  • Deploy updated product: Deploy the updated Data Product to the production environment.

  • Comprehensive metadata management: Add or update semantics to maintain accurate and useful metadata for the Data Product.

  • Manage user access: Easily provide and deny access to new users to maintain control over who can interact with the Data Product.

  • Upgrade/Update and retire Products: Easily upgrade or update data products and retire them with one-command deletion to keep the system current and relevant.

By following these steps, you can continuously improve your Data Product to better meet user needs and business objectives.