Data Product Configurations¶

The following attributes are declared for every Data Product deployed in a DataOS context. Some of these attributes/fields need to be mandatorily declared, while others are optional.

Structure of Data Product manifest file¶

# product meta section

name: ${{product-affinity-cross-sell}} # mandatory
version: ${{v1beta}} # mandatory
entity: ${{product}} # mandatory
type: ${{data}} # mandatory
tags:   # optional
  - ${{DPDomain.Sales}}
  - ${{DPDomain.Marketing}}
  - ${{DPUsecase.Customer Segmentation}}
  - ${{DPUsecase.Product Recommendation}}
  - ${{DPTier.DataCOE Approved}}
description: ${{Leverages product affinity analysis to identify cross-sell opportunities, enabling businesses to enhance customer recommendations and drive additional sales by understanding the relationships between products purchased together}} # optional
refs:  # optional
  - title: ${{Workspace Info}} # optional
    href: ${{https://dataos.info/interfaces/cli/command_reference/#workspace}} # mandatory

# data product specific section

v1beta: # mandatory
  data: # mandatory
    meta: # mandatory
      title: ${{Product Affinity & Cross-Sell Opportunity}}
      sourceCodeUrl: ${{https://bitbucket.org/tmdc/product-affinity-cross-sell/src/main/}}
      trackerUrl: ${{https://rubikai.atlassian.net/browse/DPRB-65}}

    collaborators: # optional
      - name: ${{iamgroot}}
        description: ${{developer}}
      - name: ${{iamthor}}
        description: ${{consumer}}

    resource: # mandatory
      refType: ${{dataos}}
      ref: ${{bundle:v1beta:product-affinity-bundle}}
      description: ${{'Ingest data'}}
      purpose: ${{'ingestion'}}

    inputs: # mandatory
      - refType: ${{dataos}}
        ref: ${{dataset:icebase:customer_relationship_management:customer}}
        description: ${{'Ingest data'}}
        purpose: ${{'ingestion'}}

      - refType: ${{dataos}}
        ref: ${{dataset:icebase:customer_relationship_management:purchase}}
        description: ${{'Ingest data'}}
        purpose: ${{'ingestion'}}

      - refType: ${{dataos}}
        ref: ${{dataset:icebase:customer_relationship_management:product}}
        description: ${{'Ingest data'}}
        purpose: ${{'ingestion'}}

    outputs: # optional
      - refType: ${{dataos}}
        ref: ${{dataset:icebase:customer_relationship_management:product_affinity_matrix}}
        description: ${{'Ingest data'}}
        purpose: ${{'ingestion'}}

      - refType: ${{dataos}}
        ref: ${{dataset:icebase:customer_relationship_management:cross_sell_recommendations}}
        description: ${{'Ingest data'}}
        purpose: ${{'ingestion'}}

    ports: # optional
      lens:
        ref: ${{lens:v1alpha:cross-sell-affinity:public}}
        refType: ${{dataos}}

      talos:
        - ref: ${{service:v1:cross-sell-talos:public}}
          refType: ${{dataos}}

Product meta section¶

This section serves as the header of the manifest file, defining the overall characteristics of the Data Product you wish to create. It includes attributes common to all types of Products in DataOS.

`name`¶

Description: Unique identifier for the Product.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	• alpha numeric values with the RegEx `[a-z0-9]([-a-z0-9]*[a-z0-9])`; a hyphen/dash is allowed as a special character • total length of the string should be less than or equal to 48 characters

Example Usage:

name: product-affinity-cross-sell

Data Product card showing the title on the Metis

`version`¶

Description: The version of the Product manifest file

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	v1alpha

Example Usage:

version: v1beta

entity

Description: Indicates the DataOS Entity to which the attributes apply, specified as "product".

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	product, resource, workspace, domain

Example Usage:

entity: product

`type`¶

Description: Indicates the type of product.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	data

Example Usage:

type: data

`tags`¶

Description: Tags associated with the product.

Data Type	Requirement	Default Value	Possible Value
list of strings	optional	none	list of strings

Example Usage:

tags: 
  - DPDomain.Sales
  - DPDomain.Marketing
  - DPUsecase.Customer Segmentation
  - DPUsecase.Product Recommendation
  - DPTier.DataCOE Approved

Following are the predefined tags which you can customize as per the specific Data Product.

Tags	Description
`DPDomain.Sales`	Refers to Data Products related to the sales domain, focusing on sales data and analytics. similarly you can assign any related doamin to the Data Product.
`DPDomain.Marketing`	Pertains to Data Products in the marketing domain, emphasizing customer outreach and campaigns.
`DPUsecase.Customer Segmentation`	Assigns the customer segmentation use case.
`DPUsecase.Product Recommendation`	Assigns the product recommendation use case.
`DPTier.DataCOE Approved`	Assigns the tier to the Data Product. Indicates that the Data Product meets standards set by the Data Center of Excellence.

Data Product card showing the tags on the Data Product Hub

Data Product card showing the tags on the Metis

`description`¶

Description: A brief description of the product.

Data Type	Requirement	Default Value	Possible Value
string	optional	none	any string

Example Usage:

description: Leverages product affinity analysis to identify cross-sell opportunities, enabling businesses to enhance customer recommendations and drive additional sales by understanding the relationships between products purchased together # optional

Data Product card showing the description on the Data Product Hub

Data Product card showing the description on the Metis

owner

Description: The owner of the Product.

Data Type	Requirement	Default Value	Possible Value
string	optional	none	valid user-id

Example Usage:

owner: iamgroot

refs

Description: References associated with the product that can provide the additional information about it.

Data Type	Requirement	Default Value	Possible Value
list of mappings	optional	none	mappings containing title and href address

Example Usage:

refs:
  - title: Lens Info 
    href: https://dataos.info/interfaces/lens/

Data Product card showing the reference links on the Metis

Data Product-specific section¶

Data Product-specific section is different for different versions. This section comprises attributes specific to the Data Product for each version. The attributes within the section are listed below:

`v1beta`¶

Description: Specifies the version of the Data Product.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	v1beta

Example Usage:

v1beta:
  data: # Data Product version details

`data`¶

Description: Contains additional sections related to the Data Product.

Data Type	Requirement	Default Value	Possible Value
object	mandatory	none	Contains `meta`, `collaborators`, `relatedDataProducts`, `resource`, `inputs`, `outputs`, `ports`, and `metrics`.

Example Usage:

data:
  meta:

`meta`¶

Description: An optional section for additional metadata.

Data Type	Requirement	Default Value	Possible Value
object	optional	none	Contains key-value pairs for metadata.

Example Usage:

meta: # mandatory
  title: ${{Product Affinity & Cross-Sell Opportunity}} # optional
  sourceCodeUrl: ${{https://bitbucket.org/tmdc/product-affinity-cross-sell/src/main/}}
  trackerUrl: ${{https://rubikai.atlassian.net/browse/DPRB-65}}

Data Product card showing the title on the Data Product Hub

Data Product card showing the Git and Jira links on the Data Product Hub

Below table describes each attribute of the meta section.

Key	Value	Description
`title`	`${{Product Affinity & Cross-Sell Opportunity}}`	The name of the data product for product affinity and cross-sell opportunities.
`sourceCodeUrl`	`${{https://bitbucket.org/tmdc/product-affinity-cross-sell/src/main/}}`	URL to the source code repository.
`trackerUrl`	`${{https://rubikai.atlassian.net/browse/DPRB-65}}`	URL to the issue tracker for tracking development and tasks.

`collaborators`¶

Description: Specifies the list of collaborators involved with the Data Product.

Data Type	Requirement	Default Value	Possible Value
list of objects	mandatory	none	Each object contains `name` and `description`.

Example Usage:

collaborators:
  - name: ${{iamgroot}}
    description: ${{owner}}
  - name: ${{itsthor}}
    description: ${{developer}}
  - name: ${{itsloki}}
    description: ${{consumer}}

Data Product card showing the collaborators on the Data Product Hub

Data Product card showing the collaborators on the Metis

`name`¶

Description: The name of the collaborator.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	Any valid string representing the collaborator's identity.

Example Usage:

name: ${{iamgroot}}

`description`¶

Description: A brief description of the collaborator's role.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	Example roles include owner, developer, consumer.

Example Usage:

description: ${{owner}}

`relatedDataProducts`¶

Description: Lists related Data Products connected or associated with the current Data Product.

Data Type	Requirement	Default Value	Possible Value
list of strings	optional	none	Each string references another Data Product by specifying the version and name.

Example Usage:

relatedDataProducts:
  - ${{data:v1beta:customer-360-demov3}}

`resource`¶

Description: Defines the resources utilized by the Data Product.

Data Type	Requirement	Default Value	Possible Value
object	mandatory	none	Contains fields like `description`, `purpose`, `refType`, and `ref`.

Example Usage:

resource:
  description: ${{'Ingest data'}}
  purpose: ${{'ingestion'}}
  refType: ${{dataos}}
  ref: ${{bundle:v1beta:sales-data-pipeline}}

Data Product card showing the Bundle Resource info on the Metis

`description (resource)`¶

Description: A brief description of the resource.

Data Type	Requirement	Default Value	Possible Value
string	optional	none	Any string describing the resource.

Example Usage:

description: ${{'Ingest data'}}

`purpose (resource)`¶

Description: The purpose of the resource.

Data Type	Requirement	Default Value	Possible Value
string	optional	none	Possible values include specific purposes like `ingestion`.

Example Usage:

purpose: ${{'ingestion'}}

`refType`¶

Description: The reference type of the resource.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	Example: `dataos`.

Example Usage:

refType: ${{dataos}}

`ref`¶

Description: The reference of the resource.

Data Type	Requirement	Default Value	Possible Value
string	mandatory	none	Example: `bundle:v1beta:sales-data-pipeline`.

Example Usage:

ref: ${{bundle:v1beta:sales-data-pipeline}}

`inputs`¶

Description: Lists the input data sources required by the Data Product.

Data Type	Requirement	Default Value	Possible Value
list of objects	mandatory	none	Each object describes an input with fields like `description`, `purpose`, `refType`, and `ref`.

Example Usage:

inputs: # mandatory
  - refType: dataos
    ref: dataset:icebase:customer_relationship_management:customer

  - refType: dataos
    ref: dataset:icebase:customer_relationship_management:purchase

  - refType: dataos
    ref: dataset:icebase:customer_relationship_management:product

Data Product card showing an input on the Data Product Hub

Data Product card showing the input datasets on the Metis

`outputs`¶

Description: Specifies the output data generated by the Data Product.

Data Type	Requirement	Default Value	Possible Value
list of objects	mandatory	none	Each object details an output with fields like `description`, `purpose`, `refType`, `ref`, and optional `checks`.

Example Usage:

outputs: # optional
  - refType: dataos
    ref: dataset:icebase:customer_relationship_management:product_affinity_matrix

  - refType: dataos
    ref: dataset:icebase:customer_relationship_management:cross_sell_recommendations

Data Product card showing an output on the Data Product Hub

Data Product card showing the output datasets on the Metis

`ports`¶

Description: Defines the ports for accessing the Data Product through various services.

Data Type	Requirement	Default Value	Possible Value
object	mandatory	none	Contains `lens`, `talos`, `rest`, and `postgres` sections for different types of ports.

Example Usage:

ports:
  lens:
    ref: ${{lens:v1alpha:sales360:public}}
    refType: ${{dataos}}

  talos:
    - ref: ${{service:v1:sales360-talos:public}}
      refType: ${{dataos}}

Data Product Configurations¶

Structure of Data Product manifest file¶

Product meta section¶

name¶

version¶

type¶

tags¶

description¶

Data Product-specific section¶

v1beta¶

data¶

meta¶

collaborators¶

name¶

description¶

relatedDataProducts¶

resource¶

description (resource)¶

purpose (resource)¶

refType¶

ref¶

inputs¶

outputs¶

ports¶

`name`¶

`version`¶

`type`¶

`tags`¶

`description`¶

`v1beta`¶

`data`¶

`meta`¶

`collaborators`¶

`name`¶

`description`¶

`relatedDataProducts`¶

`resource`¶

`description (resource)`¶

`purpose (resource)`¶

`refType`¶

`ref`¶

`inputs`¶

`outputs`¶

`ports`¶