Skip to content
All things DataOS
Pulsar
Initializing search
Home
Getting Started
Data Product
Glossary
Learn
All things DataOS
Home
Home
Philosophy
Architecture
Interfaces
Interfaces
Audiences
Audiences
Audiences UI
Getting Started
Bifrost
Bifrost
ABAC Implementation
Users
Use-cases
Grant
Role
Grant Requests
Heimdall Primitives
CLI
CLI
Installation
Initialization
Command Reference
Data Product Hub
Data Product Hub
Core Concepts
Data Product Discovery
Data Product Exploration
Data Product Activation
Data Product Activation
BI Sync
BI Sync
Tableau Cloud
Power BI
Tableau Desktop
Microsoft Excel
Apache Superset
AI/ML
App Development
Data APIs
Best Practices
Examples
Metis
Metis
Metis UI
Metis UI
Pre-defined Roles and Policies
Key Features
Metadata of Products
Metadata of Products
Metadata of Data Products
Metadata of Data Assets
Metadata of Data Assets
Metadata of Tables
Metadata of Topics
Metadata of Dashboards
Metadata of Resources
Metadata of Resources
Bundles
Clusters
Computes
Databases
Depots
Instance Secrets
Lakehouses
Lenses
Monitors
Operators
Pagers
Policies
Secrets
Services
Stacks
Workers
Workflows
Notebook
Notebook
Operations
Operations
Workbench
Workbench
Resources
Resources
Types
Types
Bundle
Bundle
Configurations
Templates
Cluster
Cluster
Configurations
Configurations
Manifest Reference
Connectors
Integration
Integration
Power BI
Power BI
Using Connector
Using ODBC Driver
Tableau
IBM SPSS
CLI Client
Tuning
Tuning
Query Execution Optimization
Performance Tuning
Recommended Configuration
Examples
Examples
Restart and scale Cluster on a pre-defined schedule
Multiple Cluster Setup using single manifest
Core Concepts
Core Concepts
On-demand Computing
Compute
Compute
Configurations
Templates
Database
Database
First Steps
Configurations
Recipes
Recipes
Query Database using Workbench
Building Streamlit applications backed by Database
Depot
Depot
Supported Sources
Configurations
Supported Storage Architecture
Supported Storage Architecture
Icebase
Icebase
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Fastbase
Supported Connectors
Grant
Grant
Configuration
Instance Secret
Instance Secret
First Steps
Templates
Configurations
Recipes
Recipes
Referring Instance Secrets in other Resources
Lakehouse
Lakehouse
Configurations
Configurations
Manifest Reference
Command Reference
Iceberg Metadata Tables
Recipes
Recipes
Ensure high data quality using Write-Audit-Publish (WAP) Pattern
Use Iceberg metadata tables to extract insights
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Lens
Lens
Lens Setup
Lens Setup
Lens Model Folder Setup
Develop Lens Locally
Develop Lens in DataOS
Data Modelling
Data Modelling
Overview
Concepts
Best Practices
Do's and Dont's
Errors
Configurations
Configurations
Lens Manifest
Docker Compose Manifest
Supported Data Sources
Supported Data Sources
Bigquery
Flash
Minerva
Postgres
Redshift
Themis
APIs & Integrations
APIs & Integrations
API Endpoints and Scopes
BI Integration
Reference
Reference
Working with Payload
Working with Segments
Working with Views
Working with User Groups
Exploration
Exploration
Using GraphQL
Using SQL APIs
Using Python
Using REST APIs
Using Iris dashboard
Optimizing Lens Model
Monitor
Monitor
First Steps
Configurations
Recipes
Recipes
Add context columns to Monitors
Create a Report Monitor
Create a Stream Monitor
Create an Equation Monitor on Lens
Create an Equation Monitor on Postgres
Generate incidents for certificate expiration
Operator
Operator
Configurations
Recipes
Recipes
Orchestrate Hightouch pipeline using Hightouch Factory Operator
Pager
Pager
Configurations
Recipes
Recipes
Create a Custom Body Template for Pager
Policy
Policy
First Steps
Configurations
Recipes
Recipes
End-to-end use-case on how to implement access policy
Core Concepts
Secret
Secret
First Steps
Configurations
Recipes
Recipes
Refer secrets in other DataOS Resources
Setup secrets to pull images from a private container registry
Service
Service
Configurations
Stacks
Stacks
Beacon
Beacon
Benthos
Benthos
CLI Stack
CLI Stack
Container
Container
Data Toolbox
Data Toolbox
DBT
DBT
Flare
Flare
Flash
Flash
Create a Flash Service
Configurations
Supported Data Sources
Monitoring Flash
Best Practices
Do's and Dont's
Errors and Issues
Errors and Issues
Handling Empty Tables
Recipes
Recipes
Use cached datasets in Lens models
Use cached datsets in Talos APIs
Scanner
Scanner
Soda
Soda
Steampipe
Steampipe
Talos
Talos
Setting Up Talos
Setting Up Talos
On Local
On DataOS
Configurations
Configurations
config.yaml
docker-compose.yaml
apis
service.yaml
Recipes
Recipes
Apply data masking
Fetch data from third party tools
Caching datasets
Generate API documentation
External API as data source
Example
Best Practices
Best Practices
API
SQL
Custom Stack
Custom Stack
How to create your own Stack?
Configurations
Volume
Volume
Getting Started
Configurations
Worker
Worker
First Steps
Configuration
Core Concepts
Workflow
Workflow
First Steps
Configurations
Recipes
Recipes
Implement Single-run Workflow
Run Cron/Scheduled Workflow
Orchestrate multiple Workflows from a single Workflow
Retry a Job in Workflow
Characteristics
Configuration
Best practices for YAML
SDK
SDK
DataOS Python SDK
DataOS PyFlare SDK
DataOS Flare SDK
DataOS Kotlin SDK
Getting Started
Getting Started
Data Product Consumer
Data Product Developer
Operator
Data Product
Data Product
Data Product Lifecycle
Data Product Lifecycle
Design
Build
Deploy
Iterate
Recipes
Recipes
Deploy Data Product using CI/CD pipeline
Examples
Examples
Product affinity Data Product
Configurations
Core Concepts
Glossary
Learn
Learn
Learning tracks
Learning tracks
Data Product Consumer
Data Product Consumer
Discovering Data Products
Discovering Data Products
Introduction to Data Product Hub
Introduction to Data Product Hub
Discovering Data Products
Discovering Data Products
Viewing Data Product info
Viewing Data Product info
Exploring Input and Output DataOS
Exploring Input and Output DataOS
Navigating semantic models
Navigating semantic models
Checking data quality
Checking data quality
Integrating Data Products with BI tools and Applications
Integrating Data Products with BI tools and Applications
Integration with BI tools
Integration with BI tools
Integration with AI/ML
Integration with AI/ML
Integration with Postgres
Integration with Postgres
Integration with GraphQL
Integration with GraphQL
Integration with data APIs
Integration with data APIs
Data Product Developer
Data Product Developer
Disigning Data Products
Disigning Data Products
Building Data Products
Building Data Products
Source data connectivity
Source data connectivity
Connectivity with Postgres
Connectivity with Postgres
Building and maintaining data pipelines
Building and maintaining data pipelines
Creating your first data pipelines
Creating your first data pipelines
Sheduling Workflows
Sheduling Workflows
Data quality Checks
Data quality Checks
Pipeline observability
Pipeline observability
Build a semantic model
Build a semantic model
Defining business objectives and KPIs
Defining business objectives and KPIs
Designing the conceptual Model
Designing the conceptual Model
Key concepts of Lens
Key concepts of Lens
Creating semantic model
Creating semantic model
Test Lens Locally
Test Lens Locally
None
Deploy Lens in DataOS
Deploy Lens in DataOS
Implementing quality Checks
Implementing quality Checks
Creating data APIs
Creating data APIs
Defining data policies for Lens
Defining data policies for Lens
Deploying Data Products
Deploying Data Products
Creating a deployable Bundles
Creating a deployable Bundles
Creating a Data Product Spec Fields
Creating a Data Product Spec Fields
Deploying Data Products
Deploying Data Products
Data Product CI/CD pipeline
Data Product CI/CD pipeline
DataOS Operator
DataOS Operator
Credential security
Credential security
Data source connectivity
Data source connectivity
Connectivity with Postgres
Connectivity with Postgres
Routine checks
Routine checks
DataOS upgrade and rollback strategies
DataOS upgrade and rollback strategies
System monitoring
System monitoring
Configuring alerts and dashboards using Grafana
Configuring alerts and dashboards using Grafana
Optimizing system level events
Optimizing system level events
Query Cluster management
Query Cluster management
Cluster management
Cluster management
Cluster tuning
Cluster tuning
Quick Start Guides
Quick Start Guides
Connecting with data sources
Connecting with data sources
Using GUI
Using GUI
Using CLI
Using CLI
Performing exloratory data analysis
Performing exloratory data analysis
Running your first data query
Running your first data query
Arranging and transforming data with Pivot
Arranging and transforming data with Pivot
Writing complex queries
Writing complex queries
Querying diverse data sources
Querying diverse data sources
Using data modeling layer
Using data modeling layer
Defining a Data Model
Defining a Data Model
Creating a data model
Creating a data model
Testing a data model
Testing a data model
Deploying a data model
Deploying a data model
Exploring Lens on Catalog
Exploring Lens on Catalog
Working on Lens Studio
Working on Lens Studio
Working with Payloads
Working with Payloads
Example scenarios
Pre-defined filtering using Segemnts
Pre-defined filtering using Segemnts
Defining Data Policies
Defining Data Policies
Building Data Products
Scanning metadata
Creating ETL pipelines
Creating ETL pipelines
Ingest external files into DataOS
Ingest external files into DataOS
Batch data processing using Flare
Batch data processing using Flare
Storing query results as dataset with Flare
Storing query results as dataset with Flare
Deploying and securing data applications
Deploying and securing data applications
Deploying data application using Container Stack
Deploying data application using Container Stack
Securing deployed data applications
Securing deployed data applications
Videos
Pulsar
¶
Was this page helpful?
Thanks for your feedback!
Thanks for your feedback!
Back to top