Skip to content
All things DataOS
Pulsar
Initializing search
Home
Getting Started
Data Product
Glossary
Learn
All things DataOS
Home
Home
Philosophy
Architecture
Interfaces
Interfaces
Atlas
Atlas
Dashboards
Dashboards
Adding Visualizations to Dashboard
Queries
Snippet
Alerts
Destinations
Integration
Integration
Power BI
Power BI
Using Connector
Using ODBC Driver
Tableau
IBM SPSS
Audiences
Audiences
Audiences UI
Getting Started
Bifrost
Bifrost
ABAC Implementation
User
Use-cases
Grant
Role
Grant Requests
Heimdall Primitives
CLI
CLI
Installation
Initialization
Command Reference
Data Product Hub
Data Product Hub
Data Products
Recent Updates
About
Lens
Lens
Building Lens
Building Lens
Tutorial
Data Modelling Concepts
Lens Attributes
Dynamic Parameters
Lens Explorer
Lens Ecosystem
Lens Ecosystem
Lens Query
Elements of Lens
Elements of Lens
Entities
Fields
Dimensions
Measures
Relationships
Metis
Metis
Metis UI
Metis UI
Pre-defined Roles and Policies
Key Features
Metadata of Products
Metadata of Products
Metadata of Data Products
Metadata of Data Assets
Metadata of Data Assets
Metadata of Tables
Metadata of Topics
Metadata of Dashboards
Metadata of Resources
Metadata of Resources
Bundles
Clusters
Computes
Databases
Depots
Instance Secrets
Lakehouses
Lenses
Monitors
Operators
Pagers
Policies
Secrets
Services
Stacks
Workers
Workflows
Notebook
Notebook
Operations
Operations
Workbench
Workbench
Resources
Resources
Types
Types
Bundle
Bundle
Configurations
Templates
Cluster
Cluster
Configurations
Configurations
Manifest Reference
Connectors
Setting Up CLI Client
Tuning
Tuning
Query Execution Optimization
Performance Tuning
Recommended Configuration
Examples
Examples
Restart and scale Cluster on a pre-defined schedule
Multiple Cluster Setup using single manifest
Core Concepts
Core Concepts
On-demand Computing
Compute
Compute
Configurations
Templates
Database
Database
First Steps
Configurations
Recipes
Recipes
Query Database using Workbench
Building Streamlit applications backed by Database
Depot
Depot
Supported Sources
Configurations
Supported Storage Architecture
Supported Storage Architecture
Icebase
Icebase
Icebase
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Fastbase
Supported Connectors
Grant
Grant
Configuration
Instance Secret
Instance Secret
First Steps
Templates
Configurations
Recipes
Recipes
Referring Instance Secrets in other Resources
Lakehouse
Lakehouse
Configurations
Configurations
Manifest Reference
Command Reference
Iceberg Metadata Tables
Recipes
Recipes
Ensure high data quality using Write-Audit-Publish (WAP) Pattern
Use Iceberg metadata tables to extract insights
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Lens
Lens
Setup Lens
Core Concepts
Configurations
Configurations
Supported Data Sources
Supported Data Sources
Bigquery
Minerva
Postgres
Redshift
Themis
BI Integration
Working with Payload
Reference
Reference
Resource Manifest
Docker Compose Manifest
API Endpoints and Scopes
Exploration
Exploration
Using GraphQL
Using SQL APIs
Using Python
Using REST APIs
Data Modelling
Data Modelling
Best Practices
Do's and Dont's
Errors
Optimizing Lens Model
Monitor
Monitor
First Steps
Configurations
Recipes
Recipes
Add context columns to Monitors
Create a Report Monitor
Create a Stream Monitor
Create an Equation Monitor on Lens
Create an Equation Monitor on Postgres
Generate incidents for certificate expiration
Operator
Operator
Configurations
Recipes
Recipes
Orchestrate Hightouch pipeline using Hightouch Factory Operator
Pager
Pager
Configurations
Recipes
Recipes
Create a Custom Body Template for Pager
Policy
Policy
First Steps
Configurations
Recipes
Recipes
End-to-end use-case on how to implement access policy
Core Concepts
Secret
Secret
First Steps
Configurations
Recipes
Recipes
Refer secrets in other DataOS Resources
Setup secrets to pull images from a private container registry
Service
Service
Configurations
Stacks
Stacks
Beacon
Beacon
Benthos
Benthos
CLI Stack
CLI Stack
Container
Container
Data Toolbox
Data Toolbox
DBT
DBT
Flare
Flare
Scanner
Scanner
Soda
Soda
Steampipe
Steampipe
Talos
Talos
Setting Up Talos
Setting Up Talos
On Local
On DataOS
Configurations
Configurations
config.yaml
docker-compose.yaml
apis
service.yaml
Supported Sources
Supported Sources
Lens
Flash
Redshift
Snowflake
Postgres
Recipes
Recipes
Apply data masking
Fetch data from third party tools
Caching datasets
Generate API documentation
External API as data source
Example
Best Practices
Best Practices
API
SQL
Custom Stack
Custom Stack
How to create your own Stack?
Configurations
Volume
Volume
Getting Started
Configurations
Worker
Worker
First Steps
Configuration
Workflow
Workflow
First Steps
Configurations
Recipes
Recipes
Implement Single-run Workflow
Run Cron/Scheduled Workflow
Orchestrate multiple Workflows from a single Workflow
Retry a Job in Workflow
Characteristics
Configuration
YAML files
SDK
SDK
DataOS Python SDK
DataOS PyFlare SDK
DataOS Flare SDK
DataOS Kotlin SDK
Getting Started
Getting Started
Data Product Consumer
Data Product Developer
Operator
Data Product
Data Product
Data Product Lifecycle
Data Product Lifecycle
Design
Build
Deploy
Iterate
Recipes
Recipes
Create Data Product template using Cookiecutter
Deploy Data Product using CI/CD pipeline
Examples
Examples
Financial Services Accelerator Data Product
Sales 360 Data Product
Configurations
Core Concepts
Glossary
Learn
Learn
Quick Start Guides
Quick Start Guides
Connecting with Data Sources
Connecting with Data Sources
Using GUI
Using GUI
Using CLI
Using CLI
Performing Exloratory Data Analysis
Performing Exloratory Data Analysis
Running your first data Query
Running your first data Query
Arranging and Transforming data with Pivot
Arranging and Transforming data with Pivot
Writing Complex Queries
Writing Complex Queries
Querying Diverse Data sources
Querying Diverse Data sources
Using Data Modeling Layer
Using Data Modeling Layer
Defining a Data Model
Defining a Data Model
Creating a Data Model
Creating a Data Model
Testing a Data Model
Testing a Data Model
Deploying a Data Model
Deploying a Data Model
Exploring Lens on Catalog
Exploring Lens on Catalog
Working on Lens Studio
Working on Lens Studio
Working with Payloads
Working with Payloads
Example Scenarios
Pre-defined Filtering using Segemnts
Pre-defined Filtering using Segemnts
Defining Data Policies
Defining Data Policies
Scanning Metadata
Creating ETL Pipelines
Creating ETL Pipelines
Batch Data Processing using Flare
Batch Data Processing using Flare
Storing Query Results as Dataset with Flare
Storing Query Results as Dataset with Flare
Videos
Pulsar
¶
Was this page helpful?
Thanks for your feedback!
Thanks for your feedback!
Back to top