Skip to content
All things DataOS
Pulsar
Initializing search
Home
Getting Started
Data Product
Glossary
Learn
All things DataOS
Home
Home
Philosophy
Architecture
Interfaces
Interfaces
Audiences
Audiences
Audiences UI
Getting Started
Bifrost
Bifrost
ABAC Implementation
Users
Use-cases
Grant
Role
Grant Requests
Heimdall Primitives
CLI
CLI
Installation
Initialization
Command Reference
Data Product Hub
Data Product Hub
Core Concepts
Data Product Discovery
Data Product Exploration
Data Product Activation
Data Product Activation
BI Sync
BI Sync
Tableau Cloud
Power BI
Power BI
Power BI Desktop
Power BI Service
Tableau Desktop
Microsoft Excel
Apache Superset
AI/ML
App Development
Data APIs
Best Practices
Examples
Metis
Metis
Metis UI
Metis UI
Pre-defined Roles and Policies
Key Features
Metadata of Products
Metadata of Products
Metadata of Data Products
Metadata of Data Assets
Metadata of Data Assets
Metadata of Tables
Metadata of Topics
Metadata of Dashboards
Metadata of Resources
Metadata of Resources
Bundles
Clusters
Computes
Databases
Depots
Instance Secrets
Lakehouses
Lenses
Monitors
Operators
Pagers
Policies
Secrets
Services
Stacks
Workers
Workflows
Notebook
Notebook
Operations
Operations
Workbench
Workbench
Resources
Resources
Types
Types
Bundle
Bundle
Configurations
Templates
Cluster
Cluster
Configurations
Configurations
Manifest Reference
Connectors
Integration
Integration
Power BI
Power BI
Using Connector
Using ODBC Driver
Tableau
IBM SPSS
CLI Client
Tuning
Tuning
Query Execution Optimization
Performance Tuning
Recommended Configuration
Examples
Examples
Restart and scale Cluster on a pre-defined schedule
Multiple Cluster Setup using single manifest
Core Concepts
Core Concepts
On-demand Computing
Compute
Compute
Configurations
Templates
Database
Database
First Steps
Configurations
Recipes
Recipes
Query Database using Workbench
Building Streamlit applications backed by Database
Depot
Depot
Supported Sources
Configurations
Supported Storage Architecture
Supported Storage Architecture
Icebase
Icebase
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Fastbase
Supported Connectors
Grant
Grant
Configuration
Instance Secret
Instance Secret
First Steps
Templates
Configurations
Recipes
Recipes
Referring Instance Secrets in other Resources
Lakehouse
Lakehouse
Configurations
Configurations
Manifest Reference
Command Reference
Iceberg Metadata Tables
Recipes
Recipes
Ensure high data quality using Write-Audit-Publish (WAP) Pattern
Use Iceberg metadata tables to extract insights
Create, Get, and Drop Dataset
Table Properties
Schema Evolution
Partitioning
Maintenance (Snapshots and Metadata listing)
Lens
Lens
Semantic Modeling
Semantic Modeling
Key concepts
Segments
Views
User Groups
Supported Data Sources
Supported Data Sources
Bigquery
Flash
Minerva
Postgres
Redshift
Snowflake
Themis
Exploration
Exploration
Using GraphQL
Using GraphQL
Overview
Examples
Using Python
Using SQL APIs
Using SQL APIs
Overview
Supported functions and operators
Using REST APIs
BI Integrations
BI Integrations
Power BI
Power BI
Power BI Desktop
Power BI Service
Superset
Tableau
References
References
Best Practices
Errors
Optimizing Semantic Model
Aggregation Strategies
Configurations
Cataloging
Governance
Observability
Monitor
Monitor
First Steps
Configurations
Recipes
Recipes
Add context columns to Monitors
Create a Report Monitor
Create a Stream Monitor
Create an Equation Monitor on Lens
Create an Equation Monitor on Postgres
Generate incidents for certificate expiration
Operator
Operator
Configurations
Recipes
Recipes
Orchestrate Hightouch pipeline using Hightouch Factory Operator
Pager
Pager
Configurations
Recipes
Recipes
Create a Custom Body Template for Pager
Policy
Policy
First Steps
Configurations
Recipes
Recipes
End-to-end use-case on how to implement access policy
Core Concepts
Secret
Secret
First Steps
Configurations
Recipes
Recipes
Refer secrets in other DataOS Resources
Setup secrets to pull images from a private container registry
Service
Service
Configurations
Stacks
Stacks
Beacon
Beacon
Bento
Bento
Setting Up Bento
Setting Up Bento
Bento on Local Host
Bento on DataOS
Bloblang
Bloblang
Overview
Core Features
Functions
Methods
Arithmetic
Advanced Bloblang
Configurations
Configurations
Cloud Credentials
Dynamic Inputs and Outputs
Error Handling
Fields Paths
Interpolation
Message Batching
Metadata
Monitoring
Performance Tuning
Processing Pipelines
Resources
Secrets
Streams Mode
Streams Mode
Streams API
Streams Via Config Files
Streams Via REST API
Synchronous Responses
Templating
Unit Testing
Using CUE
Window Processing
Components
Components
HTTP
Inputs
Logger
Metrics
Output
Processors
Rate Limit
Recipes
Recipes
Fetching Data from Instagram API
Twitter API Data Processing
Discord Bot
Stock Data API to Icebase
Performing Rate Limit
Performing Pagination
CLI Stack
CLI Stack
Container
Container
Data Toolbox
Data Toolbox
DBT
DBT
Flare
Flare
Quick Guide
Flash
Flash
Create a Flash Service
Configurations
Supported Data Sources
Monitoring Flash
Best Practices
Do's and Dont's
Errors and Issues
Errors and Issues
Handling Empty Tables
Recipes
Recipes
Use cached datasets in Lens models
Use cached datasets in Talos APIs
Lakesearch
Lakesearch
Key concepts
Set up
Set up
Lakesearch Service
Query Rewriter
Configurations
Governance
Index Searching
Observability
Troubleshooting
Scanner
Scanner
Soda
Soda
Steampipe
Steampipe
Talos
Talos
Setting Up Talos
Supported Sources
Supported Sources
Bigquery
Flash
Lens
MySQL
Postgres
Redshift
Snowflake
Configurations
Configurations
config.yaml
apis
service.yaml
Governance
Recipes
Recipes
Apply data masking
Fetch data from third party tools
Caching datasets
Generate API documentation
External API as data source
Validating Parameters
Error Handling
Example
Best Practices
Best Practices
API
SQL
Custom Stack
Custom Stack
How to create your own Stack?
Configurations
Volume
Volume
Getting Started
Configurations
Worker
Worker
First Steps
Configuration
Core Concepts
Workflow
Workflow
First Steps
Configurations
Recipes
Recipes
Implement Single-run Workflow
Run Cron/Scheduled Workflow
Orchestrate multiple Workflows from a single Workflow
Retry a Job in Workflow
Characteristics
Configuration
Best practices for YAML
SDK
SDK
DataOS Python SDK
DataOS PyFlare SDK
DataOS Flare SDK
DataOS Kotlin SDK
Getting Started
Getting Started
Data Product Consumer
Data Product Developer
Operator
Data Product
Data Product
Core Concepts
Data Product Lifecycle
Data Product Lifecycle
Design Phase
Build Phase
Deploy Phase
Iteration & Optimization
Data Product Examples
Data Product Examples
Building from Scratch with Snowflake
Using DataOS Lakehouse as a Source
Configurations
Recipes
Recipes
Deploy Data Product using CI/CD pipeline
Glossary
Learn
Learn
Learning tracks
Learning tracks
Data Product Consumer
Data Product Consumer
Introduction to Data Products
Introduction to Data Products
Features and importance of Data Products
Features and importance of Data Products
Discovering Data Products
Discovering Data Products
Introduction to Data Product Hub
Introduction to Data Product Hub
Discovering Data Products
Discovering Data Products
Viewing Data Product info
Viewing Data Product info
Exploring Input and Output DataOS
Exploring Input and Output DataOS
Navigating semantic models
Navigating semantic models
Checking data quality
Checking data quality
Integrating Data Products with BI tools and Applications
Integrating Data Products with BI tools and Applications
Integration with BI tools
Integration with BI tools
Integration with AI/ML
Integration with AI/ML
Integration with Postgres
Integration with Postgres
Integration with GraphQL
Integration with GraphQL
Integration with data APIs
Integration with data APIs
Custom apps to consume Data Products
Custom apps to consume Data Products
Building an App with Appsmith
Building an App with Appsmith
Data Product Developer
Data Product Developer
Understanding business goals
Understanding business goals
Designing Data Products
Designing Data Products
Building Data Products
Building Data Products
Source data connectivity
Source data connectivity
Connectivity with Postgres
Connectivity with Postgres
Building and maintaining data pipelines
Building and maintaining data pipelines
Creating your first data pipelines
Creating your first data pipelines
Scheduling Workflows
Scheduling Workflows
Data quality checks
Data quality checks
Pipeline observability
Pipeline observability
Implementing quality checks
Implementing quality checks
Build a semantic model
Build a semantic model
Defining business objectives and KPIs
Defining business objectives and KPIs
Designing the conceptual Model
Designing the conceptual Model
Key concepts of Lens
Key concepts of Lens
Creating semantic model
Creating semantic model
Test Lens Locally
Test Lens Locally
Deploy Lens in DataOS
Deploy Lens in DataOS
Defining data policies for semantic model
Defining data policies for semantic model
Creating data APIs
Creating data APIs
Deploying Data Products
Deploying Data Products
Creating a deployable Bundles
Creating a deployable Bundles
Creating a Data Product Spec Fields
Creating a Data Product Spec Fields
Deploying Data Products
Deploying Data Products
Data Product CI/CD pipeline
Data Product CI/CD pipeline
DataOS Operator
DataOS Operator
Credential security
Credential security
Data source connectivity
Data source connectivity
Connectivity with Postgres
Connectivity with Postgres
Routine checks
Routine checks
DataOS upgrade and rollback strategies
DataOS upgrade and rollback strategies
System monitoring
System monitoring
Configuring alerts and dashboards using Grafana
Configuring alerts and dashboards using Grafana
Optimizing system level events
Optimizing system level events
Query Cluster management
Query Cluster management
Cluster management
Cluster management
Cluster tuning
Cluster tuning
Quick Start Guides
Quick Start Guides
Connecting with data sources
Connecting with data sources
Performing exploratory data analysis
Performing exploratory data analysis
Running your first data query
Running your first data query
Arranging and transforming data with Pivot
Arranging and transforming data with Pivot
Writing complex queries
Writing complex queries
Querying diverse data sources
Querying diverse data sources
Using data modeling layer
Using data modeling layer
Defining a Data Model
Defining a Data Model
Creating a data model
Creating a data model
Testing a data model
Testing a data model
Deploying a data model
Deploying a data model
Exploring Lens on Catalog
Exploring Lens on Catalog
Working on Lens Studio
Working on Lens Studio
Working with Payloads
Working with Payloads
Example scenarios
Pre-defined filtering using Segments
Pre-defined filtering using Segments
Defining Data Policies
Defining Data Policies
Building Data Products
Scanning metadata
Creating ETL pipelines
Creating ETL pipelines
Ingest external files into DataOS
Ingest external files into DataOS
Batch data processing using Flare
Batch data processing using Flare
Storing query results as dataset with Flare
Storing query results as dataset with Flare
Deploying and securing data applications
Deploying and securing data applications
Deploying data application using Container Stack
Deploying data application using Container Stack
Listing Your App on DataOS Home
Listing Your App on DataOS Home
Securing deployed data applications
Securing deployed data applications
Videos
Pulsar
¶
Was this page helpful?
Thanks for your feedback!
Thanks for your feedback!
Back to top