Setting up Talos for Flash¶

Pre-requisites¶

Flash set up
Docker initialization

Steps¶

Create a repository, open the repository with a code editor (VS Code), and create a config.yaml manifest file and copy the below code. Update the name, description, version, dataos context, Flash as type, and Flash name.

name: flash
description: A talos-flash app
version: 0.1.6
auth:
  heimdallUrl: https://liberal-donkey.dataos.app/heimdall
logLevel: 'DEBUG'
sources:
  - name: flash # profile name
    type: flash
    flashName: 'public:flash-service'

In the same repository, create docker-compose.YAML manifest file, copy the below-provided code, and update the volumes path /home/iamgroot/Desktop/talos-examples/lens with the actual path of your repository, add your dataos username and dataos API key in DATAOS_RUN_AS_USER and DATAOS_RUN_AS_APIKEY respectively.

version: "2.2"
services:
  talos:
    image: rubiklabs/talos:0.1.6
    ports:
      - "3000:3000"
    volumes:
      - /home/iamgroot/Desktop/talos-examples/flash:/etc/dataos/work
    environment:
      DATAOS_RUN_AS_USER: 
      DATAOS_RUN_AS_APIKEY: 
      DATAOS_FQDN: liberal-donkey.dataos.app
    tty: true

Create a new file Makefile in the same repository, and copy the below code.

start:
    docker-compose -f docker-compose.yaml up -d

stop:
    docker-compose -f docker-compose.yaml down -v

Create a folder named apis inside the same repository, inside apis create the files customer.sql which will contain the SQL query, and customer.yaml to define the path to access sales data in your API as shown below.
```
SELECT * FROM mycustomer LIMIT 20
```
```
urlPath: /flash/customers
description: list customers from customer table
source: flash
```

Run docker-compose up on the terminal. The output should look like the following:

output

docker-compose up
[+] Running 1/0
 ✔ Container depot-postgres-talos-1  Created                                                                                                                                                                  0.0s 
Attaching to depot-postgres-talos-1
depot-postgres-talos-1  | 👉 /etc/dataos/work/config.yaml => {
depot-postgres-talos-1  |   "name": "postgres_domain",
depot-postgres-talos-1  |   "description": "A talos-depot-postgres app",
depot-postgres-talos-1  |   "version": "0.1.6",
depot-postgres-talos-1  |   "auth": {
depot-postgres-talos-1  |     "heimdallUrl": "https://liberal-donkey.dataos.app/heimdall"
depot-postgres-talos-1  |   },
depot-postgres-talos-1  |   "logLevel": "DEBUG",
depot-postgres-talos-1  |   "sources": [
depot-postgres-talos-1  |     {
depot-postgres-talos-1  |       "name": "postgre01",
depot-postgres-talos-1  |       "type": "depot"
depot-postgres-talos-1  |     }
depot-postgres-talos-1  |   ],
depot-postgres-talos-1  |   "schemaPath": "",
depot-postgres-talos-1  |   "cachePath": "tmp"
depot-postgres-talos-1  | }
depot-postgres-talos-1  | Get Depot Service Depot Fetch URL:  https://liberal-donkey.dataos.app/ds/api/v2/depots/postgre01
depot-postgres-talos-1  | Get Depot Service Secrets Fetch URL:  https://liberal-donkey.dataos.app/ds/api/v2/secrets/postgre01_r
depot-postgres-talos-1  | 🧑‍🤝‍🧑 sources => [
depot-postgres-talos-1  |   {
depot-postgres-talos-1  |     "name": "postgre01",
depot-postgres-talos-1  |     "type": "pg",
depot-postgres-talos-1  |     "connection": {
depot-postgres-talos-1  |       "host": "usr-db-dataos-ck-vgji-liberaldo-dev.postgres.database.azure.com",
depot-postgres-talos-1  |       "port": 5432,
depot-postgres-talos-1  |       "database": "postgres",
depot-postgres-talos-1  |       "user": "--REDACTED--",
depot-postgres-talos-1  |       "password": "--REDACTED--"
depot-postgres-talos-1  |     }
depot-postgres-talos-1  |   }
depot-postgres-talos-1  | ]
depot-postgres-talos-1  | - Building project...
depot-postgres-talos-1  | 2024-07-24 10:56:22.626  
depot-postgres-talos-1  | DEBUG [BUILD]
depot-postgres-talos-1  | Initializing data source: mock
depot-postgres-talos-1  | 2024-07-24 10:56:22.627  
depot-postgres-talos-1  | DEBUG [BUILD] Data source mock initialized
depot-postgres-talos-1  | 2024-07-24 10:56:22.628  
depot-postgres-talos-1  | DEBUG [BUILD] Initializing data source: bq
depot-postgres-talos-1  | 2024-07-24 10:56:22.628  
depot-postgres-talos-1  | DEBUG [BUILD] Data source bq initialized
depot-postgres-talos-1  | 
depot-postgres-talos-1  | 2024-07-24 10:56:22.629  DEBUG
depot-postgres-talos-1  | [BUILD] Initializing data source: clickhouse
depot-postgres-talos-1  | 2024-07-24 10:56:22.629  
depot-postgres-talos-1  | DEBUG [BUILD] Data source clickhouse initialized
depot-postgres-talos-1  | 
depot-postgres-talos-1  | 2024-07-24 10:56:22.630  DEBUG [BUILD] Initializing data source: duckdb
depot-postgres-talos-1  | 2024-07-24 10:56:22.636  DEBUG
depot-postgres-talos-1  | [CORE] Create connection for talos.cache
depot-postgres-talos-1  | 2024-07-24 10:56:22.637  
depot-postgres-talos-1  | DEBUG [CORE] Open database in automatic mode
depot-postgres-talos-1  | 2024-07-24 10:56:22.650  
depot-postgres-talos-1  | DEBUG
depot-postgres-talos-1  | [CORE] Installed httpfs extension
depot-postgres-talos-1  | 2024-07-24 10:56:22.653  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: access_mode = automatic
depot-postgres-talos-1  | 2024-07-24 10:56:22.653  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: allow_persistent_secrets = true
depot-postgres-talos-1  | 2024-07-24 10:56:22.654  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: checkpoint_threshold = 16.0 MiB
depot-postgres-talos-1  | 2024-07-24 10:56:22.654  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: debug_checkpoint_abort = none
depot-postgres-talos-1  | 2024-07-24 10:56:22.654  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: storage_compatibility_version = v0.10.2
depot-postgres-talos-1  | 2024-07-24 10:56:22.654  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: debug_force_external = false
depot-postgres-talos-1  | 2024-07-24 10:56:22.655  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: debug_force_no_cross_product = false
depot-postgres-talos-1  | 2024-07-24 10:56:22.655  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: debug_asof_iejoin = false
depot-postgres-talos-1  | 2024-07-24 10:56:22.655  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: prefer_range_joins = false
depot-postgres-talos-1  | 2024-07-24 10:56:22.655  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: debug_window_mode = NULL
depot-postgres-talos-1  | 2024-07-24 10:56:22.656  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: default_collation =
depot-postgres-talos-1  | 2024-07-24 10:56:22.656  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: default_order = asc
depot-postgres-talos-1  | 2024-07-24 10:56:22.656  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: default_null_order = nulls_last
depot-postgres-talos-1  | 2024-07-24 10:56:22.656  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: disabled_filesystems =
depot-postgres-talos-1  | 2024-07-24 10:56:22.656  
depot-postgres-talos-1  | DEBUG [CORE] Duckdb config: disabled_optimizers =  
depot-postgres-talos-1  | 2024-07-24 10:56:22.657  DEBUG [CORE] Duckdb config: enable_external_access = true
depot-postgres-talos-1  |