API Endpoints and Scopes¶
Each REST API endpoint belongs to an API scope, e.g., the /v2/load endpoint belongs to the data scope. API scopes provide a way to secure access to API endpoints, allowing you to limit accessibility to specific users or roles, or disallow access entirely. By default, API endpoints in all scopes (except for jobs) are accessible to everyone, allowing broad access unless specifically restricted.
You can manage API access using the user_groups. The default user group ensures that API endpoints in all scopes are accessible to everyone. You can create custom user groups by defining roles and associating specific users with these roles in the user_group.yml file. To know more about user groups click here.
The following api_scopes are currently supported:
| API scope | REST API endpoints |
|---|---|
| meta | /v2/meta |
| data | - /v2/load - /v2/sql |
| graphql | /v2/graphql |
| source | - /v2/default/schemas/ - /v2/default/schemas/sandbox/tables - /v2/default/schemas/retail/tables/city |
Prerequisites¶
Authentication¶
Lens uses API tokens to authorize requests and also for passing additional security context.
The API Token is passed via the Authorization Header. The token itself is a dataos-user-apikey.
meta scope¶
Provides access to metadata-related endpoints. This scope allows users to view metadata, which typically includes information about sources, authors, timezones, security context, user groups, etc.
meta endpoint¶
Get meta-information for lens and views defined in the data model. Information about lens and lens with "public: false" will not be returned.
Example response:
{
"name":
"description":
],
"authors": [
],
"devMode": true,
"source": {
"type": "minerva",
}
},
"timeZones": [
"UTC"
],
"tables": [
{
"name": "product_analysis",
"type": "view",
"title": "Product Analysis",
]
}
data scope¶
load endpoint¶
Executes queries to retrieve data based on the specified dimensions, measures, and filters. When you need to perform data analysis or retrieve specific data from the data model.
Use POST request method along with /load endpoint to add or update the data.
You can use either of the following methods:
In the POST request body, include the query parameters in the JSON Query Format:
Example Query
{
"query": {
"dimensions": ["customer.customer_id", "customer.annual_income"],
"measures": ["customer.total_customers", "customer.average_age"],
"limit": 50
}
}
Example response:
{
"query": [
{
"name": "Customers",
"title": "Customers",
"meta": {
"someKey": "someValue",
"nested": {
"someKey": "someValue"
}
},
"connectedComponent": 1,
"measures": [
{
"name": "customers.count",
"title": "Customers Count",
"shortTitle": "Count",
"aliasName": "customers.count",
"type": "number",
"aggType": "count",
"drillMembers": ["customers.id", "customers.city", "customers.createdAt"]
}
],
"dimensions": [
{
"name": "customers.city",
"title": "Customers City",
"type": "string",
"aliasName": "customers.city",
"shortTitle": "City",
"suggestFilterValues": true
}
],
"segments": []
}
]
}
sql endpoint¶
Alternatively, you can use /sql endpoint.
configure the body with the JSON Query Format similar to /load.
graphql¶
Grants access to GraphQL endpoints. GraphQL is a query language for APIs that allows clients to request only the data they need. This scope enables users to perform GraphQL queries and mutations. To know more about How to use GraphQL click here
source¶
Grants access to source-related endpoints.
| `Parameter` | Description | Action |
|---|---|---|
| `schemas` | An array of schema names in the data source. | GET |
| `tables` | An array of schemas and table names in the data source. | GET |
| `describe-table` | Describes the given source table. | GET |
| `load-source` | Loads all datasets of the source. | POST |
Default Schema Collection Tables Endpoint¶
This endpoint fetches all tables within a specified schema (collection).
Example Requests
For instance, if you want to fetch all tables in sandbox schema
Example Response
{
"data": {
"members": [
"schema_name",
"table_name"
],
"dataset": [
[
"sandbox",
"customer"
],
[
"sandbox",
"locations"
],
[
"sandbox",
"products"
],
[
"sandbox",
"items"
],
[
"sandbox",
"transactions"
],
[
"sandbox",
"company_data"
],
[
"sandbox",
"sales"
],
[
"sandbox",
"ratings"
],
]
}
}
Specific Table in Schema Collection Endpoint:¶
This endpoint retrieves detailed information about a specific table within a schema.
Example Requests
For example, to retrieve the details of the city table within the sales360 schema, you can use the following query.
Example Response
{
"data": {
"members": [
"column_name",
"table_name",
"schema_name",
"data_type",
"foreign_keys"
],
"dataset": [
[
"__metadata",
"city",
"retail",
"map(varchar, varchar)",
[]
],
[
"city_id",
"city",
"retail",
"varchar",
[]
],
[
"zip_code",
"city",
"retail",
"integer",
[]
],
[
"city_name",
"city",
"retail",
"varchar",
[]
],
[
"county_name",
"city",
"retail",
"varchar",
[]
],
[
"state_code",
"city",
"retail",
"varchar",
[]
],
[
"state_name",
"city",
"retail",
"varchar",
[]
],
[
"version",
"city",
"retail",
"varchar",
[]
],
[
"ts_city",
"city",
"retail",
"timestamp(6) with time zone",
[]
],
[
"decimal_column",
"city",
"retail",
"decimal(7,3)",
[]
],
[
"string",
"city",
"retail",
"varchar",
[]
]
]
}
}
Possible Responses¶
Continue wait¶
If the request takes too long to be processed, Lens Backend responds with { "error": "Continue wait" } and 200 status code. This is how the long polling mechanism in Lens is implemented. Clients should continuously retry the same query in a loop until they get a successful result. Subsequent calls to the Lens endpoints are idempotent and don't lead to scheduling new database queries if not required by the refresh_key. Also, receiving Continue wait doesn't mean the database query has been canceled, and it's actually still being processed by the Lens. Database queries that weren't started and are no longer waited by the client's long polling loop will be marked as orphaned and removed from the querying queue.
Possible reasons of Continue wait:
-
The query requested is heavy, and it takes some time for the database to process it. Clients should wait for its completion, continuously sending the same REST API request. continueWaitTimeout can be adjusted in order to change the time Lens waits before returning Continue wait message.
-
There are many queries requested and Lens backend queues them to save database from overloading.
Error Handling¶
Lens REST API has basic errors and HTTP Error codes for all requests.
| Status | Error response | Description |
|---|---|---|
| 400 | Error message | General error. It may be a database error, timeout, or other issue. Check error message for details. |
| 403 | Authorization header isn't set | You didn't provide an auth token. Provide a valid API Token or disable authorization. |
| 403 | Invalid token | The auth token provided is not valid. It may be expired or have an invalid signature. |
| 500 | Error message | Lens internal server error. Check error message for details. |