Skip to content

Scanner for MySQL

You can scan metadata details, such as schemas, tables, view details, etc., from MSSQL with depot/non-depot Scanner workflows. In this document, you will find requirements and YAML configurations to connect to MySQL for extracting entity metadata.

Requirements

  • Credentials to connect to the MySQL database.
  • To extract metadata, the MySQL user needs to have access to the INFORMATION_SCHEMA.
  • Metadata Scan is supported for MySQL (version 8.0.0 or greater)
  • MySQL user must grant SELECT privilege to fetch the metadata of tables and views.

Depot Scan Workflow

DataOS allows you to connect to a database with the JDBC driver to read data from tables using Depot. You can also scan metadata from a MySQL-type depot with the Scanner workflows. The Depot enables access to all schemas visible to the specified user in the configured database, MySQL.

You can apply database, schema, and table filter patterns while scanning metadata.

version: v1
name: mysql-scanner
type: workflow
tags:
  - mysql-scanner
description: The workflow scans schema tables and registers metadata
workflow:
  dag:
    - name: mysql-scanner
      description: The job scans schema from MySQL depot tables and registers metadata to metis2
      spec:
        tags:
          - scanner
        stack: scanner:2.0
        compute: runnable-default
                runAsUser: metis
        stackSpec:
          depot: mysql01
          sourceConfig:
            config:
              markDeletedTables: false
              includeTables: true
              includeViews: true
              databaseFilterPattern:
                  includes:
                      - database1
                      - database2
                  schemaFilterPattern:
                    includes:
                      - schema1
                      - schema2
                  tableFilterPattern:
                    includes:
                      - table1
                      - table2

After the successful workflow run, you can check the metadata of scanned Tables on Metis UI.