Home

AI Sync

Jump to Step 3 for 'AI Sync'

Step 1 – Create a Data Environment

  • Add a new Data Environment (e.g., Production, Staging, Development, or a specific business unit).

  • Each Organization can support unlimited Data Environments (depending on your plan: Team vs Enterprise).

Step 2 – Configure a Data Connection

Each Data Environment requires a data connection before syncing can begin.

  1. Select your data source (e.g., PostgreSQL, Snowflake, Databricks, BigQuery, Amazon Athena).

  2. Enter the required credentials. Refer Integrations to check pre-requisites for your data source.

  3. Use Test & Save to verify the connection.

  4. (Optional Step for Advanced Customization) Select the scope for synchronization, execution and materialization. You can also skip changing the scope and directly click on 'Save'.

  5. After you click on 'Save', the system starts the auto-sync process.

Example:

  • Select the data scope

  • Cost limits: As part of the setup process, you can define a Cost Limit for each data connection. This limit controls the maximum query and sampling costs allowed in the environment.

    Set cost limit
    Set cost limit

Step 3 – AI Sync

After the connection is verified, the sync process runs automatically.

It consists of 5 stages, visible in the Environment Dashboard:

  1. Data Environment Catalog Discovery

    • Detects schemas, tables, views, and objects in your environment.

    • Shows detailed progress by columns, databases, objects, and schemas.

  2. Data Workspace Catalog Initialization

    • Builds the catalog structure inside your Data Workspace.

    • Ensures that objects are organized and ready for use.

  1. Inferring PII by AI

  2. Data Statistics Collection:

    1. During this step, the system gathers column-level metadata and statistics to help the AI learn and better understand your data.

  3. Data Workspace Catalog Finalization:

    1. The sync concludes by finalizing the Data Workspace Catalog, making the metadata available in a human-readable format under 'Context Engine' module for data governance.

Step 4 – Create Additional Data Workspaces (Optional)

  • Within each Data Environment, you can add unlimited Workspaces (depending on your Plan Tier).

  • Workspaces allow you to restrict access to specific departments or use cases (e.g., Marketing, Finance, Product).

  • Users can be assigned to multiple Workspaces at no extra cost.

Monitoring Your Environment

The Data Environment Overview Dashboard provides:

  • Number of configured connections.

  • Number of workspaces.

  • Last sync status (with live progress).

This gives you a real-time snapshot of your environment’s settings.

Last updated