Skip to content
Alien Giraffe

Configure Data Sources

This guide is an API-first companion to the API Reference. Data source configuration in a10e-manager is managed through the admin datasource endpoints documented in the OpenAPI contract at /src/openapi.yaml in the docs repository.

Use this page to understand the flow. Use the API Reference for the exact request and response schema.

Supported Datasource Types

Section titled “Supported Datasource Types”

The current datasource types exposed by a10e-manager are:

  • postgres
  • oracle
  • mysql
  • snowflake
  • s3
  • ssh

Core Admin Endpoints

Section titled “Core Admin Endpoints”

These are the primary endpoints used to manage datasource definitions:

All of them are part of the admin datasource surface in the API Reference.

Configuration Flow

Section titled “Configuration Flow”

The normal datasource lifecycle is:

  1. List existing datasources with GET /v1/admin/datasources.
  2. Create a datasource with POST /v1/admin/datasources.
  3. Adjust non-secret fields or rotate secrets with PATCH /v1/admin/datasources/{id}.
  4. Update scan behavior with PATCH /v1/admin/datasources/{id}/catalog-policy.
  5. Disable a datasource with DELETE /v1/admin/datasources/{id} when it should no longer be used.
  6. Restore it with POST /v1/admin/datasources/{id}/restore if needed.

If you need to review disabled entries first, use GET /v1/admin/datasources/disabled.

Create A Datasource

Section titled “Create A Datasource”

Datasource creation uses the AdminDatasourceUpsertRequest schema from the OpenAPI contract. The request body contains:

  • name
  • type
  • scan_policy
  • secret_backend
  • config_fields

The important field is config_fields. Each entry follows the AdminDatasourceConfigFieldInput shape:

  • key
  • value
  • is_secret
  • clear

Example:

{
  "name": "production-postgres",
  "type": "postgres",
  "scan_policy": "enabled",
  "secret_backend": "file",
  "config_fields": [
    { "key": "host", "value": "db.company.com", "is_secret": false },
    { "key": "port", "value": "5432", "is_secret": false },
    { "key": "database", "value": "production_db", "is_secret": false },
    { "key": "ssl_mode", "value": "require", "is_secret": false },
    { "key": "username", "value": "reader_user", "is_secret": true },
    { "key": "password", "value": "replace-me", "is_secret": true }
  ]
}

Use POST /v1/admin/datasources to submit that payload. Refer to the API Reference for the exact schema returned by AdminDatasourceResponse.

Secret Backends

Section titled “Secret Backends”

The current implementation supports these datasource secret backends:

  • file
  • java-keystore
  • aws-secrets-manager

secret_backend controls where secret-marked config values are written when you submit or update a datasource. The admin API is the write path for secret values; later reads return the datasource definition without exposing the stored secrets again.

In practice, the flow is:

  1. submit config_fields
  2. mark secret entries with is_secret: true
  3. choose the backend with secret_backend
  4. let a10e-manager store backend-managed references for those secret values

Those references are later resolved inside the request execution boundary rather than being returned as plaintext to admins or end users.

Update A Datasource

Section titled “Update A Datasource”

Use PATCH /v1/admin/datasources/{id} with the AdminDatasourceUpdateRequest schema when you need to:

  • change non-secret connection fields
  • rotate credentials
  • add new config fields
  • clear previously stored values

Updates use the same config_fields model as create. Existing secret values remain in place unless you explicitly replace or clear them.

Example update:

{
  "scan_policy": "enabled",
  "config_fields": [
    { "key": "schema", "value": "analytics", "is_secret": false },
    { "key": "password", "value": "rotated-secret", "is_secret": true }
  ]
}

Manage Scan Policy

Section titled “Manage Scan Policy”

Datasource scan policy is managed separately through:

That endpoint uses the AdminDatasourcePolicyUpdateRequest schema:

{
  "scan_policy": "enabled"
}

Use it when you want to change catalog scanning behavior without editing the broader datasource definition.

Disable, Restore, And Permanently Hide

Section titled “Disable, Restore, And Permanently Hide”

The admin datasource API distinguishes between disabling and permanently hiding:

To inspect disabled entries before restoring or hiding them, use:

Secret Handling

Section titled “Secret Handling”

The admin datasource API separates non-secret and secret configuration:

  • non-secret fields are stored as normal config fields
  • secret fields are submitted through config_fields with is_secret: true
  • secret_backend determines which backend receives those secret values
  • secret values are not returned as plaintext after save
  • the datasource keeps backend-managed references rather than raw credential material

This means the create and update payloads carry the write-time values, while subsequent reads return the datasource definition without exposing stored secret material.

Type-Specific Examples

Section titled “Type-Specific Examples”

The exact config keys depend on the datasource type. A few common examples:

PostgreSQL

Section titled “PostgreSQL”

Typical keys:

  • host
  • port
  • database
  • ssl_mode
  • schema
  • username as secret
  • password as secret

Oracle

Section titled “Oracle”

Typical keys:

  • host
  • port
  • database or service_name
  • username as secret
  • password as secret

MySQL

Section titled “MySQL”

Typical keys:

  • host
  • port
  • database
  • charset
  • username as secret
  • password as secret

SSH

Section titled “SSH”

Typical keys:

  • host
  • port
  • path
  • username
  • password as secret, or private-key material as secret

S3

Section titled “S3”

Typical keys:

  • bucket
  • region
  • prefix
  • access_key_id as secret
  • secret_access_key as secret
  • session_token as secret when temporary credentials are used
Section titled “Related References”