---
type: Deployment Guide
title: Deploy ScalarDB Analytics in Public Cloud Environments
description: This guide explains how to deploy ScalarDB Analytics in a public cloud environment. ScalarDB Analytics currently uses Apache Spark as an execution engine and supports managed Spark services provided by public cloud providers, such as...
resource: https://scalardb.scalar-labs.com/docs/3.15/scalardb-analytics/deployment/
tags:
- scalardb
- v3.15
- phase:operate
- section:deploy
- edition:enterprise-option
- unmaintained
status: deprecated
product: scalardb
product_title: ScalarDB
version: '3.15'
patch_version: 3.15.9
doc_id: scalardb-analytics/deployment
lifecycle_phase: operate
breadcrumb:
- Deploy
- Deploy ScalarDB Analytics
editions:
- Enterprise Option
generated:
  by: process:okf-build/1.0.0
  at: '2026-08-04T23:50:56Z'
sources:
- id: docs-scalardb
  resource: https://github.com/scalar-labs/docs-scalardb/blob/6126dfe2f56389351d88b134752618641f9771dd/versioned_docs/version-3.15/scalardb-analytics/deployment.mdx
  title: ScalarDB documentation source (MDX)
  author: process:scalar-labs/docs-scalardb
  last_modified: '2026-08-04T15:05:02Z'
---

# Deploy ScalarDB Analytics in Public Cloud Environments

This guide explains how to deploy ScalarDB Analytics in a public cloud environment. ScalarDB Analytics currently uses Apache Spark as an execution engine and supports managed Spark services provided by public cloud providers, such as Amazon EMR and Databricks.

## Supported managed Spark services and their application types

ScalarDB Analytics supports the following managed Spark services and application types.

| Public Cloud Service        | Spark Driver | Spark Connect | JDBC |
| -------------------------- | ------------ | ------------- | ---- |
| Amazon EMR (EMR on EC2)    | ✅           | ✅            | ❌   |
| Databricks                 | ✅           | ❌            | ✅   |

## Configure and deploy

Select your public cloud environment, and follow the instructions to set up and deploy ScalarDB Analytics.

**Amazon EMR**

### Use Amazon EMR

You can use Amazon EMR (EMR on EC2) to run analytical queries through ScalarDB Analytics. For the basics to launch an EMR cluster, please refer to the [AWS EMR on EC2 documentation](https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan.html).

#### ScalarDB Analytics configuration

To enable ScalarDB Analytics, you need to add the following configuration to the Software setting when you launch an EMR cluster. Be sure to replace the content in the angle brackets:

```json
[
  {
    "Classification": "spark-defaults",
    "Properties": {
      "spark.jars.packages": "com.scalar-labs:scalardb-analytics-spark-all-<SPARK_VERSION>_<SCALA_VERSION>:<SCALARDB_ANALYTICS_VERSION>",
      "spark.sql.catalog.<CATALOG_NAME>": "com.scalar.db.analytics.spark.ScalarDbAnalyticsCatalog",
      "spark.sql.extensions": "com.scalar.db.analytics.spark.extension.ScalarDbAnalyticsExtensions",
      "spark.sql.catalog.<CATALOG_NAME>.license.cert_pem": "<YOUR_LICENSE_CERT_PEM>",
      "spark.sql.catalog.<CATALOG_NAME>.license.key": "<YOUR_LICENSE_KEY>",

      // Add your data source configuration below
    }
  }
]
```

The following describes what you should change the content in the angle brackets to:

- `<SPARK_VERSION>`: The version of Spark (`3.5` or `3.4`).
- `<SCALA_VERSION>`: The version of Scala used to build Spark (`2.13` or `2.12`).
- `<SCALARDB_ANALYTICS_VERSION>`: The version of ScalarDB Analytics.
- `<CATALOG_NAME>`: The name of the catalog.
- `<YOUR_LICENSE_CERT_PEM>`: The PEM encoded license certificate.
- `<YOUR_LICENSE_KEY>`: The license key.

For more details, refer to [Set up ScalarDB Analytics in the Spark configuration](./run-analytical-queries.md#set-up-scalardb-analytics-in-the-spark-configuration).

#### Run analytical queries via the Spark driver

After the EMR Spark cluster has launched, you can use ssh to connect to the primary node of the EMR cluster and run your Spark application. For details on how to create a Spark Driver application, refer to [Spark Driver application](https://scalardb.scalar-labs.com/docs/3.15/scalardb-analytics/run-analytical-queries.mdx?spark-application-type=spark-driver-application#develop-a-spark-application).

#### Run analytical queries via Spark Connect

You can use Spark Connect to run your Spark application remotely by using the EMR cluster that you launched.

You first need to configure the Software setting in the same way as the [Spark Driver application](https://scalardb.scalar-labs.com/docs/3.15/scalardb-analytics/run-analytical-queries.mdx?spark-application-type=spark-driver-application#develop-a-spark-application). You also need to set the following configuration to enable Spark Connect.

##### Allow inbound traffic for a Spark Connect server

1. Create a security group to allow inbound traffic for a Spark Connect server. (Port 15001 is the default).
2. Allow the role of "Amazon EMR service role" to attach the security group to the primary node of the EMR cluster.
3. Add the security group to the primary node of the EMR cluster as "Additional security groups" when you launch the EMR cluster.

##### Launch the Spark Connect server via a bootstrap action

1. Create a script file to launch the Spark Connect server as follows:

```bash
#!/usr/bin/env bash

set -eu -o pipefail
cd /var/lib/spark
sudo -u spark /usr/lib/spark/sbin/start-connect-server.sh --packages org.apache.spark:spark-connect_<SCALA_VERSION>:<SPARK_FULL_VERSION>,com.scalar-labs:scalardb-analytics-spark-all-<SPARK_VERSION>_<SCALA_VERSION>:<SCALARDB_ANALYTICS_VERSION>
```

    The following describes what you should change the content in the angle brackets to:

- `<SCALA_VERSION>`: The major and minor version of Scala that matches your Spark installation (2.12 or 2.13).
- `<SPARK_FULL_VERSION>`: The full version of Spark you are using (such as 3.5.3).
- `<SPARK_VERSION>`: The major and minor version of Spark you are using (3.4 or 3.5).
- `<SCALARDB_ANALYTICS_VERSION>`: The version of ScalarDB Analytics.

2. Upload the script file to S3.
3. Allow the role of "EC2 instance profile for Amazon EMR" to access the uploaded script file in S3.
4. Add the uploaded script file to "Bootstrap actions" when you launch the EMR cluster.

##### Run analytical queries

You can run your Spark application via Spark Connect from anywhere by using the remote URL of the Spark Connect server, which is `sc://<PRIMARY_NODE_PUBLIC_HOSTNAME>:15001`.

For details on how to create a Spark application by using Spark Connect, refer to [Spark Connect application](https://scalardb.scalar-labs.com/docs/3.15/scalardb-analytics/run-analytical-queries.mdx?spark-application-type=spark-connect#develop-a-spark-application).

**Databricks**

### Use Databricks

You can use Databricks to run analytical queries through ScalarDB Analytics.

:::note

Note that Databricks provides a modified version of Apache Spark, which works differently from the original Apache Spark.

:::

#### Prepare the secret values for the license certificate and license key

Store the license certificate and license key in the cluster by using the Databricks CLI.

```console
databricks secrets create-scope scalardb-analytics-secret # you can use any secret scope name
cat license_key.json | databricks secrets put-secret scalardb-analytics-secret license-key
cat license_cert.pem | databricks secrets put-secret scalardb-analytics-secret license-cert
```

:::note

For details on how to install and use the Databricks CLI, refer to the [Databricks CLI documentation](https://docs.databricks.com/en/dev-tools/cli/index.html).

:::

#### Prepare an init script for loading the ScalarDB Analytics library JAR

1. Download the ScalarDB Analytics library JAR file from the Maven repository. Choose the appropriate JAR file based on your Spark, Scala, and ScalarDB versions:
   - [scalardb-analytics-spark-all-3.4_2.13 (Spark v3.4, Scala v2.13)](https://repo1.maven.org/maven2/com/scalar-labs/scalardb-analytics-spark-all-3.4_2.13/)
   - [scalardb-analytics-spark-all-3.5_2.13 (Spark v3.5, Scala v2.13)](https://repo1.maven.org/maven2/com/scalar-labs/scalardb-analytics-spark-all-3.5_2.13/)
   - [scalardb-analytics-spark-all-3.4_2.12 (Spark v3.4, Scala v2.12)](https://repo1.maven.org/maven2/com/scalar-labs/scalardb-analytics-spark-all-3.4_2.12/)
   - [scalardb-analytics-spark-all-3.5_2.12 (Spark v3.5, Scala v2.12)](https://repo1.maven.org/maven2/com/scalar-labs/scalardb-analytics-spark-all-3.5_2.12/)
2. Upload the JAR file to the Databricks workspace.
3. Create an init script as follows, replacing `<PATH_TO_YOUR_JAR_FILE_IN_WORKSPACE>` with the path to your JAR file in the Databricks workspace:

```bash
#!/bin/bash

# Target directories
TARGET_DIRECTORIES=("/databricks/jars" "/databricks/hive_metastore_jars")
JAR_PATH="<PATH_TO_YOUR_JAR_FILE_IN_WORKSPACE>"

# Copy the JAR file to the target directories
for TARGET_DIR in "${TARGET_DIRECTORIES[@]}"; do
 mkdir -p "$TARGET_DIR"
 cp "$JAR_PATH" "$TARGET_DIR/"
done
```

4. Upload the init script to the Databricks workspace.

#### Launch Databricks compute

ScalarDB Analytics works with all-purpose compute on Databricks. When you launch compute, you need to configure the compute to enable ScalarDB Analytics as follows:

1. Select `Create compute` in the `Compute` menu.
2. Select `Unrestricted` from the `Policy` dropdown menu.
3. Select an appropriate Databricks runtime version that supports Spark 3.4 or 3.5.
4. Go to the `Advanced` section. In the `Access mode` tab, select `Manual` as the access mode, and choose `No isolation shared`.
5. In the `Advanced` section, select the `Spark` tab, and enter the following configurations in `Spark config`:

```
spark.sql.extensions com.scalar.db.analytics.spark.extension.ScalarDbAnalyticsExtensions
spark.sql.catalog.<CATALOG_NAME> com.scalar.db.analytics.spark.ScalarDbAnalyticsCatalog
spark.sql.catalog.<CATALOG_NAME>.license.cert_pem <YOUR_LICENSE_CERT_PEM>
spark.sql.catalog.<CATALOG_NAME>.license.key <YOUR_LICENSE_KEY>
```

    Replace the placeholders:

- `<CATALOG_NAME>`: The name of the catalog.
- `<YOUR_LICENSE_CERT_PEM>`: The PEM encoded license certificate.
- `<YOUR_LICENSE_KEY>`: The license key.

6. In the `Advanced` section, select the `init scripts` tab, and specify the path to the init script in the workspace you uploaded.
7. Select `Create`.

#### Run analytical queries via the JDBC driver

Databricks supports JDBC to run SQL jobs on compute.
After compute is launched, you can get the JDBC URL of the compute in the `Advanced` > `JDBC/ODBC` tab. To connect to the compute by using JDBC, you need to add the Databricks JDBC driver to your application dependencies. For example, if you are using Gradle, you can add the following dependency to your `build.gradle` file after replacing `<DRIVER_VERSION>` with the version of the Databricks JDBC driver you want to use:

```groovy
implementation("com.databricks:databricks-jdbc:<DRIVER_VERSION>")
```

Then, you can connect to the compute by using JDBC with the JDBC URL (`<YOUR_COMPUTE_JDBC_URL>`), as is common with JDBC applications.

```java
Class.forName("com.databricks.client.jdbc.Driver");
String url = "<YOUR_COMPUTE_JDBC_URL>";
Connection conn = DriverManager.getConnection(url);
```

For more details on how to use JDBC with Databricks, refer to the [Databricks JDBC Driver documentation](https://docs.databricks.com/en/integrations/jdbc/index.html).
