Skip to main content

​
Intro

Databricks is a cloud-based collaborative data science, data engineering, and data analytics platform that combines the best of data warehouses and data lakes into a lakehouse architecture. With Databricks you can access all your data, analytics, and AI on one lake house platform. The simple, open, and collaborative environment helps reduce the infrastructure complexity, keeps control of your data, and makes it easy for your teams to partner across the entire data and workflow. For more information about the Databricks API, visit their website. (https://docs.databricks.com/dev-tools/api/index.html ) The Databricks connector is a “Database” connector, meaning it retrieves data from a database using a query. In the Data Center, you can access the connector page for this and other Database connectors by clicking Database in the toolbar at the top of the window. You connect to your Databricks database in the Data Center. This topic discusses the fields and menus that are specific to the Databricks connector user interface. General information for adding DataSets, setting update schedules, and editing DataSet information is discussed in Adding a DataSet Using a Connector. Note: Contact your AE or CSM to have it enabled in your instance.

​
Prerequisites

To connect to a Databricks database and create a DataSet, you must have the following:
  • The hostname for the database
  • The port number for the database
  • The database name or schema name
  • Databricks personal access token
  • The HTTP Path

​
Connecting to Your Databricks Database

This section enumerates the options in the Credentials and Details panes on the Databricks Connector page. The components of the other panes on this page, Scheduling, and Name & Describe Your DataSet, are universal across most connector types and are discussed in greater length in Adding a DataSet Using a Connector.
Note: When using the copy/paste function for your credentials, ensure there is no whitespace at the beginning or end of the string. By accidentally pasting whitespace, it will cause the connector to an error when trying to connect.

​
Credentials Pane

This pane contains fields for entering credentials to connect to your database. The following table describes what is needed for each field:

Field

Description

Host

Enter the hostname for the Databricks database.

Example: db.company.com

Port

Enter the port number for the Databricks database.

Database

Enter the name of the Databricks database.

Personal Access Token

Enter your Databricks personal access token.

HTTP Path Enter the HTTP path.
Once you have entered valid Databricks credentials, you can use the same account any time you go to create a new Databricks DataSet. You can manage connector accounts in the Accounts tab in the Data Center . For more information about this tab, see Manage Connector Accounts.

​
Details Pane

In this pane you create an SQL query to pull data from your database, with or without a parameter.

Menu

Description

Query Type

Select the desired query type.

Query Type Description
Custom Query Enter the query to execute.
Query Builder Select a table and fields to autogenerate your query.

Query

Enter the Structured Query Language (SQL) query to use in selecting the data you want.

Example: select * from Employee

You can use the Query Helper parameter to help you write a usable SQL query. To use the Query Helper , do the following:

  1. Select your database table and table columns in the appropriate menus.

  2. Copy the SQL statement that appears in the Query Helper field.

  3. Paste the copied SQL statement into the Query field.

Database Tables

Select the database table you want to import into Domo.

Table Columns

Select the table columns you want to import into Domo.

Query Helper

Copy and paste the SQL statement in this field into the Query field. For more information, see Query , above.

Fetch Size

Enter the fetch size for memory performance. The default value will be used if no fetch size is specified. If an ” out of memory ” error occurs, retry decreasing the fetch size.

Partition Support Format

Select the Year, Day and Month format of the data you would like to retrieve.

Fetch Size

Enter the fetch size for memory performance. The default value will be used if no fetch size specified. If an “out of memory” error occurs, retry decreasing the fetch size.

How you would like to import data into Domo?

You can import the data from the following:

  • Standard Update (replace/append)
  • User partition
  • Use upsert

​
Other Panes

For information about the remaining sections of the connector interface, including how to configure scheduling, retry, and update options, see Adding a DataSet Using a Connector.

​
FAQs

What kind of credentials do I need to power up this connector? You need the host name, port number, database name of your Databricks database, HTTP path and your Databricks personal access token. Where can I find the values that I need to enter for my credentials? You can find the hostname, database, port number, and HTTP path by going to your cluster in Databricks and viewing the JDBC/ODBC tab in the Advanced section of the cluster details. How frequently will my data update? As often as needed. Are there any API limits that I need to be aware of? Limits depend on your server configuration. What do I need to be aware of while writing a query? Make sure that all the words, table names, and field names are correctly spelled. Refer to the Query Helper field for query help. What’s the Fetch Size? The fetch size is for memory performance. The default value will be used if no fetch size is specified. If an “out of memory” error occurs, retry decreasing the fetch size. What types of queries are supported by DataBricks Connector? The Databricks read connector supports only read operations (if you are familiar with CRUD), and also supports where clauses, joins, group by, subselect, having, case, and so on. If you use the query builder option, it will only generate a simple “select query”. If you want to use anything else in your query, you must do in the custom query field.