Create a a Connected Data Store

Note

This page walks through how to create a connected Data Store but you can also convert a non-connected Data Store into a connected Data Store by editing an existing Data Store, see the next page for more details.

Connecting to your data store is a great way to let us do the heavy lifting for you. When you connect directly to a data store you enable the following:

  • Automatic extraction of metadata that relates to the schemas and fields of your data

  • Identification of potential data quality issues (e.g. fields with multiple data types)

  • Automated notifications of potential breaking changes when schemas within the data store change

Add a new Data Store

Start the process for creating a new data store by selecting the + Create Data Store button.

../../_images/create_data_store_btn.png

Select the Data Store type

Select the type of Data Store that you want to setup.

../../_images/select_data_store.png

For this example we will select Postgres, another one of our favorite databases at Django Lineage! But you can pick any that you’d like.

Enter the Data Store details

Add the details for your data store and continue to the next step:

../../_images/create_data_store_form1.png

Add the connection details

Now, select Yes, setup the connection. Each data store has it’s own unique set of attributes that need to be filled in. Postgres has the following attributes but the data store you selected will likely be different:

  • Host

  • Port

  • Username

  • Password

  • Postgres Database

../../_images/data_store_pg_connection_details.png

Connect through jump server

You have the option to connect through a jump host. Jump hosts provide an extra level of security to you by limiting the the sources of external connections available to connect to your database. We strongly recommend setting up jump servers to use for all of your data store connections. Select a pre-configured jump server to route all traffic to your database through:

../../_images/data_store_jump.png

Verify the connection

Once you have entered all of the required fields you can select Test Connection. If you see the connection success popup then you’re good to go!

../../_images/data_store_connection_success.png

If at first you don’t succeed, try, try again! In the event that you could not connect to your Data Store we attempt to raise any errors that occurred when connecting to the Data Store to show you why the connection failed. Examples include:

../../_images/data_store_connection_failure1.png

Or:

../../_images/data_store_connection_failure2.png

Allow teams to view data

Hit Next to continue to the last step. Here you can select the teams that you want to have the ability to view data within the data store. While everyone in your organization will be able to see the data store, it’s description, the data schemas within the data store and the fields within the data schema, only the people in these groups will be able to see the underlying sample data data. You can select more than one team to give access to.

../../_images/data_store_team_access.png

Because a data store is set up with a single connection (e.g. one username), data permissions within Tree Schema are given at the data store level and not at the schema or field level. If you have different levels of data access and visibility within a single data store it is suggested to set up a new data store within Tree Schema with a different user and set of permissions for each.

You’re done! Way to go connecting to your data store!