Looker

Tree Schema can access your Looker metadata to extract the dashboard elements that are used to create Dashboards, the Looks you have saved within each of your respective folders and the Explores that are used to create your Looks and dashboards. Tree Schema is able to automatically generate data lineage for content created in Looker.

Connecting to Looker

The following fields are required for Looker:

  • Looker Server

  • Client ID

  • Client Secret

You can optionally choose verify SSL when connecting to your Looker server.

For details on how to create a client ID and client secret please refer to the Looker Documentation.

../../_images/looker_connection.png

Note

Looker connects directly to the endpoint provided via HTTPS and is not eligible to connect through a jump server.


Automated Schema Extraction

Tree Schema will automatically extract the following items from your Looker instance:

  • All Explores in your Looker instance

  • All dashboard elements and their corresponding fields

  • All Looks and the fields used within each Look

When extracting metadata, Tree Schema automatically creates the name of the schema as well as the location. Within Tree Schema each of these values is unique and generally should not be changed as it may cause future extrations and metadata Synchronization to break. The conventions for defining the names and locations are:

LookML Explores

Within Looker there can be mutliple projects, multiple LookML Models within each project and multiple Explores within each LookML. To ensure uniqueness within Tree Schema, Explores are given the following naming convention in Tree Schema:

{Looker Project Name}: {LookML Model Label}, {Explore Label}

And the location is defined as:

Project: {Looker Project Name}, LookML: {LookML Model Name}, Explore: {Explore Name}

Examples of this are:

../../_images/looker_explore_examples.png

Dashboard Elements

Dashboard elements are the individual tiles that make up a dashboard, they given the following naming convention:

{Dashboard Folder}: {Dashboard Title} - {Dashboard Element Title}

The location is defined as:

Folder: {Dashboard Folder}, Dashboard({Dashboard ID}): {Dashboard Title}, Element({Dashboard Element ID}): {Dashboard Element Title}

Examples of this are:

../../_images/looker_dash_element_examples.png

Looks

Since there can be several Looks that are given the same name across different folders wtihin Looker, Looks are given the following schema name in Tree Schema to provide uniqueness:

{Looker Folder}: {Look Title}

And the location is created as:

Folder: {Look Folder}, Look({Look ID}): {Look Title}

Examples of this are:

../../_images/looker_look_examples.png

Looker Data Lineage Overview

When extracting data lineage from Looker, Tree Schema will create lineage for the following:

  • Explores fields to Dashboard elements

  • Explore fields to Looks

  • Your upstream data source fields to Explore fields

When capturing the data lineage, Tree Schema does not include any intermediary fields. For example, if field “A” exists in your database and field “A” is used to create field “B” in your LookML Explore and in turn field “B” is used to create field “C” in your dashboard, then the lineage captured by Tree Schema would be “A” to “C”.

The image below depicts how Tree Schema will automatically create your data lineage from your source data store to an Explore and from the Explore to a dashboard element.

../../_images/looker_lineage.png

Starting the Data Lineage Extraction

To trigger the automated data lineage extraction from Looker navigate to your Looker Data store and select the Lineage Tab. Here you will have the option to trigger the lineage extraction.

Note

Make sure that you have already captured all of your fields and schemas from Looker before you extract the data lineage. Tree Schema will only create data lineage for the data assets that it knows about!

../../_images/lineage_extraction_btn.png

This will display a prompt that contains some details about how the data lineage is captured. If you have connected Tree Schema and Looker to the same upstream data sources then Tree Schema will attempt to connect your Looker Explores to the source data in the upstream data store.

For example, if you use Snowflake for your data warehouse and are creating your Looker Explores from Snowflake, if you have already extracted the Snowflake metadata into Tree Schema then Tree Schema will try to create data lineage from your Snowflake tables and views into your Looker Explores. You will need to specify which upsteam data sources you have connected to both Looker and Tree Schema.

../../_images/looker_lineage_extraction.png

Select Start Lineage Capture to begin the extraction. You may trigger this process up to once every 24 hours.

Note

Tree Schema aims to be 100% accurate when generating the automated lineage but may not be 100% comphrenesive when creating lineage in all cases. For example, Tree Schema is not able to generate lineage for derived tables in Explores.


Synchronizing Metadata

Tree Schema will check to make sure that your metadata between Looker and Tree Schema is kept in sync. This includes validating that the descriptions of your data in Looker matches the description of your data in Tree Schema. When your data is in sync you will see the successful synchronization icon on the README:

../../_images/readme_success_sync.png

Tree Schema will push your README updates to Looker for dashboard elements and Looks. When Looker does not provide a method for Tree Schema to push metadata updates into Looker you will see the following icon indicating that synchronization is not available. In these cases the metadata documentation is only available in Tree Schema.

../../_images/readme_no_sync.png