View Data Schema Details

The details page for a Data Schema gives you everything you need to know about the Data Schema all in one place.

../../_images/data_schema_overview.png

Title

The title section contains the name of the Data Schema, basic information about the Data Schema points of contact and schema type, as well as the button required to edit the Schema.

If the Data Store that this Data Schema resides in has been connected and you have the ability to view data within this Data Store then you will see a badge indicating that you are connected.

../../_images/data_schema_connected.png

When you are viewing a Data Schema that was created manually, if the Schema sits in a Data Store that has not been connected to Django Lineage, if you do not have permissions or if the existing connection is no longer valid (e.g. password change) then you will see the title without the connected badge:

../../_images/data_schema_not_connected.png

README

The Data Schema is a great place to provide information about common usage patterns, sample SQL queries, information about the frequency and volume of data for this schema and much more.

../../_images/data_schema_readme.png

Attributes

Attributes can be thought up as key-value pairs. Unlike Data Stores, there are no default attributes for Data Schemas, however, we believe this is a great place to store things such as the GitHub link to the Schema creation script or other similar values that you want to relate to the Schema.

../../_images/data_schema_attributes.png

Tags

Tags are a great way to add quick labels to your Schema.

../../_images/data_schema_tags.png

Details

The Details section of the Data Schemas contains everything about your Data Store fit’s into your overall data ecosystem.

Each of the sections below represents a tab that may be available to your Data Schema if applicable.


Fields

All of the Fields that reside within the Data Schema are listed here, you can navigate directly to the details for the Field by clicking on the Visit Field button.

../../_images/data_schema_fields.png

Non-Tabular Data

We know that not all data is tabular and it doesn’t always fit nice and neat into a table. We pride ourselves on having first-class support for other structured data types such as JSON, Parquet, or Avro to name a few. These types of Schemas often have hierarchical structures, embedded objects, and lists that can further contain objects or scalar values.

When you have these types of schemas you will notice that the fields which are objects or lists of objects end with “<”. This is telling you that this is a parent field and it has one or more children field below it. Click anywhere on this row to expand the field and view it’s children:

../../_images/data_schema_parent_field1.png

You can continue this for as many child fields exist in your schema. The rows will be color coordinated by level to help you to keep track of which fields belong to which parent.

../../_images/data_schema_parent_field2.png

Full Schema

You can view your entire schema at a glance, we provide you a snapshot view as a JSON object with the keys being the name of the field and the values being the data type.

Note

For fields that have scalar values, the data type is displayed, fields that have the data type of object or list have the actual object or list as their value

../../_images/data_schema_fields_schema_json.png

You can also toggle this to YAML if you prefer:

../../_images/data_schema_fields_schema_yaml.png

Governance

Any outstanding governance actions for this Data Schema as well as any of the Fields within this Data Schema are displayed here:

../../_images/data_store_governance.png

Experts

The two types of Experts are displayed for Data Schemas:

  1. Power Users: the users that use this Data Store the most often

  2. Volunteer SMEs: the users who raise their hands and volunteer as experts

../../_images/data_store_experts.png

Comments

You can add, edit or reply to comments for your Data Schemas.

../../_images/data_store_comments.png

Lineage

Lineage tracks the relationships between fields and displays those relationships as they exist within Data Schemas and Data Stores. Lineage provides you a way to interact and ask questions to your data, such as “What other parts of my ecosystem does this schema interact with?” or “Is this schema impacted by the change to an upstream ETL process?”.

When you view the lineage from a Data Schema it initially only loads the linege that either goes into or out of the Data Schema.

../../_images/data_schema_lineage.png

To learn more about how to use lineage, visit the Lineae and Exploration page.