What are the storage options with Toric?

Toric is a powerful data analytics workspace that provides data analysts, project teams, and business stakeholders with comprehensive data integration, engineering, visualization, and analysis tools. Our workspace provides flexible and scalable data storage options to provide users with the tools and environment needed to easily collect, manage, and analyze large data sets. It supports both transient data storage options such as Databricks, Google BigQuery, Azure, and AWS, and Toric managed data warehouses. Customers can choose where their temporary tables may reside to ensure compliance with local data regulations. Toric's data storage capabilities include:

Data Staging

Toric provides scalable data residency options to store raw data and processes it through a pipeline to clean, filter, and transform it into a structured format optimized for querying and analysis. 

Data processing techniques, such as data deduplication, normalization, and enrichment, are applied to ensure consistency, accuracy, and completeness. 

The structured data is then organized into tables and defined relationships through a data model. Business-relevant views of the data, known as data views, are created for consumption by BI tools, applications, or end-users.

Toric Transient Data Storage

Toric can store transient data until it reaches its destination, allowing the customer to decide where temporary tables may reside. User options for transient data storage are:

• Data Bricks
• Google BigQuery
• Azure
• AWS 

Managed Data Lakes / Warehouse(s)

In the Toric workspace, a Data Warehouse is a physical instance of a database deployed and managed by Toric. Data Warehouses are intended to be used to store final clean data that's used for reporting, or synching. Toric data-warehouses allow users to control the region for deployment, backup region including US, APAC and EMEA regions. 

With Data Warehouses users create tables with custom schemas supporting Primary keys, Dates, Currencies, Numbers, Units of measure, Lists, Boolean, even BIM models. Like data-tables, data-warehouses can be read/write from flows using the Write to warehouse, Select, Distinct and Summarize nodes.

Getting Started with Data Tables

Data tables are a critical feature of data storage in Toric that allows users to create and manage structured data tables easily. Users can create new tables and define their structure, including columns and data types. They can also set up primary keys and determine whether columns are required or not.

Let's dive into a step-by-step guide on how to create and use Data Tables in Toric.

Step 1: Accessing Data Tables 

To access data tables,navigate to the left panel menu and click on the data tables option. Then, click on "Create a new table."

Step 2: Defining Table Name and Columns 

Once you have accessed data tables, set up your table name. Here you can define the data types and columns in this data table. You have full control over the structure of this table, and you can use this table to aggregate information from different sources into one well-structured table.

Step 3: Adding more columns and data types

You can add more columns with different data types as needed. Additionally, you can choose whether you want one of the columns in your table to be a primary key. This works similarly to primary keys in other tools. If the data associated to the primary key already exists in the table, it will overwrite the data, amd if not, it's gonna add on to it.

Step 4: Writing Data to the Table 

After creating the data table, you can begin writing and reading data to the table within a data flow. To do this, create a new project. Add a source to your flow or use one of our sample sources. For this example, I am using a Google Sheet that is available in our sample sources. 

There are two nodes available in a data flow to perform our data warehouse operations - Select from Warehouse or Write to warehouse. 

This Write to warehouse node will map and write data to an internal or external warehouse. I am going to use the data table I created in my example above. I am going to select the data table I created as the destination warehouse, and I am going to select the columns that I want to write to the data table.

For this example, I want column one to contain data from Segment and column two to contain from Units sold. 

Once the columns and destinations are selected, you can press Write now to write the data. You will receive a notification when you data has been successfully pushed to the warehouse.

Step 5: Accessing data in the warehouse 

To access data in the warehouse, use the "Select from warehouse" node in a data flow. Select from warehouse allows Toric users to pick from a warehouse and query that data in a format very similar to a SQL query. 

Once you’ve created a new Select from warehouse node, you can select the destination you would like to query data from. I can now select a table, run query after putting in a limit, or I can add conditions.

In this case, I want to do a simple query, and with that simple query I can access all 2100 rows of data available within the warehouse. This data can then be used in flows for processing, cleansing, analysis, and visualzation.

Data tables are particularly useful when you want to combine data from different sources into one place, such as multiple versions of the same project, which is a feature in Toric, or different projects with the same data type, so you can compare pro projects against each other

How to get started with flexible data storage in Toric's data analytics workspace 

Toric's workspace provides a powerful tool for data storage and aggregation. With its data tables feature, you can combine data from different sources into one well-structured Table. Additionally, Toric's data storage capabilities, including data staging, transient data storage, and managing data warehouses, make it an excellent choice for storing and managing your data.

We hope this tutorial has been helpful in getting you started with Toric's data storage capabilities. With Toric, you can easily store, clean, and process your project and business data to create insights that drive better decision-making. We encourage you to sign up for a free account and explore the workspace with course videos if you haven't already.

And if you need any help getting started, don't hesitate to request a demo with our team to learn more about how Toric can help you manage your projects more efficiently. Try out Toric for free today and see for yourself the power of our all-in-one data analytics workspace!