Solved

Getting data into GoodData from multiple applications

  • 16 February 2021
  • 2 replies
  • 121 views

Hello. I am looking around to choose an analytic platform for our customers. The use case is that we have several applications from which we need to take data and use them to build the analytics for the customers. What capabilities GoodData platform has to get the data in? I do not have one data source, but we have several applications running on our servers and in public cloud. Thank you. Tom

icon

Best answer by ZDS 16 February 2021, 15:03

Hi Tom,

 

GoodData supports two high-level architecture options:

  1. Ingesting data from an existing data warehouse (e.g. Snowflake, Redshift, BigQuery, S3 files, etc.) where you consolidate data from your sources yourself. You can choose the data warehouse, connectors, and data transformation tools that best suit your needs. The data are distributed to individual customer’s workspaces where your customer’s users execute their reports, dashboards, etc.. 
  2. Ingesting data directly from your data sources, staging them in GoodData data warehouse (called ADS), and distributing it to individual customer’s workspaces. 

This documentation topic https://help.gooddata.com/doc/en/building-on-gooddata-platform/data-preparation-and-distribution  provides more details.

In April we are going to launch a container-based deployment of the GoodData platform that you’ll be able to deploy to your datacenter (local, public or private cloud) side by side with your data source (again cloud data warehouses like Snowflake and many other databases like Postgres, etc.). Stay tuned.

 

ZD 

View original

2 replies

@Tom Harris one thing you may want to consider is scope vs. cost of your project and how tied to a specific vendor you want to be.

For example, let’s say you decide to go with ProviderA for your data ingestion of all your various sources.   You setup a data pipeline to perform timed data harvests from each source and you’ve got it configured to transform the data so you can perform analytics and dashboarding duties.

2-3 years down the line, you’ve learned a lot and you realize ProviderA wasn’t really the right fit for your evolving needs.  Now you need to find someone else.   Because ProviderA also did all the data ingestion and transformation, you’re going to have to spend a lot of time redoing that data pipeline.  Depending on how well that project was documented and managed, you may have to spend the same amount of time (again) understanding how you need to transform the data from those various sources so it can work together.

Alternatively, you could consider managing the data pipeline separately and manage your own data warehouse; which gives you a little more flexibility with what’s on the market today and may make it a little easier to implement the analytics and visualization aspect, but that will force some design thought around data security and data literacy by the managing group(s).

There’s advantages and disadvantages to both approaches and a poor implementation plan will cost you more in the end no matter what you do.  If you’re looking around today, I hope that you’ve clearly defined what “success” looks like for your project and that you’ve got a general exit strategy on what to do if “success” is not met.

 

 

Hi Tom,

 

GoodData supports two high-level architecture options:

  1. Ingesting data from an existing data warehouse (e.g. Snowflake, Redshift, BigQuery, S3 files, etc.) where you consolidate data from your sources yourself. You can choose the data warehouse, connectors, and data transformation tools that best suit your needs. The data are distributed to individual customer’s workspaces where your customer’s users execute their reports, dashboards, etc.. 
  2. Ingesting data directly from your data sources, staging them in GoodData data warehouse (called ADS), and distributing it to individual customer’s workspaces. 

This documentation topic https://help.gooddata.com/doc/en/building-on-gooddata-platform/data-preparation-and-distribution  provides more details.

In April we are going to launch a container-based deployment of the GoodData platform that you’ll be able to deploy to your datacenter (local, public or private cloud) side by side with your data source (again cloud data warehouses like Snowflake and many other databases like Postgres, etc.). Stay tuned.

 

ZD 

Reply