To GoodData homepage

Data modeling & Integration

Data modeling & Integration

92 Topics

R
Ron BorensteinNew Participant
asked in Data models

CSV missing values

Hi, I am new to GoodData and trying to get started, but having an issue I can’t get around.I am uploading a csv, created using a jupyter notebook. The file is 15.5mb and has 56k rows. Everything seems fine when I upload it, but when I try to create my first insight, the data is different than in the file.I have a measure called `timeSpent` . When I sum `timeSpent`  the number in GoodData is significantly lower than it is in the file.sum of timeSpent in GoodData - 1,849.10 sum of timeSpent in the file - 19,670,743.42How can I check what’s wrong? 

2
R
1 year ago
M
Mike S.New Participant
asked in Data sources

Help with properly setting up excel data for export to CSV

I’m looking to develop a dashboard based on a response to a series of survey questions. I’m having difficulty in understanding how to first organize the questions and scoring criteria with in excel for each dashboard module, which I can then export as a CSV.Any help or suggested excel templates I can use would be appreciated. Thank you.

1
Joseph Heun
GoodData Employee
1 year ago
R
Rick FlaggNew Participant
asked in Data models

Automating Data Load from Amason S3 Bucket

I am receiving a Connection Error trying to connect to my S3 bucket. I am the admin user on both the GOodData and Amason systems. My connection configuration is shown below:I am uncertain what is expected in the Access key and Secret key fields.

1
Iva Gasparekova
GoodData Employee
1 year ago
R
Rick FlaggNew Participant
asked in Data sources

Reviewing Data After CSV Import

When importing data from a CSV file, there is a step to review the first 50 lines of the data file prior to uploading the full file. Is there a facility to review the “raw” data AFTER importing and without creating a new insight?

2
Martin Burian
Community Manager
2 years ago
T
tf_brianNew Participant
asked in Data sources

Mapping Validation Failed column "cp__id" doesn't exist when it does actually exist.

The csv in question after downloading form S3The logs: 2021-04-02T22:16:13.640+0200 [INFO]: Data distribution worker started2021-04-02T22:16:13.640+0200 [INFO]: Request id: "data_load_uVsUQjWYuv_YTqtNkRU4v:DR7f6UuIF6vgQhKZ:otffmYmekgTqXoYn"2021-04-02T22:16:13.640+0200 [INFO]: Data source: "604fed9d72e8480930e9fe25"2021-04-02T22:16:13.640+0200 [INFO]: Additional parameters: {GDC_DATALOAD_DATASETS=[{"dataset":"dataset.testdata","uploadMode":"FULL"}], PROCESS_ID=1a159b0a-9204-42e1-bb2f-abb07aed7b23}2021-04-02T22:16:13.640+0200 [INFO]: Synchronization mode: selected datasets (default)2021-04-02T22:16:13.943+0200 [INFO]: Synchronized datasets: [dataset.testdata]2021-04-02T22:16:14.057+0200 [INFO]: ====================== Data distribution scope ======================2021-04-02T22:16:14.057+0200 [INFO]: Project="i9kr8f8fd7802orlklcvvhlsr74l057i"; datasets=[{dataset.testdata, loadDataFrom=2021-02-10T02:50:27}]2021-04-02T22:16:14.057+0200 [INFO]: ====================== End of Data distribution

3
D
GoodData Employee
2 years ago
T
Tom HarrisNew Participant
asked in Data sources

What is difference between ADS and other cloud data warehouses you support?

Hello. I can see that GoodData can provide the ADS data warehouse and also supports other data warehouses like Redshift. BigQuery, Snowflake… maybe more. Is there any reason or even advantage to use ADS over the cloud data warehouses? Thanks. Tom

1
M
GoodData Employee
2 years ago
J
John DoeNew Participant
asked in Data sources

Snowflake "All Projects Failed to Load" Error

I’ve connected Snowflake’s test data to GoodData, added the model and tried to load the data but getting an error that says “All Projects Failed to Load” error everytime I try to upload the data. Here is the error code:2021-02-26T17:33:08.304+0100 [ERROR]: Fail to load projects "[clr5mf5skkcnvxg3nez8a5zevd9urahu]". Reason: Unable to unload: StatementCallback; bad SQL grammar [CREATE OR REPLACE STAGE STAGE__20210226__GhFKNxK2 COPY_OPTIONS = ( on_error='skip_file' )]; nested exception is net.snowflake.client.jdbc.SnowflakeSQLException: SQL execution error: Creating stage on shared database 'SNOWFLAKE_SAMPLE_DATA' is not allowed.: SQL execution error: Creating stage on shared database 'SNOWFLAKE_SAMPLE_DATA' is not allowed.2021-02-26T17:33:08.310+0100 [INFO]: ====================== End of downloading and integrating data ======================2021-02-26T17:33:08.311+0100 [ERROR]: Data distribution worker failed. Reason: All projects failed to load.Has anyone experienced that and w

3
A
2 years ago
T
tf_brianNew Participant
asked in Data sources

S3 integration error

I'm trying to connect S3 to good data, I setup an IAM user on S3, got access credentials, setup a bucket, gave IAM user S3 access, setup get object bucket policy, included IAM user as principal for the bucket. When I test the connection it says "successful." When I try to load the data I get "Failed to load projects: Connection Validation failed" I tried to search various S3 errors and on GD forums but didn't see anything.  2021-03-16T20:26:35.418+0100 [ERROR]: Fail to load projects "[{project id here}]". Reason: Connection validation failed Thats pretty much all the logs say, so I’m a bit stumped considering the connection test is successful.

1
Joseph Heun
GoodData Employee
2 years ago
R
Rick FlaggNew Participant
asked in Data sources

Creating a New Data Field

I have a workspace with several datasets; dataset 1 and dataset 2. each dataset has a dollar value fact that I’m plotting. I’d like to provide an insight that splits the sum of dataset 1 value and dataset 2 value. Is this possible in GoodData or does my model need to calculate the totals outside of GoodData and inport into another dataset?

2
Martin Burian
Community Manager
2 years ago
R
Rick FlaggNew Participant
asked in Data sources

Cloning a workspace without including data

I have followed the instructions on Clone a Workspace via Gray Pages and obtain a new workspace with all the data model and insights; however, I get errors when importing new data into the new workspace. The error lists all fields from my import file as a_fieldname cannot be found. The only way I have found to import data is to rebuild the data model which destroys the insights and dashboards.

5
Iva Gasparekova
GoodData Employee
2 years ago
T
Tom HarrisNew Participant
asked in Data sources

Getting data into GoodData from multiple applications

Hello. I am looking around to choose an analytic platform for our customers. The use case is that we have several applications from which we need to take data and use them to build the analytics for the customers. What capabilities GoodData platform has to get the data in? I do not have one data source, but we have several applications running on our servers and in public cloud. Thank you. Tom

2
James Roettger
2 years ago
T
Tom HarrisNew Participant
asked in Data sources

LCM and ADD

Hello.I am trying to understand the concepts of Automated Data Distribution (ADD) and Life Cycle Management (LCM) and I am not quite sure if I understands them well. Could you elaborate on the features and how they cooperate, if at all?Thank you,Tom

2
T
2 years ago
J
John DoeNew Participant
asked in Data sources

Snowflake Data Load Erro "Mapping Validation Failed, See Log for More Details"

I’ve created a simple data table in Snowflake to test a single column import of data after successfully connecting the Data Model but still getting an error:2021-02-26T22:55:22.219+0100 [INFO]: Data distribution worker started 2021-02-26T22:55:22.219+0100 [INFO]: Request id: "data_load_1h4PkxFpZ8_RUH4bf9GoD:Q3tKkCkN9cVTUIzM:n31986nQVtgWoWAZ" 2021-02-26T22:55:22.219+0100 [INFO]: Data source: "603911cc1a4f3c4043bcd9f7" 2021-02-26T22:55:22.219+0100 [INFO]: Additional parameters: {GDC_DATALOAD_DATASETS=[{"dataset":"dataset.call_center","uploadMode":"FULL"}], PROCESS_ID=e7e79a92-fbb4-4101-81f2-e7a42ee9bad9} 2021-02-26T22:55:22.219+0100 [INFO]: Synchronization mode: selected datasets (default) 2021-02-26T22:55:22.433+0100 [INFO]: Synchronized datasets: [dataset.call_center] 2021-02-26T22:55:24.735+0100 [ERROR]: Mapping validation failed. 2021-02-26T22:55:24.736+0100 [INFO]: ====================== Mapping Validation ====================== The dataset dataset.call_center does not exist =======

1
Moises Morales
GoodData Employee
2 years ago
J
John DoeNew Participant
asked in Data sources

No Facts Or Dates for CSV Import

I’ve imported a CSV file with attributes, dates and numbers but GoodData is only recognizing attributes and not giving me the option to select some of those columns as numbers.I’ve tried it with various CSV imports but all of them continue to product the same results. Anyone experience that and found a fix?

3
Moises Morales
GoodData Employee
2 years ago
T
tramtbtranNew Participant
asked in Data models

Why do some items disappear from Data Catalog?

One DIM has multiple relationships with FactsIn my model, there is a DIM table has a primary key that is WORKORDER_KEY, and it has relationships with 3 FACTs by WORKORDER_KEY.In Analyze tab I added a attribute of one fact then the others of remaining facts were hidden. Can you help me figure out what am I wrong here?

1
Petr Olmer
GoodData Employee
2 years ago
R
Rick FlaggNew Participant
asked in Data sources

Best Practice For Importing Date FIelds

I’ve been working in GoodData setting up multiple insights and a KPI dashboard. I have a question about handling dates that I’ve not found an answer to.My data has an order completion date (ship date) that is either in the form MM/DD/YYYY or MMM YYYY (eg. FEB 2021) or YYYY/MM/DD. I have tried all three formats when importing the data. When I import the data, I have the attribute marked as text. I wish to produce a line chart based on month and year of the shipments showing the total value of the orders shipped. When I setup the insight, I get the dates in alphabetic sequence (Apr 18, Apr 19, Apr 20, Aug 18, Aug, 19, Aug 20 etc.). What do I need to do to get GoodData to see these values as dates and report by month?

2
Iva Gasparekova
GoodData Employee
2 years ago
S
sethjbrParticipating Frequently
asked in Data sources

New Data button vs. Load Button

In the January 15th dashboard release, the “Load” button for the Good Data dashboard was changed to “Data” and now there isn’t any way to see my previous uploads from CSV to my dashboard and when I try using the new “Load” function, I get error messages that the data already exists and can’t be updated or replaced. It looks like I need to rebuild everything, including all my insights, from scratch to make this work again. Am I the only one here that was uploading from CSV files to the dashboard and having insights attached to those CSV uploads in the past but now have issues with uploading after the January 15th software release? Thanks,Seth

14
Andrey
GoodData Employee
2 years ago
D
DonnieNew Participant
asked in Data sources

Debugging Big Query Integration

I’m trying to link Good Data with Google Big Query. I can connect to Big Query and create the LDM and publish, but when I try to load the data  I’m seeing the following errors in the log during the “download and integrate” step:Project "xxx123" was not integrated Data distribution worker failed. Reason: All projects failed to load.I’m pretty sure I’m missing something silly.

2
D
2 years ago
S
SimonbeeNew Participant
posted in Data sources

Loading Apache Spark CSV data from S3 with ADD

I’m just getting started with Gooddata and have successfully loaded CSV files from an S3 bucket with ADD, after a bit of fiddling to get my filenames matching my model name etc. (This article was very helpful, thanks!). I can see the data populating in my model and view it in a dashboard - so far, so good...However the data I really want to load is being written by Apache Spark (on AWS EMR) as a CSV file that is written in “append” mode, so the structure of the file in S3 looks like this:[aws-bucket]   |- [analytivcs-data]             |- hourly-data-dump.csv                         |- part-00000-00231726-b34a-454f-b534-bd9e492ab361-c000.csv                         |- part-00000-00a86ed4-a0b1-4c45-909c-49753c6ce877-c000.csv                         |- part-00000-00c9ebc7-550e-4632-b909-1aa88cd86b8e-c000.csv                         … There are potentially hundreds of the part-00000-xxx files written by Spark. I control the name of the “folder” hourly-data-dump.csv but not the files within

2
S
2 years ago
J
JamesParticipating Frequently
asked in Data sources

Publishing Data Model: CSV Data Validation Failed

Hi I’ve imported a CSV, and the model looks right, but then I click on Publish I get a message saying Data Validation Failed. Is there anywhere that I can get more detailed messages for the failure? When I used the Load screen to load up the CSV I did get reasons as to why the publish would have failed, but it seems disjointed. I had hoped to deal with all my data work in the Data Integration Console, and not need to hop around so much ThanksJames ​​​​​​​

3
Joseph Heun
GoodData Employee
2 years ago
J
JamesParticipating Frequently
asked in Data sources

Are Geo Charts Available in the Free / Growth Version?

I have added geo data using the latitude;longitude format, but I don’t seem to be able to use it for a geo chart. Is this feature available in the free version? In the modeller I am only able to set the field as Text.   

2
J
2 years ago
A
adhira93New Participant
asked in Data sources

IP white listing and finding what my data center is

I want to let GoodData pull data from our database but I don’t want to open the port to the entire world. I found this page about IP white listing but I’m not sure how to find out which set of IP addresses applies to me.According to the data centers documentation page, I should be using host on Mac to figure out “data colocation”. I’m using windows but I was able to find out that my GoodData hostname points to secure.gooddata.com (not secure-na3.na.gooddata.com or secure.eu.gooddata.com). Can you please confirm which data center it is?

1
Moises Morales
GoodData Employee
2 years ago
J
JamesParticipating Frequently
asked in Data sources

Are there any plans to add integration to Amazon Timestream?

It’s only just been given general availability this week, but Amazon Timestream would be a great data source to be able to work with in GoodData. https://www.businesswire.com/news/home/20200930006011/en/AWS-Announces-General-Availability-of-Amazon-Timestream Is it likely that you’d add support in the Growth Plan for this?  

3
Pavel
2 years ago
M
Mason PriceNew Participant
asked in Data sources

JSON-lines files

Along with CSVs, we’re using JSON-lines files. Will structured data files like that be available in the future for the upload and modeling interface?

1
Iva Gasparekova
GoodData Employee
2 years ago
M
Michelle
asked in Data models

modeling with cloud DWH

Can I use the drag and drop modeling experience with any other DWH besides Snowflake?

1
Michael Ullock
GoodData Employee
2 years ago
Welcome to the GoodData Community!Cookie settings