To GoodData homepage

Data models

Data models

  • 34 Topics
  • 103 Replies

34 Topics

hanka
hankaKnown Participant
asked in Data models

Dates and time zones

Hello!I was reading docs regarding custom time zones as well as custom time dimensions, and I’m still not sure how to deal best with users different time zones and proper display of dates.Most of our users are in Pacific Standard Time zone, however, our data are stored in UTC format. Because by default there’s no support for timestamps, we’re sending only dates to GoodData. But this causes some dates are displayed differently in GoodData and in the main application. For example Apr 21, 2023 - 6:13pm Pacific time displayed in the app is already 22/04/2023 in GoodData (as it’s UTC).One solution I see is to normalize dates from UTC to timezone that should be used in a given workspace for data upload. That would work well if all workspace users were in the same time zone.Would there be any good solution that could work for a workspace that can be used by users in different time zones? So they could see base date displayed in their desired zone? I don’t see how custom time dimension could h

5
Moises Morales
GoodData Employee
3 months ago
D
davidhParticipating Frequently
asked in Data models

Display view

Hi,i am problem with display view from my DB.I configured data source and use non public schema (custom schema name)After connect over data connector to my DB (in Model) - returns back only tables no views. The user has set permission to select from views.If I try the same situation with just the public schema, I see tables and views.Isn't the problem that I have the view in a different schema than the public? Thank you for your answersBest regardsDavid Haškovec

12
Jan Kos
GoodData Employee
5 months ago
hanka
hankaKnown Participant
asked in Data models

CSV data upload and big datasets

Hi,My workspaces are populated by ADD bricks that load CSV files stored on S3. As there’s a limit on headers amount in CSV files, which is 250, and not only dataset fields count but also relations to date and other datasets, or additional labels, how can I handle the situation where I’d need more fields? For example if I need a dataset “projects” that already has 200 fact fields, 40 relations to custom date fields, and I’d need to add additional 20 custom fact fields, is there some way to either use more than 1 CSV file to upload data for this dataset or split it to have 2 fact datasets, but use them as if they were one in the reports? Best,Hanna

2
hanka
5 months ago
hanka
hankaKnown Participant
asked in Data models

LDM changelog/revisions

Hi!As our LDM is evolving, sometimes, especially in case any errors occur, it would be handy to see historical changes that were made in the model. Is it already available in any way? If not, are there any options that could help automating the process of building some kind of a changelog, like triggering some callback on release brick for example? Best,Hanna

1
Joseph Heun
GoodData Employee
5 months ago
hanka
hankaKnown Participant
asked in Data models

Custom date fields and performance

Hello,I want to give our customers a possibility to use custom date fields and as I know that date dimensions consume disk space and if there are too many of them it might have an impact on performance, I’d like to understand better the breadth of possible impact.In our product we have few segments that define basic LDM variations and inside them customer workspaces. Each customer can freely choose how many of available custom date fields they will use, not used will be hidden (marked as deprecated).Considering setup with segments A, B and C, each with a different amount of available custom date fields (let’s say 50, 100 and 150), and customer workspaces where in segment C there is client c1 using many of them and also loading bigger amounts of data, where performance issues can be visible? Will it be on c1 customer workspace only, or maybe on the whole segment, or actually on all segments, for the whole product? Best,Hanna

2
hanka
6 months ago
mares
maresParticipating Frequently
asked in Data models

Model query

Hi folks,I'have been reading few posts from the community in order to structure my model. Although have a few questions here.This is my current model Overviewpoc_date_link: contains all FK's from the different tables at date granularity.In this example, a subscription has a start date and end date. Generated all days (date field) for the user subscription, as a user can not perform any action without having a subscription, (as an annotation, among other tables I did not put in the model cause it would be a bit big). Additionally, for A "user", "subscription", "date" created a unique id. Then I disseminated the ID in the other fact tables.This allows me then to associate different table models by that particular id, which then grant me access to all fields. Done it this way as my Model has lots of dates (as I think most models) allowing me to play all metrics around a unique date.Enough introduction, as the rest of the model is kind of the same.My question is regarding dimensions, facts

2
mares
7 months ago
JeffC
JeffCParticipating Frequently
asked in Data models

Updating Data Models when Souece Updated

So I have a postgress database that has been updated to add new columns. However, my data model doesn't seem to pick these up or volunteer that they were found so that I can choose to add them for future loads. Without breaking metrics how can I add the missing columns in the least manual way?  

3
Iva Gasparekova
GoodData Employee
8 months ago
M
MashaKnown Participant
asked in Data models

Moving attributes between datasets

Hello,I have a question about possibility of moving attributes between datasets.Let's say i want to move an attribute Account name from Subscription fact table to Performance dimension table at the LDM level. It seems possible operationally (using a “move” button) but I cannot see how can I map it properly when it lands in the Performance dimension table. Seems like I cannot tell that the data source for this specific column should be another source table (Subscription fact table in this case). If so, what is the purpose behind  a “move” button?My general context is to test how can/can’t I modify my LDM in case any Insights/Dashboards already exist on top of the LDM. And is there any recommended scenario for the approach when I want to populate my workspace with data incrementally?Thank you,  

10
M
11 months ago
P
PD5Known Participant
asked in Data models

Error when updating data model

Hi, I have a number of similar workspaces that share the same data structure. Recently I have added some fields to the tables in my data warehouse and have changed my data models on Good Data to reflect these changes. This has been fine on 3 workspaces, but on the 4th when I try to save the changes, I receive the following error and the changes are not saved: [{"errorClass":"GDC::Exception::Base","requestId":"modeler_4WjdYDoOXy_DWDqXo0YWP:dOzuVDGsk4yGDtRA:110lnxnig7moh5gy","parameters":["handle_exception","hide_internal"],"component":"Stock::MD","message":"Internal error [%s, %s]."}] Is there any way to fix this? Also, related to this, when saving data model changes - even though the only changes are to add new fields - some calculated metrics and insights are destroyed when saving the changes. It’s nice that there is a warning, but is there any way round this? It would be understandable if I was changing fields that were already in use, but in this case I’m only adding new fields.   

2
P
1 year ago
N
nikoParticipating Frequently
asked in Data models

Using "event" dataset

Hello,I found an interesting article on your help site regarding analysis using dates: https://help.gooddata.com/doc/enterprise/en/data-integration/data-modeling-in-gooddata/if-too-many-date-dimensions-try-eventsI think it will suite my needs as I need to be able to slice data using dates that have different meaning. The scenario described in the link only presents a case where the event is related to single dataset, in my case this is quite different as the event has to be connected with two datasets (which in the ends happens to create so called “alternative paths”). I wanna make sure the model above will be valid or maybe there’s a different approach to the problem that won’t result in alternative paths? Example report I would like to build using those datasets:Report showing # of projects were in a given phase month by monthDo you think the approach above is OK? I’m pretty sure building right model is really important before I can proceed with data transformation on my side, so I w

9
N
1 year ago
Kishan
KishanNew Participant
asked in Data models

Is there any way we can use N:M relationship using source column

I go through the documentation but didn’t find a way to establish many to many relationship directly using source column. source:https://help.gooddata.com/doc/enterprise/en/data-integration/data-preparation-and-distribution/data-preparation-and-distribution-pipeline/data-pipeline-reference/automated-data-distribution-reference/naming-convention-for-output-stage-objects 

1
Moises Morales
GoodData Employee
1 year ago
d.novikov
d.novikovParticipating Frequently
asked in Data models

Trying to load a big database into a model leads to browser tab crash

We're trying to load a big database into a model. I can create PDM and LDM through API calls, but when I press the "Create Model" button in the UI a browser tab crashes.Are there any limitations on the database size?

4
R
GoodData Employee
1 year ago
N
nechaldevNew Participant
asked in Data models

Separation of field names

Hello!In our databases, there are fields that are named the same, so when the manager sees the fields in the gooddata in the analytical model, they do not differ in any way, this leads to the fact that you need to spend time determining which particular table this field belongs to. Is it possible to somehow specify a specific field name in a file or through another mechanism so that the fields are named more clearly? For example, according to the template "{Table name} {Field name}".Thanks for understanding. 

1
Joseph Heun
GoodData Employee
1 year ago
P
PD5Known Participant
asked in Data models

Receiving an internal error when publishing data model

I have just cloned a workspace (workspace A) + its data to a new workspace (workspace B). I then created a new PostgreSQL data source which I want to map to the fields in workspace B. The new database is 90% the same as the old one with just a few minor changes.If I just switch the data source on Workspace B, it loses all field mappings and I have to set them all again by hand.To work around this, I have exported the data model from workspace A as a Json file which I have then imported into Workspace B after connecting the new data source. This seems to work and all fields are now mapped as they should be.The differences between the old and new data sources mean I must add a new column in two tables and drop 3 columns in another couple of tables. This seems to go well when editing the data model.However, when I click the publish button I see the following error and the model will not save:[{"message":"Internal error [%s, %s].","requestId":"modeler_EMwODbrZNn_IYHR2xoIsp:Kkpz9mqkiecUeTtG

7
P
1 year ago
O
GoodData Employee
Oldriska ZikovaGoodData Employee
asked in Data models

Comparison of data in GD.CN

We are facing an issue with GD.CN. Our clients need to be able to compare the results of individual generations of data with each other. In order to do that you have the FOR PREVIOUS function is used to create metrics. However, the problem occurs when the data is generated irregularly (for example, once every four days, once every two days, or vice versa five times a day - the spacing is never given). The function cannot take this into account, and in addition the second input argument of the function cannot even work with facts (this also includes the possibility to send information about the spacing between generations from the database to the dataset).Unfortunately, we have not been able to find any other function or way to prove this - how to compare the data of the last generation with its previous generation if the spacing between the generations is irregular?The second part of the problem is the question of whether it is possible to work with smaller time units in GD and display

1
Moises Morales
GoodData Employee
1 year ago
C
Chris MacomberParticipating Frequently
asked in Data models

Error when loading data into column that's not in the data model

Hello,I have a column in one of my tables that I've removed from the data model, but when I try to load the data, it is throwing an error because it can't load data into that column (even though I removed it) because the values are too large for the type. Why is it trying to load data into a column I don't want GoodData to worry about?  Does the Load process load all data regardless of what’s been selected in the Model? Chris

8
C
1 year ago
T
Thomas NordNew Participant
asked in Data models

Alias tables in LDM

I want to bring in a certain table several times with different names to the LDM, is that at all possibile in Gooddata? Essentially the same as creating views on a table on the database side. I have a table that holds descriptive labels for some dimensions, for some reason they are all in one table. So I need to bring it in once as “Age Dimension” (with filter on a certain value and a join to a age_id in the fact table) and then once again as “Gender Dimension” (with filter on another value and join on gender_id in the fact table). Etc.   

3
Boris
GoodData Employee
1 year ago
N
nikoParticipating Frequently
asked in Data models

Analyzing "multi-select" values

Hello,I think I need a guidance on the data modeling issue.In our app we have multi-select fields that we want to analyze in GoodData. From the user perspective it's possible to fill out some forms with fields of different types, e.g. text fields/numeric fields and select fields. The select fields are basically single select and multi select ones. All the fields are custom (means they are a separate entity in the database and can be created/deleted/modified, they are NOT a static attribute).From the database perspective we have a model called “Scoop” which can have many “Fields”, and those “Fields” can have multiple “Choices” if the field type is multi-select. And now we want to analyze such fields and their values. Example:Field: Panel TypesField Choices: Panel 1, Panel 2Report: Show % of scoops with Panel1 and Panel2 vs. those with just Panel1. (pie chart)Is that possible to achieve?If you need any more details just let me know.Thanks in advance!

3
Boris
GoodData Employee
1 year ago
P
PD5Known Participant
asked in Data models

Best method for updating a table in a data model

HiAfter initially creating a data model with various tables I sometimes need to update a table as more columns have been added to the external (PostgreSQL) data warehouse that I am using.I have noticed that if I go to the Data page and edit the model, I can drag the column from the listing on the left of the page to the main data area and set the column types as usual. However, after clicking the import button, the changes do not appear and the table remains as it was previously.It is also possible to delete the table entirely from the data model and then drag it into the data area and set the column types and references again. After clicking the import button this does replace the old table, but then saving the model results in all custom metrics and insights being wiped even if all tables and columns used are still present in the updated table(s).My question is, is it possible to update a table in a data model in such a way that any metrics or insights are kept? 

1
Joseph Heun
GoodData Employee
1 year ago
P
PD5Known Participant
asked in Data models

How do you create many to many relationships in your data models?

I have read the documentation and university courses on data modelling and the text often mentions many-to-many relationships such as product tags. This is displayed in the data model as having a bi-directional arrow and will allow filtering of products by tag on data dashboards.  However, when I import database tables from my database, the model only ever shows the arrows going from products to tags only, which means I can never use the tags table to filter my products. My pivot table has the following columns:id (the id of the pivot item)product_id (foreign key that links to id on the products table)tag_id (foreign key that links to id on the tags table) Each product can have multiple tags and each tag can have multiple products, but this is not reflected in the data model diagram. What am I doing wrong? 

2
P
1 year ago
Y
yossisun23New Participant
asked in Data models

importing data model from active record ORM

We are looking to avoid manually creating the data model by potentially converting our activrecord schema into a model Json file - had anyone did that before? that will be amazing!

1
D
GoodData Employee
2 years ago
A
ashokKnown Participant
asked in Data models

Ubable to import models form GDCloud to GDCN

Hi Team,We are unable to imapoer the model from GDCloud to GDCN  Screen hangsup ..

1
R
GoodData Employee
2 years ago
R
rohit mathurNew Participant
asked in Data models

Custom Sorting Order Does not Work

Has anyone tried implementing custom sorting order on an attribute using the LDM Attribute Label ? It does not seem to work for me so wanted to see if there are specific tricks / process around how best to use it. We have a text field whose sorting should not be alphabetical but follow a specific order

3
Iva Gasparekova
GoodData Employee
2 years ago
R
Ron BorensteinNew Participant
asked in Data models

CSV missing values

Hi, I am new to GoodData and trying to get started, but having an issue I can’t get around.I am uploading a csv, created using a jupyter notebook. The file is 15.5mb and has 56k rows. Everything seems fine when I upload it, but when I try to create my first insight, the data is different than in the file.I have a measure called `timeSpent` . When I sum `timeSpent`  the number in GoodData is significantly lower than it is in the file.sum of timeSpent in GoodData - 1,849.10 sum of timeSpent in the file - 19,670,743.42How can I check what’s wrong? 

2
R
2 years ago
R
Rick FlaggNew Participant
asked in Data models

Automating Data Load from Amason S3 Bucket

I am receiving a Connection Error trying to connect to my S3 bucket. I am the admin user on both the GOodData and Amason systems. My connection configuration is shown below:I am uncertain what is expected in the Access key and Secret key fields.

1
Iva Gasparekova
GoodData Employee
2 years ago
Welcome to the GoodData Community!Cookie settings