Hey all, I have a strange scenario that I'm exper...
# gooddata-cloud
s
Hey all, I have a strange scenario that I'm experiencing in a new insight I'm trying to build. I'm purposefully using the name of a value to try and join records together instead of some other unique identifier. However when I do this, in my filter and my table I'm seeing multiple values all with the same result. Seems like for some reason what should be joining across the string is giving duplicate results.
k
What you’re experiencing is a common scenario in GoodData (and in data modeling generally) when you join or filter on a value that is not unique—such as a name or other non-unique string. If the attribute you use to join or filter has duplicate values, GoodData will treat each occurrence as a separate entity, which leads to duplicated rows or results in your insight. Key points from the knowledge sources: - If you use a non-unique value (like a name) as a join key or filter, you will see multiple rows for each occurrence of that value, because GoodData cannot distinguish between them. This is expected behavior, as the platform relies on unique identifiers (primary keys) to join and aggregate data correctly. Using a non-unique value for joins or filters will result in duplicated results in your reports and filters, as you observed [GoodData UI Attribute Filter Docs](https://www.gooddata.com/docs/gooddata-ui/latest/references/filters/attribute_filter/#using-attributes-with-duplicate-values). - The recommended approach is to always use a unique identifier (such as an ID column) for joins and filters. You can then display a more readable label (like a name) in the UI, but the underlying logic will use the unique ID to ensure correct results. For example, you can define the filter using the unique ID and set the display label to the name, so users see the readable value but the logic remains correct [GoodData UI Attribute Filter Docs](https://www.gooddata.com/docs/gooddata-ui/latest/references/filters/attribute_filter/#using-attributes-with-duplicate-values). - If you must allow duplicate display values in filters, you can use the
enableDuplicatedLabelValuesInAttributeFilter
prop in the GoodData UI SDK, but you should still define the filter using the unique primary label to avoid ambiguity and duplication in results [GoodData UI Attribute Filter Docs](https://www.gooddata.com/docs/gooddata-ui/latest/references/filters/attribute_filter/#using-attributes-with-duplicate-values). Summary: To avoid duplicate results, always join and filter on unique identifiers, not on non-unique names or values. You can display the name as a label, but the logic should use the unique ID. If you use a non-unique value, duplicates are expected and unavoidable due to the way GoodData (and most analytics platforms) handle joins and aggregations. If you need further help with your specific data model, consider sharing more details or a screenshot of your LDM for more targeted advice.
s
But I don't want that, I want to take advantage of the string matching.
i
Hello Steve, Could you please share the link to the insight you are trying to create please?
I swear we have used string matching like this a lot in our analysis design, if this no longer works we've got some issues on our end.
Hmm, I think I see why its happening when I download the debug package, there is a left join on our dim_project_task by task_order. Therefore when we bring in say 3 records of the same name but different sort oder -> why we have duplicates:
but i have no idea why its pulling in that task_order
i
Hi Steve, thank you for the links. What you are describing is the expected behaviour as when we use non unique identifiers like "Mowing", we cannot reliably differentiate them as they are not unique. The recommended approach is to always use a unique identifier (such as an ID column) for joins and filters. You can then display a more readable label (like a name) in the UI, but the underlying logic will use the unique ID to ensure correct results. For example, you can define the filter using the unique ID and set the display label to the name, so users see the readable value but the logic remains correct https://www.gooddata.com/docs/gooddata-ui/latest/references/filters/attribute_filter/#using-attributes-with-duplicate-values
s
I don't understand. this has to be new behavior.
Why would there be a new field in the results?
Would this be why?
i
I understand. I will check this internally and will update you here. Thank you
s
I think this sort by is why its causing my duplicates. I don't disagree your suggestion is more correct, however this something we've taken advantage of. I'm going to test it to verify by cloning the attribute -> and if it "solves" my issue I suggest an enhancement in the LDM or the editing of an insight to indicate the sort by is being used.
@Ismail Karafakioglu https://analytics.golmn.com/analyze/#/lmn_single_authoring/9f728e7c-20c5-477f-92e6-f59aaac20e54/edit This gives me the desired results -> added a new atttribute called Task Name Clean and then when I use this one (without hte sort by on the LDM) I get 1 record by name like I would expect.
I don't think I understand why using the unique attribute would give me the desired results as suggested. Example: Data: [{ Type: Mowing, Value: 1, }, { Type: Mowing, Value: 100 }, { Type: Mulch, Value: 10 } ] If I want to do a group by across type: I don't want 3 records I want 2 Mowing: SUM(101), Count(2) Mulch: SUM(10), COunt(1)
Yeah, I see this sortColumn in my LDM however I don't see any indication this is there other than clicking on the LDM while editing it..... 100% suggest bringing this to the UI somewhere would be critical to avoid this type of confusion.
Copy code
{
                        "id": "TASK_NAME",
                        "title": "Task Name",
                        "description": "Task Name",
                        "labels": [],
                        "tags": [
                            "Task"
                        ],
                        "sourceColumn": "TASK_NAME",
                        "sourceColumnDataType": "STRING",
                        "sortDirection": "ASC",
                        "sortColumn": "TASK_ORDER"
                    },
i
Thank you for your valuable description. I will consult my colleagues and will give you an update on this. Thank you for your patience.
s
I'll check the article you sent my way to see what I could do to more uniquely reference from unique ids, but would surely expect at some level we have to be able to aggregate to join records together.
p
🎉 New note created.
j
Hi Steve, fyi I have just submitted a product feedback requesting this feature to be visible in the Analyze/Insight UI (e.g., tooltip or label note) when a field is sorted by another column so users immediately understand why duplicates may appear.