Hi team, we are having a bit of an urgent issue with userDataFilters, we had implemented this some time ago and it was working as expected, but suddenly for our first customer using this we started seeing this error 500 when a user data filter is created for a certain user, if we remove the filter then the data loads correctly, but also the filter is created in an accepted format (the endpoint fails if not). Could someone please help us with this issue?

Hi Jhonatan, Thanks for reaching out and sharing the image showing the error message with the traceId. While I investigate this, allow me to ask a few questions. You mentioned that this was working just fine before, may I ask when this issue started ,and if anything was changed that could have potentially caused this. Also, does this affect all users or specific ones, and If you can share a direct link to the report to allow us impersonate and trouble shoot further. Feel free to share via dm.

Hi @Jhonatan Teixeira could you please provide more context about the issue? I understand you mentioned that visualizations were displaying with the particular userDataFilter definition. • Could you please tell me when the issue started? • Did you make any changes to the LDM prior the issue? • Is the problem displaying data only limited to some specific visualizations / dashboards you shared with my colleague or all visualizations are affected? • Does the problem occurs only with the mentioned UDF or does it happen with other user data filer definitions?

Hi @Jan Kos • issue was first reported yesterday, but could have started earlier • NO recent changes to LDM • The visualisations shared were examples, it seems this is affecting all visualisations • We only have this UDF implemented in our org Hope this helps

Thanks for the details, @Raffaella Gozzini! We’re investigating this and will update you as soon as we have some progress.

Thanks for the input @Francisco Antunes. We'll check the UDF. Also, are users currently having issues when opening their visualizations? The user who is seeing errors is the one we've shared in the chat with your colleagues, together with a few example visualisations. They are seeing all the embedded visualisations in our app throwing an error.

@Francisco Antunes currently we see an error 500 happening, on your side do you have any indication of what error this is? Maybe this could help

@Jhonatan Teixeira, Yes, you’re right in the sense that we would expect to see no records instead of a 500. However, the 500 error is occurring when our backend attempts to generate the SQL query for your data source. It fails when trying to locate certain fields. Checking the LDM, I can see that there are connections between

survey_attempt_answers

and the rest of the model, but those are

M:N

(many-to-many), and in many cases indirect. UDFs do apply across the LDM, but they do so following the direction of the arrows connecting it to other datasets (meaning: if a dataset is on the “right” of the model, the UDF will not propagate to those on the left of it). The

M:N

connections complicate things, though, to the point where it’s not reliable whether the user IDs from the

survey_attempt_answers

will actually limit the users from the

Users

dataset - and if the LDM or connections are complex enough (which they seem to be), this can break visualizations that rely on

M:N

connections. E.g.:

survey_attempt_answers

connects to

Users

via

Survey attempts

, through 2

M:N

connections. Meaning: Multiple users can take surveys; Surveys can be taken by multiple users; answers can come from multiple surveys, and each survey has multiple answers. If you try to limit the users from those IDs present at the survey answers, it would not limit any users who didn’t make any survey attempts, for example. If that is the case, the query will not be able to connect to any other part of the model that requires

Users

as a connection point - and that can lead to a 500. In one example visualization,

BLG Objectives report

, I found that by removing

Team Name

from the filters, it stops the 500 from occurring. Then we get the “No Data for your Filter Selection” we expect. This suggests that this particular connection point is confusing our backend to the point of throwing the 500. Okay, that was quite a lot of words! 🙂 I hope it helps explain what’s going on, though. Here’s my proposed fix: Modify the UDF so that it applies to the

Users

dataset, which from what I can see is central to the LDM and from where a lot of those connections and interactions stem. I believe that will resolve (or at least greatly improve) the issue. Does that make sense?

Thanks @Francisco Antunes. I have changed the UDF MAQL to {label/users.id}, this has so far stopped for error 500, and we are now analysing to see if the data looks sane, but it's a good progress so thanks a lot for that!

That’s great news! Let me know if there’s any further issues

Hi Francisco, Can I please ask you some follow up questions?

UDFs do apply across the LDM, but they do so following the direction of the arrows connecting it to other datasets (meaning: if a dataset is on the “right” of the model, the UDF will not propagate to those on the left of it).

What does that mean in practice in our case? In summary, we use these user filters so that our users (team managers) can only see records related to their team members (the array of user IDs we are using for the filters). When you say "the UDF" wouldn't propagate to other data sets, do you mean there is a possibility of exposing data pertaining to other users? And for data tables that don't contain a user ID, should these users with UDF applied see no data or all data? Thanks in advance for the clarification

Hi @Raffaella Gozzini, sure thing! I’m happy to help. I think in this case it’s important to keep the explanation from our Documentation in mind:

Simply put, the UDF will be applied to all dataset attributes that are in the LDM along the direction of the arrows. The filter is not applied against the direction of the arrows in the LDM.

So if a dataset doesn’t contain the ID, it will not be filtered at all. Here’s an example from my environment: 1. Order Lines and Product are linked, like shown in screenshot 1. 2. I created a simple UDF that limits my user’s visibility based on Order line id, to a single record (screenshot 2). 3. If I check an attribute from the Products dataset, it will show me everything (see #3). 4. However, once I bring Order line id into the mix, it limits the data to only those brands that match the ID I have available for my user. What I’m hoping to illustrate here is that while the filtering will not apply to datasets that do not contain the ID you’re using, you can easily structure either your visualizations or your LDM so that either the unfiltered data is always contextualized by the filtered one, or that it won’t matter. I.e.: knowing what brands are available in the data (in my example LDM) is not at all sensitive, and doesn’t impact the limitation on the order lines. In the end, it is up to your LDM - if there are other pieces of information that are sensitive and not at all related to Users, then the solution would be to also add them to a UDF. If only the info related to Users matters, than the UDF applied to the main User dataset will definitely do the trick.

Thank you for the clarification, much appreciated!