Hi GD team, I have another question regarding Flex...
# gooddata-platform
a
Hi GD team, I have another question regarding FlexConnect as a data source. If I • join 2 tables (
A
and
B
) from a FlexConnect data source via
A_id
and
B_id
respectively • chose a table `B`'s field (field
X
) for filtering ◦ selected several values, • and then placed a field (Field
Y
) from Table
A
on
Rows
, • the request then comes to my FlexConnect service to retrieve field
Y
but with the filters applied. The request hits the FlexConnect function implemented for Table
A
(yes reasonable), but the filter's context doesnt take into account the linkage that I've done on LDM. It'd send me the filter field
X
above that comes from Table
B
, which of course wouldnt exist in Table A's context. Im wondering if this is expected? And then that means, I should also maintain a linkage in my own BE code used for the implementation of the FlexConnect data source? If so, I think it can lead to drifts (i.e. users changing linkages on GD, but BE does not have this information)
y
Hi Alson, Thanks for reaching out to us, happy to help. Did you use a 'By' clause to connect the datasets? The BY keyword is used to set a minimum level of granularity by which a metric can be broken down. BY effectively sets an aggregation floor, overriding report attributes that would serve to break the metric down into smaller units of granularity than is specified in the BY clause. https://help.gooddata.com/doc/enterprise/en/dashboards-and-insights/maql-analytical-q[…]y-keywords/overriding-insight-attributes-with-by-keyword/by/ You can also have a look at our older documentation which explains explicit lifting https://help.gooddata.com/doc/enterprise/en/dashboards-and-insights/maql-analytical-query-language/maql-use-cases-and-tutorials/explicit-lifting/ If this doesn’t help, kindly share more information with us about this, to help us troubleshoot further. Feel free to share screenshots of the LDM and a link to the workspace via dm.
a
Hi Yvonne, thanks for your reply. However, I dont think it's related to these. Let me do further investigation and I'll get back to you on this in a more detailed manner
j
The FlexConnect table function receives execution context of the visualization. This context contains information about applied dashboard and visualization filters. This information can be used by the table function to push down relevant filters to narrow down returned records of the FlexConnect dataset. SQL query is executed on top of the datasets and returned data from multiple FlexConnect calls for multiple datasets are joined and dashboard and visualization filters are applied as WHERE or HAVING conditions in the SQL query. In your scenario it depends if Field X exists as foreign key in the table A. In such case data of table A can be used for the query and retrieval of data for table B may be not necessary. You can implement in table function pushing down of relevant filter predicates to reduce number of transferred data but if you do not implement it, the data will be filtered by SQL query executed on top of provided data by the table functions.
Note that execution context does not provide information about all applied filters in metric definitions but it contains information about used metrics in the visualization.
a
Thanks Jakub, yes indeed, currently Im hindered by this statement
In your scenario it depends if Field X exists as foreign key in the table A. In such case data of table A can be used for the query and retrieval of data for table B may be not necessary.
My Table A is like a Fact table, so it has a large number of rows. Due to another conversation that we're having, currently that whole table is being returned. So Im hoping to use Table B, a dimension table, to perform filtering, and then query data off Table A, so that we will return less data. With these 2 joined in LDM (sry let's use the terms
A_key
and
B_key
for easier reference), I was hoping that the execution context when sent to
Table A
, would then mention
A_key
and the values it should filter for, but Im receiving
B_key
, which naturally does not exist in
Table A
j
This is beyond FlexConnect capabilities. There is no dependency between individual requests. You would need to interpret the filters of execution context, perform joining in the implemented code of flexconnect function to apply the filter on fact table if column used for filtering exists only in the dimension dataset otherwise the filter will be performed only on top of the results obtained from all the function calls.
a
I better understood what you meant after looking into the logs deeper. I noticed 2 parallel requests being sent - 1 to Fact table, 1 to Dimension table, so Im guessing this is the furthest that will be done in terms of linkage on GD in LDM. • Case A: If the filter is applied on dimension table, then dimension table will be filtered, and fact table wont be filtered. If fact table size is small enough, I can kinda get away with it. But if it isnt, then we run into the issue we spoke in the other thread (i.e. size is too large 😞) • Case B: For the other way round - if filter is applied on fact table, then I send a filtered down fact table and a full dimension table
If you say that there's no dependency, Im afraid that I will have to perform pre-filtering in Case A when fact table is very large, as I will probably timeout before I can send data over to GD for its SQL query execution. Do you think that if this would be what you guys recommend: 1. Use filter to run against dimension table, get the corresponding PK (primary key) values remaining 2. Then when running query against fact table, apply a
$in
(we're using MongoDB, but it's equivalent to
IN
in SQL) for its corresponding FK based on the obtained Step (1) values above
j
Yes. You may need to pre filter data in fact table if the data volume is too big. Otherwise inner join between datasets during SQL query will filter out data from fact table if data is prefiltered in dimension table or the WHERE condition applied in the SQL query will apply the filter as the last resort option.
🙏 1