We're getting a 400 error when calling gooddata fr...
# gooddata-cnj
James Lee
08/01/2023, 10:50 PMWe're getting a 400 error when calling gooddata from python sdk:
"message": "Reached the limit of max dimension size. Limit=10000, Size=11883",
Is there a way to get paginated results using python sdk?
i
Ivana Gasparekova
08/01/2023, 11:13 PMHi James,
The error doesn’t seem to be related to Python SDK itself, but to the breach of our platform limits.
I agree that pagination would be helpful for similar cases and I can assure you that it was already brought to attention of our Product team as a pretty useful feature to have.
Similar topic was recently discussed here.
j
Jan Soubusta
08/02/2023, 10:02 AMAFAIK 10,000 is the default limit in the community edition.
Am I right that you use the community edition?
Jan Soubusta
08/02/2023, 10:03 AMAnyway, you can bump the limit by setting the following variable in environment section:
Copy code
LIMIT_MAX_RESULT_XTAB_DIMENSION: 100000Jan Soubusta
08/02/2023, 10:08 AMAlso you may need to bump resource limits in Redis accordingly (caches are stored there):
https://www.gooddata.com/developers/cloud-native/doc/2.4/deploy-and-install/community-edition/advanced-config/#AdvancedConfigura[…]-ReconfigureRedis
j
James Lee
08/02/2023, 2:43 PMok thanks. And if i just want to check that any data exists for a set of attributes, is there a different way to do it instead of pulling all for the selected attributes?
j
Jan Soubusta
08/02/2023, 4:01 PMThere are the following options:
• in AD app in the left panel, each attribute has "?", mouse over shows example values
• Create Top(X) filter in AD. It is applied to SQL, so you don't hint the issue
• Create a SQL dataset in LDM Modeler, show data preview (first X rows are listed)
Not sure if at least one of these options works for you.
If not, let me know.
j
James Lee
08/02/2023, 5:43 PMthese won't work. is there a way to do this programmatically from the python-sdk client?
j
Jan Soubusta
08/02/2023, 5:53 PMAll rows are collected unless a TOP(X) filter is applied.
Jan Soubusta
08/02/2023, 5:59 PMHm, unfortunately the TOP(x) filter can be applied only on metrics. Preview attribute values use case is not compatible with that.
Jan Soubusta
08/02/2023, 6:03 PMUnfortunately, I can't see any solution for you.
We need to change the current report execution flow to allow returning only subset of rows to prevent hitting the limit.
It could be implemented with a new parameter of AFM execution, which would force our SQL generator/executor to fetch only subset of rows.
p
Productboard
08/02/2023, 6:05 PM🎉 New note created.
p
Phanindra
08/02/2023, 6:45 PM@Jan Soubusta I'm assuming that is a new product feature, is that something that can be done in a short order and made available in the python sdk ?
c
Cayce Collins
08/03/2023, 2:52 PMhttps://gooddataconnect.slack.com/archives/C01P3H2HTDL/p1690970533956249?thread_ts=1690930225.564929&cid=C01P3H2HTDL
No, we have the licensed full version