Hello, have a question on LDM source refresh : We have a set up where multiple LDMs are based on individual schemas within a Snowflake database. The database has more than 10 schemas but all LDMs are always based on tables from a single schema. We noticed that when the source is refreshed all tables in the 10 + schemas are scanned and this takes longer than if only one specific schema is scanned. Is there a setting where we can restrict good data to limit scanning only the schema used in the data connection ? Thanks !

Hi Sagar, thank you for this. To gain a better understanding, could you please share the HAR file from the action you performed when you clicked the refresh button? Additionally, could you provide the name or ID of a schema or table that you did not expect to be scanned?

Mauricio, thanks for your response. How can I create a ticket for this issue ? I am not able to cut-paste any information or attach a file on slack. Thanks.

Hi Sagar, this is Francisco, with the Technical Support Team. I don’t think the HAR file is required here, actually. I’m still checking from our end, but I can tell you that the behavior you noticed looks to be the default when scanning from the UI. When you click the “refresh” button on the LDM page, a POST to

api/v1/actions/dataSources/{dataSourceId}/scan

is done, and it checks all schemata. I don’t think which schemata it scans can be customized from the UI at this time (but I will confirm this with our devs). However if you make the same call directly via the APIs, instead, there is an option for specifying

schemata

which can be used to limit the scan to a list of schemata.

Hello Francisco, thanks for your detailed message above. If you can confirm the UI part ( customizing scans from the UI) that would be great.

Hi @Sagar Sonar I discussed it with our engineers, and they agree that the scan should actually scan as few schemas as possible. I’ve opened a ticket with them, and they will work on developing a fix for this, so that it works in the way you expected! But until then, there is indeed no way in the UI to limit the DB scans.

@Francisco Antunes, I believe we did set up the schema in the db connection but the refresh was scanning all the schema in the database, will confirm this.