Hi everyone, can anybody help me with implementing...
# gooddata-platform
m
Hi everyone, can anybody help me with implementing LCM to a customer with no ADS but Snowflake directly connected to GD. Also with 2 environments DEV, PROD. I know how to setup LCM with ADS included - 1 Service workspace for all schedules, 1 Development workspace (I can have only 1 for both PROD and DEV), 1-X Master workspaces, X Customer's workspaces. The issue I am running into now is how to setup correctly LCM with no ADS but snowflake only. I would like to have only 1 development workspace (only on DEV) and create Master workspaces on PROD from this particular one on DEV. The best solution I came into is to have: • Service workspace - with LCM schedules • Development workspace - with Snowflake data load, all reports,insight, metrics etc. and LDM (because data loading requires LDM) • Client's + Master workspaces. But I don't see there how to have only 1 Development workspace for both environment, so the customer does not have to keep Development workspace updated on both ENV. Is there any best practice for dealing with this type of situation or do you have any advice? Thanks
j
Hi Martin, you can create a Development master workspace in the DEV environment with all the necessary objects you wish to have. Then set up a Service workspace with LCM bricks and schedules. You can set up Release Brick and point it to the Development master workspace in the DEV environment. When you run the brick it’ll create segment master workspace for each segment which will inherit all the objects from the Development master. That way you can only have one workspace (Development master) to maintain. Then you set up Provisioning Brick and Rollout Brick to provision new clients and synchronize changes from segment master workspaces to clients. You would need to specify access to the Snowflake instance in the bricks parameters where it’s needed. You can also check the following community article for more details.
m
Hi Martin, if you know how to setup LCM with ADS, setting it up without ADS and with Snowflake should not be much different. All the LCM bricks should support connection to Snowflake. The biggest difference would be that for loads from ADS you are using ADDv1 which is deployed to every single workspace, while for loads from Snowflake, you will use ADDv2 which has two modes: • load into a single workspace • load into a whole LCM segment In case of ADDv2 the best practice is this: • in the development workspace (in DEV) use the ADDv2 in “single workspace” mode • for the client workspaces, use the ADDv2 in the service workspace in the “whole segment” mode Otherwise the setup of LCM is the same - as Jan posted above - you can use the same DEV workspace for multiple LCM environments - you just point the Release brick to it (and its environment)
m
Thanks for the quick response@Jan Kos and @Michal Hauzírek. Thanks for the best practice with single workspace mode in DEV development workspace, I didn't know about that. So on PROD environment I won't have any development workspace (I will point the Release brick to the DEV development workspace) - that is clear. But what about ADDv2 load on PROD environment - customer have PROD data sources on PROD environment. That means this ADDv2 brick on PROD environment will be setup in "LCM whole segment" mode. The red flag I still see are LDMs - It looks like to me there will be 3 "unique" LDMs and I don't know how It should be synchronized/maintained. We have: • LDM in DEV development workspace • Service project on DEV (Where DEV ADDv2 will be) • Service project on PROD (Where PROD ADDv2 will be) Is this setup correct? Maybe there won't be no issues with this, but I want to be sure. Thanks a lot.
m
the “service project” (in any environment) typically does not have any LDM it is really just a “service” / technical envelope to host the various bricks (which need to exist in context of a workspace). But the LDM itself is only in the DEV project and the client projects (and in the LCM master projects which are automatically created by the bricks from the DEV project and has the same LDM)
So you do not need to synchronize LDM to any of the service projects. The service project typically only contains the (data) pipeline (=the bricks + ADDv2). To synchronize this configuration between environments you can use the “Import & Export Pipeline feature” https://help.gooddata.com/pages/viewpage.action?pageId=94054259 This assumes that there are the same processes and schedules in DEV and PROD service projects. They of course can differ in values of some parameters (i.e. connection strings, names of domains, passwords etc.) The best practice is to externalize all such difference into a datasource. Then you can very simply deploy service project configuration from DEV to PROD.
m
Thanks for the answer @Michal Hauzírek. Based on the info you provide is clear that: • ADDv2 load in "Current workspace" mode will be in DEV development workspace (there is only 1 development workspace on both domains). • Service workspaces on both domains. Things which are not clear to me are: • Is there any ADDv2 (in which mode?) in Service Workspace (on PROD/DEV/both? - is there any difference). • How will be the ADDv2 load brick "Released" to the Master workspace (where is needed for client's workspaces to load client's data) (currently I do have DEV devlepoment workspace with ADDv2 load with Current workspace mode - after Release the Master workspace has no process/schedule at all - so does client's workspaces). • Or maybe there is no need to have ADDv2 in every client's workspace and the load to client's workspaces is done from Service workspace. If so, I was not able to deploy the brick (ADD Load - I tried both modes Current workspace/LCM) without LDM. Maybe there is some catch, where I can point the ADDv2 in Service workspace to the LDM in different workspace. Thanks!
m
Let me go point by point: • yes, exactly - ADDv2 in dev in "current workspace" mode will be in development workspace • yes, service workspace will exiat in both domains • In servive workspace there IS an ADDv2 process and it should be in the "segment" mode - it will load all client workspaces in the selected segment. In this mode, it uses (automatically) the LDM of the master workspace of that particular segment (not LDM of the service workspace). So before you set up ADDv2 in the service mode you need to have the segment in that domain, have the master there and be logged in as domain admin • the ADDv2 from the dev workspace is autonatically skipped during release - it is not propagated to the master/client workspaces because... • ... exactlly as you write, there is no need in ADDv2 to have the process in each workspace (this is the main difference from ADDv1)