Hi - I deleted a column from a Snowflake table. Th...
# gooddata-platform
m
Hi - I deleted a column from a Snowflake table. The column wasn't referenced in the data model, but other columns in the table are. I'm now getting an error saying the column is missing, but I'm confused why it is happening. It's almost as if the column is cached somewhere. Is there any way to safely remove unreferenced columns from source tables?
m
Hi Mitchel, when you’re going to change your LDM and delete some attributes in datasets you can check the impact like so: • In GoodData Platform, there is an indicator of impact in the summary of your changes. Click on “Dependencies detected” as displayed on the image. • GoodData Cloud released a similar feature in October, called Impact Analysis. https://www.gooddata.com/developers/cloud-native/doc/2.4/model-data/evolve-your-model/monitor-model-changes/ Please note that both UI’s search for impact only in the given workspace. So before removing columns from source tables, you should always check the impact.
m
Hi @Michael Ullock - I removed the table from Snowflake only. It was never referenced in GoodData and as far as I can tell, exists nowhere in my LDM.
m
Can you please tell me which GoodData product you’re using? Is this related to the GoodData Platform or GoodData Cloud?
m
I believe I'm on Platform
image.png
@Michael Ullock - Here are the logs. CURRENT_FOLLOWER_COUNT (INTEGER) was in the table, but it wasn't referenced anywhere. I added a placeholder field back so that it would refresh again, but I don't know why the field is necessary to refresh.
Another view of my model... no reference to the field anywhere.
Ok @Michael Ullock - I tried adding the field into the model, and now I'm getting a new error: LDM field fact.linkedin_daily_follower_statistics.current_follower_count is not mapped to any source column It's definitely mapped. And when I tried to delete the field again, the error now persists and the data won't refresh.
The only reference to the field that I can find is in the Manage > Model section where the field is listed as a deleted attribute
m
Sorry for the delay in getting back to you - From the last screenshot you have attached, I can see that the Attribute has not been deleted, but rather “Disabled” - This means that the attribute was only deleted from user interface only. The way how to remove attributes and facts from the project is to use MAQL DDL, using DROP attribute function, which you can find described here: https://help.gooddata.com/doc/enterprise/en/data-integration/data-modeling-in-gooddata/data-modeling-and-maql/maql-ddl/
Regarding the new error:
Copy code
LDM field fact.linkedin_daily_follower_statistics.current_follower_count is not mapped to any source column
Please remove this new added field from your LDM
m
Yeah, I removed the filed in the UI, but the error persists.
Looking into the drop API call now
@Michael Ullock - Ok now I get this error: Can't delete object (%s) while referenced by dependent objects (%s). Use 'CASCADE' to force a drop of all references to this object. A 'CASCADE' drop will make the following changes: DROP %s, MODIFY %s.
I tried DROP CASCADE and CASCADE DROP but got a syntax error for both. What's the best way to structure the query to drop this field
Sorry, need to run to pick up the kids. What if I just delete the entire table from the model and add it back? Would that work or would I end up with all the fields stuck hidden on the back-end?
m
You call should look something like this:
Copy code
DROP {attr.example} CASCADE;
SYNCHRONIZE {dataset.example} PRESERVE DATA;
As mentioned in the article above - To prevent deletion of data from synchronized datasets, use the
PRESERVE DATA
keyword.
PRESERVE DATA
works only with
SYNCHRONIZE
. If
SYNCHRONIZE
is run without
PRESERVE DATA
, it deletes data from the synchronized dataset
m
Got it, I was putting the word CASCADE before the attribute, not after. Ok, I was able to get it to work. Thanks so much for your help and patience.
m
You’re welcome Mitchel & thanks for getting back to me and letting me know you’re all set. Hope you have a great rest of the day ahead 🙂