hi all:slightly_smiling_face: i want to create an ...
# gooddata-platform
m
hi all🙂 i want to create an alert on a 'latest loaded datapoint' kpi change. my goal is to track whether my latest loaded datapoint is eg more than 2 hours behind the current timestamp. for this purpose it obvi doesn't make much sense to compare my current kpi value with some absolute number as shown at the screenshot. so my question is whether it's technically possible to compare it to another metric? or the best approach will be just to create another kpi that will be outputting the difference between those two metrics in a numeric format and just compare that output to some preset absolute value? thanks in advance for your advices here!
m
Hi Masha, it is currently not possible to compare metric to another metric for the purpose of an alert. So your approach with another metric that will be constructed as a difference or ratio of those two metrics sounds like a good approach here. Regarding the alerts, please note that you will only receive it once when a threshold is crossed. It will not alert you about a consequent event, unless the value goes below the threshold again. So for example if you set "alert above 100" and the metric will behave like this: 7AM - 68 8AM - 95 9AM - 102 (alert triggered) 10AM - 150 (alert NOT triggered) 11AM - 96 (alert not triggered, but reset) 12AM - 180 (alert triggered)
BTW - if your goal is more internal monitoring than business-user alerting, maybe this article could help as well: https://www.gooddata.com/blog/gooddata-slack-integration/ It describes how to integrate GD notifications (more complex than just KPI alerts to Slack (or other tool with webhook).
But in both cases, I am not exactly sure how to handle the "2 hours behind current timestamp" condition. Since GD platform does not have the time element in date dimension. Maybe by loading the current timestamp value to some other fact/dataset frequently. But that could potentially become expensive because of frequent loads.
m
thanks for the fast and detailed reply Michal! noted regarding the approach and thanks for pointing on the threshold logic. also noted regarding current timestamp, i hoped GD has some solution for it cos you're right that frequent loads are not the best approach but i'll think of some other options a bit more. btw regarding the webhook article, we tried to set it up last year but it didn't work out. we found another solution but maybe this feedback will be helpful for you - as far as i recall channelConfigurations part never worked. it was populating 'Custom HTTP headers' with some extra values. we were removing those via api calls as instructed by GD team but even after that it never worked eventually🥲)
m
Oh, sorry to hear that. I used this integration with slack a few years ago and it worked back then. Regarding the timestamp - depending how you are loading the data, maybe loading current timestamp with your actual data (with the same frequency and process) would still work. If you are querying your database, you can have there a view with current_timestamp function or something like that and it will get you the live timestamp at the moment of querying the data for load by GoodData. So if it is enough to alert you after you have loaded the data that the data might be too old, it would still work. It would not let you know that the whole load did not happen for whatever reason, but for that you might have dataload error notification… But anyways your initial idea with KPI alert would anyhow be triggered by a data load. Of course if you are loading data not from a live database but from i.e. some exported CSV, you won’t get “live” timestamp of the load this way.
m
yeah actually this 'latest loaded datapoint' that we already have is basically the datapoint of the latest event included in the latest load. we're monitoring the dataloads failures separately while this alert is planned to monitor whether we're doing a dry fetch so to say. so it's more of a database health metric exposed to our internal non tech users. does that make sense? so i guess we should: • either just create a separate load that will be loading just 'current timestamps' which is also not perfect for many reasons (the biggest one being that as far as i understood the minimal frequency interval is still 15 mins) • or just monitor our database state somewhere outside of GD (eg in airflow)