In this post, I would like to introduce a relatively new product of Yellowfin Suite: Signals. Yellowfin introduced Signals in late 2018 as an automated analysis and discovery tool of business data. Signals is a product to be used on top of standard BI features. What I mean by standard BI functionalities are features like connecting to a database, extracting the data, manipulating it and lastly presenting it visually with reports, graphs, dashboards etc.
Usually, Data Analysts prepare dashboards or reports and then try to find insights and anomalies in the data. Preparing the dashboards and reports is the traditional process of BI world, all the BI tools provide similar functionalities. For me, the most important part begins after this. How will we interpret the reports and take actions by looking at the figures? When I see a sudden increase of a certain product in my daily sales report, how can I understand if this is important or not? What could be the reasons behind this increase? These are the questions that must be addressed first, answered secondly, and then finally these answers must lead to a business decision. It could still have been a manageable task to deep dive into a very small dataset consisting only several dimensions. But when you have a dataset that contains many dimensions, features, columns this analysis job becomes almost impossible to do repetitively and without mistakes.
Signals come into picture here. It is aimed to remove the burden of Data Analyst whose task is to find anomalies and interpret them. The cool thing about Signals is that it makes use of computing power that we conveniently have today to automatize the Data Analyst’s job. When I say anomalies, I mean the differences than the norm/normal. An anomaly can be a (positive or negative) change in the trend, a spike (or drop) or differences between periods. To be able to realize and grasp the importance these changes (if there is any), we need to make lots of manual calculations.
Just to illustrate, I can easily see a sudden increase in sales of a certain product in my daily report. But I need to make use of statistics to interpret if this increase is within normal ranges (for my business) so that I can decide to investigate further or not. Confidence intervals, moving averages must be calculated. If this sudden increase is statistically important, then further analysis is needed. I also need to answer why I have this increase. I need to find possibly related and correlated metrics. For example, there might be another related product whose sales also increased. Maybe purchase of one, leads to purchase of another. Then I would like to know the relative importance of this product. What is the percentage of sales of this product among all products? If it is a product that has a very low percentage in my total sales, I might ignore it.
Lots of questions, right? And it does not end here, we can even go into deeper by looking at other metrics that contribute (lead) to this increase by different magnitudes. Following the same example, the different dimensions might contribute/affect this increase. Let’s assume after the manual calculations, we find that stores and days of the week are the two metrics that cause this change. I.e. different stores and different weekdays create statistically significant effects on this change. Then I still need to calculate which specific stores or which days of the week affect this variance.
In the above example, all the manual calculations are handled by Signals, seamlessly. You just need to do the initial setup with all the relevant inputs, metrics, dimensions, time frames and the analysis type you want Signals to make. Then the magic starts. Either as a Data Analyst who does these analyses, or as a consumer of these analysis (i.e. manager, store manager etc) you will get automatic notifications as there are important changes (Signals) occur in your data. Once you get a Signal, it automatically means that it is statistically significant, so you need to deep dive into it. Signals also create related and correlated metrics to this Signal. What are the metrics that move together with this increase? (An important Statistics 101 reminder: Correlation does not imply causation)
Moreover, Signals shows the relevance of this metric, i.e. how important this metric is in the overall picture.
Last but not least, Signals provide variance analysis to show the metrics that contribute to the change.
All are automatic. All are as it happens. You can schedule Signals runs as you wish, monthly, weekly, daily, every hour etc, no limits here.
Interested to see more of Yellowfin Signals? We are at booth 78 at the Jaarbeurs Utrecht the 18th and 19th of September 2019.
More information about Signals can be found at: https://www.yellowfinbi.com/suite/signals
