Real time visualization of location wise financial transactions (volume and amount) for various schemes[such as PMAY(G), MGNREGS, PMGSY,NRLM, DDU-GKY]
Abstract : In this paper we are proposing an adaptive and real-time approach to resolve real-time fnancial data integration latency problems and semantic heterogeneity. Due to constraints that we have faced in some projects that requires real-time massive fnancial data integration and analysis, we decided to follow a new approach by combining a hybrid fnancial ontology, resilient distributed datasets and real-time discretized stream. We create a real-time data integration pipeline to avoid all problems of classic ExtractTransform-Load tools, which are data processing latency, functional miscomprehensions and metadata heterogeneity. This approach is considered as contribution to enhance reporting quality and availability in short time frames, the reason of the use of Apache Spark. We studied Extract-Transform-Load (ETL) concepts, data warehousing fundamentals, big data processing technics and oriented containers clustering architecture, in order to replace the classic data integration and analysis process by our new concept resilient distributed DataStream for online analytical process (RDD4OLAP) cubes which are consumed by using Spark SQL or Spark Core basics.
? Mixed reality (MR) merges real and virtual world elements giving the sensation that they coexist in the same environment.
? The design of WireVis was based on an analysis of the current work of fraud analysts with their existing tools. To manage the enormous amount of information, analysts first filter the data by geographic region using a set of specific keywords and other criteria (like amounts).
? Most existing clustering techniques such as k-means O(kn) or single-link clustering O(n 2 ) require minutes to hours to compute as n becomes large, which is unacceptable in our case.
? To resolve classic ETLs problems, reduce latency of data processing, we are going to use a big data dedicated technologies with an enhanced data processing pipeline using a based ontology metadata and resilient distributed datasets.
? To avoid this problem, we need to consolidate general ledger and sub-ledgers in a common repository, in other words, a structured and detailed data warehouse.
? The main issue is heterogeneity of these information systems relational schema. We need to have a unifed business metadata base.
• Several works in the information visualization field have been proposed such as Table Lens and data sheets and are now part of commercial information visualization suites.
• The clusters are often not optimal for the analyst’s current purpose, so interactive reclustering has proved highly useful in exploratory visualization since it will provide much better clusters for further exploratory analysis.
• An increase in false negatives would cast serious doubts on the financial institutions who would appear as purposely harboring fraudulent activities, not to mention having to pay large fines.
? The main advantages of virtualization is that security and management technics are already defined and there is no need of extra configuration for a good processing.
? The use of containers provides several benefits like using native performance, startup time and memory space requirement. It shows a good performance and decrease latency in a short time frame report delivery.
? A relational OLAP (ROLAP), multidimensional OLAP (MOLAP) and hybrid OLAP (HOLAP), a mixed approach, which combines advantages of ROLAP and MOLAP.
|