A leading technology company engaged the Focal Point Data Analytics team to perform a data quality assessment for their 11 top-tier systems and assist in developing a roadmap to modernize their data warehousing environment. The Tech Company was only eight months away form its initial product launch, but had serious concerns about the data quality and data management practices and sought to gain enhanced visibility into their data assets and supporting infrastructure.
The Focal Point team was initially tasked with executing a data quality assessment for the Tech Company’s critical SQL database and DW environments, ERP system (SAP), and top-tier applications. After the successful execution of the data quality assessments, the Tech Company involved Focal Point in the creation of a continuous monitoring solution for ongoing data quality support, data migration assessments, and the development of a data governance plan.
Ultimately, the Focal Point team helped the Tech Company overcome three significant data quality challenges.
Project Summary
Challenge 1: Identifying Data Quality Issues
The Tech Company had three key priorities, understanding their existing: 1) data assets, 2) data quality health, and 3) supporting infrastructure. Lacking an established data governance program contributed to systemic data integrity issues, which led to system redundancies and errors, outages, operational inefficiencies, and diminished confidence from data consumers.
Before the Tech Company could address the problem, they needed visibility into the impact of the data quality issues on critical IT systems and business operations. Focal Point performed a design assessment of the Tech Company’s top-tier systems and integration tools. The Focal Point team then executed a data quality assessment of these systems, which involved advanced data profiling techniques and targeted testing using Alteryx. The team also integrated the company’s on-premise Tableau server and dashboarding platform to provide continuous data quality that monitored KPIs.
The data quality assessment and continuous data quality monitoring solution conducted by Focal Point allowed the Tech Company to increase the data quality health measurement of its top-tier systems by 23%. In addition, Focal Point’s initial design assessments led to the discovery of 60 observations, which provided the Tech Company with valuable insights into weaknesses in system and data integration and data management practices. With this information, the Tech Company was able to fund additional initiatives to harden integration, clean up data management systems, deploy data quality tools and practices, and develop data governance policies and procedures.
Challenge 2: Understanding Cost Impact
The Tech Company sought insights into the cost drivers related to the data quality of their 11 in-scope top tier systems, as well as clarity on the relationship between the Tech Company’s business metrics and IT service costs. Focal Point uncovered 60 observations in the following areas that had an impact on cost:
- Data Integration (23): The main cost drivers in this area included inconsistent formulation and maintenance of rules within in-scope integration tools, misalignment of business rules and data integration logic, data disparity (difference in data structure, format, and use of values), and significant reliance on alternate manual steps developed outside of integration tools to address deficiencies with the integration technology.
- IT Operations and Maintenance (16): Data quality issues resulted in deficiencies that impacted the effectiveness of IT operations, including system maintenance and data management practices.
- Data Modeling & Design (11): These deficiencies depicted the Tech Company’s lack of understanding around its data assets, misaligning and miscommunication of data requirements, and the lack of documenting or retaining evidence related to data modeling and design during SDLC.
- Data Governance & Stewardship (10): These deficiencies represented data quality issues that resulted from the immediate lack of master data management and data governance programs within the organization.
Focal Point developed a remediation plan for each of these data quality observations as part of the initial design assessment. Focal Point then applied a risk ranking methodology to help the company prioritize its remediation efforts, which included details on the level of complexity and associated cost impact and ROI.
Challenge 3: Modernizing the Data Warehouse
Focal Point assisted in the development of an enterprise data warehouse (EDW) modernization roadmap. Previous attempts by the Tech Company had been unsuccessful due to the following design problems:
- Lack of ETL utilities and data modeling expertise meant the SQL EDW was simply used as a data repository. In the existing architecture, dedicated SQL databases (DBs) stored raw tables from source systems extracted via SQL Server Batch Jobs. As a result, the Tableau server was leveraged by a separate IT organization to prepare the data rather than preparing it upfront in the EDW.
- Lack of data governance, data management, and change management policies led to mismanagement of the SQL EDW environment. Consequently, ongoing issues with system availability and integrity adversely impacted the business.
- The existing SQL EDW was not able to manage (i.e., store, model, analyze) streaming data, including data coming from the devices of the Tech Company.
Success
Through a collaborative effort, Focal Point helped design a data warehouse modernization plan, including architecture design, tool selection, and a phased transition plan, that would incrementally migrate data and/or functionality to the component technologies, which would eventually make up the future EDW environment.
