Data Engineer - Big Data Platform

Data Engineer - Big Data Platform   On behalf of our client, Procom Consultants Group is currently searching for a Data Engineer - Big Data Platform. The Senior Data Engineer will join the Information Management team and focus on the development of big data reporting and analytics solutions, working closely with business partners and stakeholders to meet their decision-support requirements. As the consolidation point for all business intelligence and business analytics functions across the company, the Information Management team's goal is to empower the organization to drive efficiencies and savings across all facets of the business. This is accomplished by providing internal and external customers with precision reporting, query and analysis, statistical, descriptive analysis, and performance measurement (dashboards, scorecards and indicators). Data Engineer - Big Data Platform Job Description: • Working in collaboration with customers across the organization (Claims, Insurance, Finance, Strategic Analytics, etc.) to plan, scope, execute and sustain data-based solutions • Acting as a Lead Data Engineer in machine learning projects by building and deploying models. • Responding to internal and external ad hoc requests, review and clarify data requirements, ensuring report content is acceptable within policy and privacy protocols • Providing subject matter expertise within the department and to clients on data sources, reporting workflows, business process, and the appropriate tools with which to analyze their data • Participating with corporate data user teams, developing data validation and test plans, performing user acceptance testing,  and providing feedback to development and sustainment teams • Conducting analysis for moderate to complex requests, defining data fields and determining data availability, developing information layout, format and interactivity. Presenting findings and providing clarification.   Data Engineer - Big Data Platform Mandatory Skills:  • Advanced skill in programming languages such as Scala or Java • Working experience with Big Data platforms, with exposure to Hadoop ecosystem (HDFS, Hive, Spark, Kafka) • Working experience in building and deploying machine learning models • Advanced working SQL knowledge and experience working with relational databases and SQL • Strong analytic skills related to working with unstructured datasets. • Demonstrated ability to work with large and complex datasets, while managing priorities and responding to time pressures • Detail-oriented with demonstrated ability to meet deadlines, manage multiple priorities and work effectively under pressure • Strong data quality management process understanding, data analysis and data profiling • Ability to apply critical thinking skills to troubleshoot and perform root cause analysis on technical problems and solution design • Providing technical advice and guidance to staff in resolving complex data ingestion and transformation issues • Experience with performance tuning and code optimization • Design, develop and enforce best practices and standards around data engineering • Ability to work effectively with a team or independently, as well as lead small teams as needed • Understanding of Agile Methodologies • Excellent interpersonal, verbal and written communication skills to work with Managers, Directors & Executive level stakeholders • Experience with reporting and visualization tools, such as Tableau, user interface design, and iterative customer-driven design processes would be an asset. Data Engineer - Big Data Platform Start Date: ASAP Data Engineer - Big Data Platform Job Duration: Permanent Data Engineer - Big Data Platform Work Location: British Columbia, 2 days onsite and 3 days remote. Full Vaccination Proof (2 Two shots) is mandatory prior to hire.