Acquire and manage data from primary and secondary data sources
Identify, analyze, and interpret trends or patterns in complex data sets
Transform existing ETL logic on AWS and Databricks
Innovate new ways of managing, transforming and validating data
Implement new or enhance services and scripts (in both object-oriented and functional programming)
Apply quality assurance best practices to all work products
Analyze, design and implement business-related solutions and core architectural changes using Agile programming methodologies with a development team
Comfortable learning cutting edge technology stacks and applications to greenfield projects
Qualifications
Proficiency in advanced Python programming, with extensive experience in utilizing libraries such as Pandas and NumPy.
Experience in code and infrastructure for Big Data technologies (e.g. Spark, Kafka, Databricks etc.) and implementing complex ETL transformations
Experience with AWS services including EC2, EMR, ASG, Lambda, EKS, RDS and others
Experience developing APIs leveraging different back-end data stores (RDS, Graph, Dynamo, etc.)
Experience in writing efficient SQL queries
Strong understanding of linear algebra, statistics, and algorithms.
Strong Experience with UNIX shell scripting to automate file preparation and database loads
Experience in data quality testing; adept at writing test cases and scripts, presenting and resolving data issues
Familiarity with relational database environment (Oracle, SQL Server, etc.) leveraging databases, tables/views, stored procedures, agent jobs, etc.
Strong analytical skills with the ability to collect, organize, analyze, and disseminate significant amounts of information with attention to detail and accuracy
Strong development discipline and adherence to best practices and standards.
Preferred Skills
Experience in Data Science, Machine Learning and AI is a plus
Financial Services and Commercial banking experience is a plus
Familiarity with NoSQL platforms (MongoDB, AWS Open Search) is a plus