Leads the delivery of solution or infrastructure development services for a large or more complex project, using advanced technical capabilities
Takes accountability for the design, development, delivery and maintenance of solutions or infrastructure, driving compliance with and contributing to the development of relevant standards
Fully understands business and user requirements and ensures design specifications meet the requirements from a business and technical perspective
Responsibilities:
Design and development of Data Ingestion, Transformation components to support business requirements.
Should be able to process heterogenous source data formats (excels, csv, txt, pdf, database, web) and be able to perform EDA (Exploratory Data Analysis), handle outliers, data cleansing and data transformation.
Should be highly data driven and able to write complex data transformation programs using PySpark, Databricks, Python.
Experience in data integration and data processing using Spark and Python.
Has hands-on experience in providing performing tuning techniques to resolve the performance issues in the data pipelines.
Provides advanced technical expertise to maximize efficiency, reliability and value from current solutions, infrastructure and emerging technologies, showing technical leadership and identifying and implementing continuous improvement plans
Works closely with Development lead and build components in agile methodology
Develops strong working relationships with peers across Development & Engineering and Architecture teams, collaborating to develop and engineer leading solutions
Skills requirement:
4 – 6 years of hands on experience in Big Data Technologies and distributed systems computing
Strong experience in Azure Databricks/Python/Spark/PySpark.
Strong experience with SQL, RESTful API, JSON
Experience with Azure Cloud resources is preferable
Experience with Angular would be nice to have
Exposure to any noSQL Databases (MongoDB, Cosmos DB and etc) is a plus
EY exists to build a better working world, helping to create long-term value for clients, people and society and build trust in the capital markets.