InfoTech Consultancy - Data Engineer - Python/PySpark

You will be part of the data delivery team and will have the opportunity to develop a deep understanding of the domain/function. You will design and drive the work plan for the optimization/automation and standardization of the processes incorporating best practices to achieve efficiency gains. You will run data engineering pipelines, link raw client data with data model, conduct data assessment, perform data quality checks, and transform data using ETL tools. You will perform data transformations, modelling, and validation activities, as well as configure applications to the client context. You will also develop scripts to validate, transform, and load raw data using programming languages such as Python/ PySpark/ SparkSQL. In this role, you will determine database structural requirements by analysing client operations, applications, and programming. Given the pace of this role, you will develop cross-site relationships to enhance idea generation and manage stakeholders. Lastly, you will collaborate with the team to support ongoing business processes by delivering high-quality end products on-time and perform quality checks wherever required. QUALIFICATIONS : - Bachelor's degree / master's degree with high rankings from reputed colleges - Preferably 5-8 years ETL /Data Analysis experience with a reputed firm - Expertise in Big Data Managed Platform Environment like Databricks using Python/ PySpark/ SparkSQL - Experience in handling large data volumes and orchestrating automated ETL/ data pipelines using CI/CD and Cloud Technologies. - Experience of deploying ETL / data pipelines and workflows in cloud technologies and architecture such as Azure and Amazon Web Services will be valued - Experience in Data modelling (e.g., database structure, entity relationships, UID etc.) , data profiling, data quality validation. - Experience adopting software development best practices (e.g., modularization, testing, refactoring, etc.) - Conduct data assessment, perform data quality checks and transform data using SQL and ETL tools - Excellent written and verbal communication skills in English - Self-motivated with strong sense of problem-solving, ownership and action-oriented mindset - Able to cope with pressure and demonstrate a reasonable level of flexibility/adaptability - Track record of strong problem-solving, requirement gathering, and leading by example - Able to work well within teams across continents/time zones with a collaborative mindset (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 3/26/2025