Data Engineer (Databricks, Python, Pyspark and SQL)- 12 + Years Experience MUST
About the role
Role: Data Engineer (Databricks, Python, Pyspark and SQL) Long Term Contract Mississauga, ON- 3 Days a week Onsite MUST
Interview Process: Online Assessment Coding Test, 2 In-Person Round of Interviews
1 Professional Reference MUST
Key Responsibilities: Design, develop, and maintain scalable and efficient data pipelines using Databricks, Python, Pyspark, and SQL. Write optimized and complex SQL queries to extract, transform, and load data. Develop and implement data models, schemas, and architecture that support banking domain requirements. Collaborate with data analysts, data scientists, and business stakeholders to gather data requirements. Automate data workflows and ensure data quality, accuracy, and integrity. Manage and coordinate release processes for data pipelines and analytics solutions. Monitor, troubleshoot, and optimize the performance of data systems. Ensure compliance with data governance, security, and privacy standards within the banking domain. Maintain documentation of data architecture, pipelines, and processes. Stay updated with the latest industry trends and incorporate best practices.
Required Skills and Experience:
Proven experience as a Data Engineer or in a similar role with a focus on Databricks, Python, Pyspark, and SQL. Strong understanding of data warehousing concepts and cloud data platforms, especially Snowflake. Hands-on experience with release management, deployment, and version control practices. Solid understanding of banking and financial services industry data and compliance requirements. Proficiency in Python scripting and Pyspark for data processing and automation. Experience with ETL/ELT processes and tools. Knowledge of data governance, security, and privacy standards. Excellent problem-solving and analytical skills. Strong communication and collaboration abilities.
Preferred Qualifications:
Good Knowledge in Azure and Databricks in highly preferred. Knowledge of Apache Kafka or other streaming technologies. Familiarity with DevOps practices and CI/CD pipelines. Prior experience working in the banking or financial services industry.
About Q1 Technologies, Inc.
Q1 is a professional organization that delivers quality products and services. Q1 specializes in Software Development, Business Consulting and Technology Integration. We provide end-to-end integrated solutions that include professional services, functional and technical support and ongoing maintenance using our on-site, off-site and off-shore resources. We offer a comprehensive range of managed services for enterprise business and technology solutions with a team of highly experienced professionals. Q1 is also a Value Added Reseller of Hardware and Software products for our partners including IBM, HP, Cisco, Lenovo, Riverbed, Oracle, Microsoft, Symantec, Lexmark and several others.
Similar Jobs
Data Engineer (Databricks, Python, Pyspark and SQL)- 12 + Years Experience MUST
About the role
Role: Data Engineer (Databricks, Python, Pyspark and SQL) Long Term Contract Mississauga, ON- 3 Days a week Onsite MUST
Interview Process: Online Assessment Coding Test, 2 In-Person Round of Interviews
1 Professional Reference MUST
Key Responsibilities: Design, develop, and maintain scalable and efficient data pipelines using Databricks, Python, Pyspark, and SQL. Write optimized and complex SQL queries to extract, transform, and load data. Develop and implement data models, schemas, and architecture that support banking domain requirements. Collaborate with data analysts, data scientists, and business stakeholders to gather data requirements. Automate data workflows and ensure data quality, accuracy, and integrity. Manage and coordinate release processes for data pipelines and analytics solutions. Monitor, troubleshoot, and optimize the performance of data systems. Ensure compliance with data governance, security, and privacy standards within the banking domain. Maintain documentation of data architecture, pipelines, and processes. Stay updated with the latest industry trends and incorporate best practices.
Required Skills and Experience:
Proven experience as a Data Engineer or in a similar role with a focus on Databricks, Python, Pyspark, and SQL. Strong understanding of data warehousing concepts and cloud data platforms, especially Snowflake. Hands-on experience with release management, deployment, and version control practices. Solid understanding of banking and financial services industry data and compliance requirements. Proficiency in Python scripting and Pyspark for data processing and automation. Experience with ETL/ELT processes and tools. Knowledge of data governance, security, and privacy standards. Excellent problem-solving and analytical skills. Strong communication and collaboration abilities.
Preferred Qualifications:
Good Knowledge in Azure and Databricks in highly preferred. Knowledge of Apache Kafka or other streaming technologies. Familiarity with DevOps practices and CI/CD pipelines. Prior experience working in the banking or financial services industry.
About Q1 Technologies, Inc.
Q1 is a professional organization that delivers quality products and services. Q1 specializes in Software Development, Business Consulting and Technology Integration. We provide end-to-end integrated solutions that include professional services, functional and technical support and ongoing maintenance using our on-site, off-site and off-shore resources. We offer a comprehensive range of managed services for enterprise business and technology solutions with a team of highly experienced professionals. Q1 is also a Value Added Reseller of Hardware and Software products for our partners including IBM, HP, Cisco, Lenovo, Riverbed, Oracle, Microsoft, Symantec, Lexmark and several others.