Big Data Engineer

Job Title: Big Data Engineer

Location: Bengaluru

Experience Required: 5 to 8 years

Job Description:

We are looking for a skilled Big Data Engineer with strong hands-on experience in PySpark and SQL. This is a hybrid role, requiring the candidate to work both remotely and from the office as needed. The ideal candidate should be comfortable working on large-scale data pipelines and big data processing frameworks.

Key Responsibilities:

• Develop, optimize, and maintain scalable data pipelines using PySpark.
• Write efficient and complex SQL queries for data transformation and extraction.
• Work closely with data architects, analysts, and other stakeholders to deliver high-quality data solutions.
• Handle structured and unstructured data from various sources.
• Perform performance tuning and troubleshooting of big data jobs.

Required Skills & Experience:

• Hands-on experience with PySpark for big data processing.
• Strong expertise in SQL for data manipulation and analysis.
• Good understanding of distributed data systems and big data architecture (e.g., Hadoop, Spark, Hive, etc.).
• Ability to work independently in a hybrid setup.
• Comfortable taking ownership as a Big Data Engineer.

Preferred Qualifications:

• Experience with cloud platforms (AWS, Azure, or GCP) is a plus.
• Familiarity with version control systems (e.g., Git).
• Good communication and collaboration skills.