A Data Engineer typically requires 36 years of experience in designing, building, and optimizing big data pipelines, architectures, and data sets . Key experience includes proficiency in SQL, Python/Scala/Java, ETL processes, and cloud platforms (AWS, Azure, or GCP). Candidates should possess strong skills in data modeling, warehousing (Snowflake, Redshift), and distributed systems like Spark or Kafka.

Core Experience & Qualifications

• Technical Proficiency: 3-5+ years of experience in data engineering, data warehousing, or software engineering roles.
• Data Pipelines (ETL/ELT): Extensive experience in designing, building, and maintaining robust, scalable data pipelines.
• Languages: Strong programming skills in Python, Java, or Scala for data processing and automation.
• SQL & Database Management: Expert-level SQL skills for querying and manipulating data in relational (PostgreSQL, MySQL) and NoSQL (MongoDB, Cassandra) databases.
• Big Data Technologies: Hands-on experience with Apache Spark, Hadoop, Kafka, or similar frameworks for large-scale data processing.
• Cloud Platforms: Experience with cloud services such as AWS (S3, EMR, Redshift), Azure (Data Factory, Data Lake), or GCP.
  
  

Key Responsibilities & Competencies

• Data Modeling & Warehousing: Designing efficient, secure data models (star/snowflake schemas).
• Data Infrastructure: Managing data storage systems and ensuring data quality and security.
• Collaboration: Working with data scientists and analysts to support data-driven initiatives.
• Tools: Familiarity with workflow orchestration tools like Apache Airflow.
  
  

Education & Education Background

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related quantitative field.
  
  

Preferred Skills

• Experience with containerization tools (Docker, Kubernetes).
• Knowledge of data visualization tools (Tableau, Power BI).
• Certifications in cloud platforms (AWS, GCP) or big data technologies.