Requirements
ETL Development:
Design, develop, and maintain ETL jobs using Talend Studio.
Integrate data from various sources such as databases, flat files, APIs, cloud services, and more.
Perform transformations on data to ensure consistency and accuracy before loading it into the target systems.
Data Modeling:
Work with data architects to understand the data model and help translate business requirements into technical solutions.
Design and implement data pipelines to facilitate smooth data flow.
Data Quality Management:
Apply data validation rules, cleansing, and transformation logic to ensure data quality.
Monitor and resolve data issues, ensuring timely and accurate data loading into target systems.
Performance Tuning:
Optimize ETL processes for performance, scalability, and reliability.
Monitor system performance, and troubleshoot bottlenecks in data processing.
Collaboration with Other Teams:
Work closely with business analysts, data architects, and other IT teams to gather requirements and provide technical solutions.
Collaborate with database administrators (DBAs) to ensure the smooth performance of databases used in data integration.
Documentation:
Document ETL workflows, technical specifications, and processes for future reference.
Ensure that all solutions adhere to best practices for security, performance, and scalability.
Testing and Debugging:
Perform unit testing of data integration jobs to ensure functionality.
Debug and troubleshoot Talend jobs and workflows, addressing any issues during deployment.
Maintenance and Support:
Provide ongoing support for Talend-based data integration solutions.
Regularly update and maintain the Talend environment.
Skills & Qualifications:
Technical Skills:
Proficiency in Talend Studio (Open Studio, Talend Cloud, or Talend Data Integration).
Strong knowledge of SQL and relational databases (such as MySQL, PostgreSQL, Oracle, etc.).
Familiarity with data warehousing concepts and platforms like AWS Redshift, Google BigQuery, or Snowflake.
Experience with cloud-based tools and integration (AWS, Azure, Google Cloud).
Knowledge of REST APIs, SOAP, and file-based integration.
Familiarity with Java or JScript for custom transformations and integration.
Soft Skills:
Strong problem-solving skills.
Good communication and collaboration skills.
Ability to handle multiple projects and meet deadlines.
