As a Data Engineer in our GenAI team, you will play a crucial role in designing, building, and optimizing the data pipelines that support our generative AI tools. You will work closely with data scientists, AI engineers, and product owners to ensure that our customer chatbot and other GenAI applications are robust, scalable, and effective in meeting our clients' needs. This role offers a unique opportunity to shape the development of innovative AI-driven tools and make a meaningful impact on customer engagement.
Your tasks a GenAI Data Engineer
- Data Pipeline Development: Design, build, and maintain data pipelines to efficiently support GenAI tooling, with a focus on handling diverse data sources and diverse databases.
- Database Expertise: Identify, deploy and maintain data services for a variety of use cases. For example, you will orchestrate and optimize our use of vector databases, used for semantic search, vector search and other variants of knowledge retrieval, which form a crucial component for Retrieval-Augmented Generation (RAG) applications. You also maintain conversation history management services to keep track of ongoing conversations and ended conversations for further analytics.
- Data Quality & Management: Ensure the integrity, accuracy, and scalability of data to support the high-performance needs of GenAI applications.
- Collaboration with AI Teams: Work closely with data scientists, AI engineers, and other team members to support the development and deployment of the chatbot and other AI-powered tools.
- Optimization & Performance Tuning: Continuously optimize and monitor data pipelines and infrastructure to maximize efficiency and reliability for GenAI applications.
- Automation & Monitoring: Develop automation tools and monitoring solutions to support continuous integration and delivery (CI/CD) of GenAI models.
- Continuous Improvement: Connect real-life conversations with data processing pipelines to extract valuable information about the quality of conversations with the GenAI bot. Extract and provide valuable feedback and ways to continuously improve the GenAI chatbot applications.
Our ideal GenAI Data Engineer
- Master’s Degree in Computer Science, Data Engineering, or a related field.
- Experience with RAG based solutions: experience in handling large amount of unstructured data is a plus.
- Cloud Expertise: Hands-on experience with cloud platforms (e.g., AWS, Azure, GCP) and their data solutions.
- Programming Skills: Strong in Python, SQL, and familiarity with additional languages such as Java is a plus.
- Familiarity with Machine Learning/AI Pipelines: Experience working with ML frameworks and an understanding of GenAI models is an advantage.
- Problem-Solving Mindset: Ability to think critically and solve complex data engineering challenges to support innovative AI projects.
What we have to offer:
- Innovative Projects: Contribute to cutting-edge AI projects that transform customer experience.
- Career Growth: Opportunity to work with and learn from a talented AI team while gaining experience in a rapidly growing field.
- Impactful Work: Your contributions will directly impact the development of new customer solutions and our company's AI capabilities.
- Collaborative Environment: Be part of a team that values diversity, knowledge sharing, and a commitment to developing groundbreaking technology.