Test Consultant

Test Consultant (GenAI)

Belgrade/Novi Sad, Serbia | Full-time

About Sixsentix

What crosses your mind when you hear 'employee experience'? Is the following something you are looking for?

Sixsentix is crafting a work experience that allows people to achieve their best potential while feeling connected, recognized and endorsed. Our aim is to be a fully agile organization and through our methodology we ensure the best test automation services to our clients. Due to our agile mindset, we are continuously pushing our organization forward by being open-minded and by welcoming new ideas and improvements.

Company culture is essential and through our values we aim to reshape the way we work. To put it nicely, with a lot of grit we break the limits, but we don't stop there because we know that continuous improvement and ownership of everything we do, combined with great trust and team spirit, is the perfect recipe for success.

Job Description

We are looking for a Test Consultant with a technical foundation and specific experience or interest in 
testing Generative AI (GenAI)
 solutions. The role focuses on validating the quality, consistency, and robustness of GenAI components (e.g., LLMs, RAG systems, prompt pipelines). This includes both 
functional
 and 
non-functional testing
, as well as the design of evaluation strategies tailored to the non-deterministic nature of GenAI.

Apart from culture fit being of paramount importance, for our current job opening we are looking for the following must-have skill set:

• Understanding of prompt engineering, Retrieval-Augmented Generation (RAG), and agent-based pipelines
• Familiarity with challenges in GenAI such as hallucinations, context sensitivity, reproducibility, and prompt drift
• Experience designing tests that cover semantic accuracy, relevance, tone, and factuality of generated outputs
• Awareness of GenAI-specific risks: misuse, bias, safety, and toxicity
• Experience with test case design, automated test execution, regression testing, and edge case simulation
• Familiarity with API testing tools (e.g. Postman) and test frameworks (e.g. Pytest, unittest)
• Ability to integrate test suites into CI/CD pipelines
• Comfortable writing validation scripts in Python
• Familiarity with MLflow, or similar tools for model lifecycle management and result tracking
• Experience testing GenAI APIs (e.g., OpenAI, Azure OpenAI, Anthropic, Hugging Face, etc.)
• Exposure to containerized environments (Docker) and cloud platforms (e.g. AWS, Azure)
• Knowledge of version control (Git) and collaboration tools (e.g., Jira, Confluence)
• Solid grasp of data structures and formats (JSON, CSV, Parquet)
• Ability to generate or curate high-quality evaluation datasets for GenAI outputs
• Experience defining and applying both automated and human-in-the-loop evaluation approaches
• Academic degree in Information Technology, Computer Science, or a comparable field.
• Proficiency in written and verbal communication in English; knowledge of German is a plus.

The following skills are good-to-have:

• Knowledge of metrics and evaluation techniques for LLMs (BLEU, ROUGE, BERTScore, etc.)
• Understanding of explainable AI (XAI) concepts to support model interpretability and testability
• Experience with MLOps workflows and AI/ML model validation pipelines
• Prior exposure to LLM fine-tuning or prompt tuning is a plus
• Strong attention to detail and critical mindset
• Comfortable working with uncertainty and non-deterministic systems
• Collaborative and communicative – able to align across disciplines (engineering, data science, product)
• Curious and up-to-date with emerging AI technologies and testing practices

If you are ambitious to gain more experience in:

• Designing and implementing testing strategies for systems built with LLMs and GenAI tools
• Evaluating the behavior of prompts, agents, and pipelines under diverse inputs and edge cases
• Defining measurable evaluation criteria for output correctness, consistency, and safety
• Analyzing, reporting, and communicating test results to technical and non-technical stakeholders (Sixsentix customers)

We seek to continuously improve our benefits package and currently you can discover the following:

• International career opportunities with 6 global locations
• Disciplines where you can contribute and develop locally on a strategic level
• All-hands meetings where we foster transparency
• Individual budget for education and professional development
• Loyalty program
• Work flexibility tailored to individual needs
• New parents' benefits package
• Tailor-made team events and team buildings for each team
• Wellness program (Private health insurance, accident insurance 24/7)
• 15 days of sick leave without salary reduction with 3 days without doctor's notice
• Extra vacation days plus additional company days off for Holidays, between New Year and Christmas.

And for more stories about us, you can check out Virtual Tour on our website.

We look forward to receiving your electronic application (CV, reference letters, etc.) by email to

Find more information about Sixsentix on