Lead AI Trainer / Prompt Evaluator Job at Social Discovery Group, Remote

Mi9YU2ZYUjA4eHk0V1hTcTMwNVZnb3ljN0E9PQ==
  • Social Discovery Group
  • Remote

Job Description

( Social Discovery Group ( (SDG) is the 3rd largest social discovery company in the world, uniting 60+ brands with 500 million users. We solve the problems of loneliness, isolation, and disconnection by transforming virtual intimacy into the new normal. Our portfolio includes online communication platforms focusing on AI, game mechanics, and video streaming - Dating.com, DateMyAge, Cupid Media, Dil Mil, Kiseki, and others. 

SDG invests in IT startups around the world.  Our investments include Open AI, Patreon, Flo, Clubhouse, Woebot, Flure, Astry, Coursera, Academia.edu, and many others.

We bring together a team of like-minded people and IT professionals specializing in the creation and development of globally impactful social discovery products. Our international team of 1200 professionals and digital nomads works all over the world. 

Our teams of digital nomads  work remotely from  Cyprus, Malta, the USA, Armenia, Georgia, Kazakhstan, Montenegro, Poland, Latvia, Serbia, Spain, Portugal, UAE, Israel, Turkey, Thailand, Indonesia, Japan, Hong Kong, Australia and many other locations. 

In August 2024, we achieved  Great Place to Work US Certification™!  This achievement reflects our core belief that a truly exceptional workplace is built on trust, pride, and camaraderie—not just great perks.

We are looking for a meticulous Lead AI Trainer / Prompt Evaluator to oversee and enhance  the quality of AI model dialogues  with users. 

Your main tasks will be:

  • Review and assess the quality of datasets produced by the content team, ensuring accuracy, consistency, and adherence to labeling guidelines.
  • Provide constructive feedback and help maintain a high quality bar across annotators.
  • Personally perform high-quality annotation and dataset creation as a subject-matter expert.
  • Validate conversational AI model outputs (e.g., SbS evaluations, error tagging, scenario stress-testing) to detect issues before production release.
  • Work closely with the ML team to refine instructions, dataset specifications, and evaluation criteria for model training cycles.
  • Identify recurring issues in annotations or model behavior and suggest improvements to workflows, QA processes, or validation pipelines.
  • Contribute insights on prompts and data strategies to accelerate model quality improvements.

We expect from you:

  • Experience in dataset annotation and content validation for conversational AI / chat models / NLP products.
  • Hands-on experience with SbS (side-by-side) or other model evaluation frameworks (e.g., quality scoring, tagging, error categorization).
  • Ability to analyze conversational responses and identify accuracy, tone, safety, and contextual relevance issues.
  • Experience preparing or following annotation guidelines and QA checklists.
  • Strong attention to detail and ability to consistently apply evaluation criteria.
  • Ability to provide clear and constructive written feedback to other annotators.
  • Familiarity with prompt engineering practices is a plus.
  • Familiarity with LLM-based automated evaluators / AI-as-a-judge is a plus.
  • English proficiency (B2+) in reading and writing.

What do we offer:

  • REMOTE OPPORTUNITY  to work full time;
  • Vacation  28 calendar days per year;
  • 7 wellness days per year  (time off) that can be used to deal with household issues, to lie down and recover without taking sick leave;
  • Bonuses up to $5000  for recommending successful applicants for positions in the company;
  • 50%  payment for professional training, international conferences and meetings ;
  • Corporate discount for  English lessons;
  • Health benefits.  According to the paychecks, if you are not eligible for corporate medical insurance, the company will compensate you with up to $ 1,000 gross per year per employee. This can be spent on self-purchase of health insurance or on doctor's fees for yourself and close relatives (spouse, children);
  • Workplace organization.  The company provides all employees with an equipped workplace and all the necessary equipment (table, armchair, wifi, etc.) in our offices or co-working locations. In the other locations, the company provides reimbursement of workplace costs up to $ 1000 gross once every 3 years, according to the paychecks. This money can be spent on the rent of the co-working room, on equipping the working place at home (desk, chair, Internet, etc.) during those 3 years;
  • Internal gamified gratitude system:  receive bonuses from colleagues and exchange them for our merchandise, team building activities, massage certificates, etc. 

Sounds good? Join us now!  

Job Tags

Remote job, Full time, Work at office,

Similar Jobs

Brookfield Schools

Special Education-Science Teacher Job at Brookfield Schools

 ...disabilities*, has an opening for a full-time highly qualified teacher who possess a teacher of the handicapped or students with disabilities...  ...Dental insurance * Health insurance * Vision insurance Education: * Bachelor's (Required) Experience: * Teaching: 1 year (... 

ReNEW Schools

Grades K-2 Special Education Teacher Case Manager Job at ReNEW Schools

 ...POSITION DETAILS Seeking a passionate Special Education Teacher Case Manager (TCM) with expertise in ELA and Math, TCMs co-teach,...  ...tracking system in collaboration with the Special Education Coordinator, facilitating effective decision-making for individual program... 

Ascend Rehab Services Inc

Paraprofessional/ Special Education Teacher Aide-Modesto Job at Ascend Rehab Services Inc

 ...Instructional Aide/ Paraprofessional (Special Education) Join Our Mission to Empower Young Learners About Ascend: Ascend is a multidisciplinary...  ...students in a classroom setting. Under the guidance of a teacher, the Instructional Assistant will help facilitate... 

Community Health System

Clinical Research Specialist Job at Community Health System

 ...seniority ~ Free money toward retirement with a 403(b) and matching contributions Responsibilities We are looking for a Clinical Research Specialist to join our growing team! Working closely with the Principal Investigator, you will be responsible for clinical... 

medmix

Injection Molding Operator - Night Shift Job at medmix

 ...Switzerland. Our shares are traded on the SIX Swiss Exchange (SIX: MEDX). Job Description Effectively run plastic injection molding machines within safety and quality standards To be able to work safety and follow all safety guidelines. Able to work a...