Lead AI Trainer / Prompt Evaluator Job at Social Discovery Group, Remote

Mi9YU2ZYUjA4eHk0V1hTcTMwNVZnb3ljN0E9PQ==
  • Social Discovery Group
  • Remote

Job Description

( Social Discovery Group ( (SDG) is the 3rd largest social discovery company in the world, uniting 60+ brands with 500 million users. We solve the problems of loneliness, isolation, and disconnection by transforming virtual intimacy into the new normal. Our portfolio includes online communication platforms focusing on AI, game mechanics, and video streaming - Dating.com, DateMyAge, Cupid Media, Dil Mil, Kiseki, and others. 

SDG invests in IT startups around the world.  Our investments include Open AI, Patreon, Flo, Clubhouse, Woebot, Flure, Astry, Coursera, Academia.edu, and many others.

We bring together a team of like-minded people and IT professionals specializing in the creation and development of globally impactful social discovery products. Our international team of 1200 professionals and digital nomads works all over the world. 

Our teams of digital nomads  work remotely from  Cyprus, Malta, the USA, Armenia, Georgia, Kazakhstan, Montenegro, Poland, Latvia, Serbia, Spain, Portugal, UAE, Israel, Turkey, Thailand, Indonesia, Japan, Hong Kong, Australia and many other locations. 

In August 2024, we achieved  Great Place to Work US Certification™!  This achievement reflects our core belief that a truly exceptional workplace is built on trust, pride, and camaraderie—not just great perks.

We are looking for a meticulous Lead AI Trainer / Prompt Evaluator to oversee and enhance  the quality of AI model dialogues  with users. 

Your main tasks will be:

  • Review and assess the quality of datasets produced by the content team, ensuring accuracy, consistency, and adherence to labeling guidelines.
  • Provide constructive feedback and help maintain a high quality bar across annotators.
  • Personally perform high-quality annotation and dataset creation as a subject-matter expert.
  • Validate conversational AI model outputs (e.g., SbS evaluations, error tagging, scenario stress-testing) to detect issues before production release.
  • Work closely with the ML team to refine instructions, dataset specifications, and evaluation criteria for model training cycles.
  • Identify recurring issues in annotations or model behavior and suggest improvements to workflows, QA processes, or validation pipelines.
  • Contribute insights on prompts and data strategies to accelerate model quality improvements.

We expect from you:

  • Experience in dataset annotation and content validation for conversational AI / chat models / NLP products.
  • Hands-on experience with SbS (side-by-side) or other model evaluation frameworks (e.g., quality scoring, tagging, error categorization).
  • Ability to analyze conversational responses and identify accuracy, tone, safety, and contextual relevance issues.
  • Experience preparing or following annotation guidelines and QA checklists.
  • Strong attention to detail and ability to consistently apply evaluation criteria.
  • Ability to provide clear and constructive written feedback to other annotators.
  • Familiarity with prompt engineering practices is a plus.
  • Familiarity with LLM-based automated evaluators / AI-as-a-judge is a plus.
  • English proficiency (B2+) in reading and writing.

What do we offer:

  • REMOTE OPPORTUNITY  to work full time;
  • Vacation  28 calendar days per year;
  • 7 wellness days per year  (time off) that can be used to deal with household issues, to lie down and recover without taking sick leave;
  • Bonuses up to $5000  for recommending successful applicants for positions in the company;
  • 50%  payment for professional training, international conferences and meetings ;
  • Corporate discount for  English lessons;
  • Health benefits.  According to the paychecks, if you are not eligible for corporate medical insurance, the company will compensate you with up to $ 1,000 gross per year per employee. This can be spent on self-purchase of health insurance or on doctor's fees for yourself and close relatives (spouse, children);
  • Workplace organization.  The company provides all employees with an equipped workplace and all the necessary equipment (table, armchair, wifi, etc.) in our offices or co-working locations. In the other locations, the company provides reimbursement of workplace costs up to $ 1000 gross once every 3 years, according to the paychecks. This money can be spent on the rent of the co-working room, on equipping the working place at home (desk, chair, Internet, etc.) during those 3 years;
  • Internal gamified gratitude system:  receive bonuses from colleagues and exchange them for our merchandise, team building activities, massage certificates, etc. 

Sounds good? Join us now!  

Job Tags

Remote job, Full time, Work at office,

Similar Jobs

Enterprise

Summer 2026 Management Trainee Intern - New Bedford/Dartmouth, MA - Enterprise - Fall River, Massachusetts, United States Job at Enterprise

 ...the internship will help you stand apart from the crowd, and will be more than just a line on your resume. From day one as a paid intern with Enterprise, you'll learn what it takes to run a successful business and acquire highly marketable skills in management. Our... 

System One

SAR AML Analyst Job at System One

 ...SAR AML Analyst Location: REMOTE Pay Rate: Open to Both C2C and W2 options Position Type: Multiyear Contract Description: Perform research and analysis of client account activity to identify red flags, suspicious activity, and potential fraud... 

Chameleon Integrated Services

Military Writing Expert Job at Chameleon Integrated Services

 ...Training allowance ~ PTO and more The Position : Chameleon Integrated Services is currently looking for a remote Military Writing Expert to support one of our clients in downtown St. Louis. This role will start as a remote role and may grow to onsite... 

Green Oaks Physical Therapy

Medical Receptionist Job at Green Oaks Physical Therapy

 ...allowing us to serve the people in our communities with compassionate and competent care. We have an immediate need for a part-timeMedical Receptionist with an hourly rate of $14-16 per hour, depending on experience. Our philosophy is to provide excellent therapy... 

Provident Bank

GRC Analyst Job at Provident Bank

 ...all we do. At Provident Bank, we are committed to enhancing our customer and employees experience. POSITION SUMMARY The GRC Analyst will collaborate with process owners, internal auditors, external auditors, and other stakeholders to assist in reviewing,...