Internal Medicine AI Evaluation Specialist

# Internal Medicine AI Evaluation Specialist

> Weekday AI · United States (Remote) · Part-time · Posted 2026-07-08

**Workplace:** remote

**Department:** AI Training

## Description

**This role is for one of our clients**

**Compensation: $130-$180 per hour**

Join a leading AI research initiative focused on advancing healthcare-focused artificial intelligence. We are seeking experienced Internal Medicine physicians to contribute their clinical expertise toward training and evaluating next-generation AI models capable of sophisticated medical reasoning.

In this role, you will develop realistic clinical scenarios, evaluate AI-generated responses against evidence-based standards, and help improve how AI systems reason about diagnosis, treatment planning, and patient management within Internal Medicine.

We welcome Internal Medicine attending physicians, hospitalists, fellows, final-year residents, and other qualified Internal Medicine professionals who are passionate about applying their expertise to the future of healthcare AI.

## Requirements

### Key Responsibilities

### Design Clinical Scenarios

• Create realistic Internal Medicine case studies based on everyday clinical practice.
• Develop prompts covering:
• Diagnostic reasoning
• Differential diagnosis
• Treatment planning
• Risk stratification
• Evidence-based guideline adherence
• Long-term patient management

### Create Reference-Quality Responses

• Produce attending physician-level reference solutions that reflect current clinical best practices.
• Ensure recommendations align with established evidence-based guidelines.

### Evaluate AI Performance

• Review and assess AI-generated clinical responses using structured evaluation frameworks.
• Identify reasoning gaps, inaccuracies, omissions, and opportunities for improvement.
• Provide detailed written feedback to improve AI model performance and clinical reliability.

### Collaborate with Clinical Teams

• Participate in onboarding sessions and specialty calibration meetings.
• Work with multidisciplinary teams to ensure consistency, accuracy, and quality across medical evaluations.

### Required Qualifications

Applicants should meet one of the following criteria

### Attending Physicians

• Board-certified in Internal Medicine.
• Current, active, and unrestricted medical license.

### Resident Physicians

• Final-year Internal Medicine resident.
• Recent residency graduates must be board-eligible.

### Fellows

• Board-certified or board-eligible in Internal Medicine or a related subspecialty.
• Current, active, and unrestricted medical license.

Professionals with expertise in Internal Medicine subspecialties are encouraged to apply, including

• Cardiology
• Hematology & Oncology
• Endocrinology
• Nephrology
• Gastroenterology
• Pulmonology
• Other Internal Medicine subspecialties

### Preferred Qualifications

• Strong knowledge of evidence-based medicine and clinical practice guidelines.
• Experience in clinical teaching, medical research, or quality improvement initiatives.
• Excellent written communication and analytical skills.
• Interest in artificial intelligence, clinical decision support, or healthcare technology.

### Engagement Details

• Fully remote with 100% asynchronous work.
• Standard commitment of approximately 20 hours per week, with opportunities for additional hours based on project demand.
• Flexible working schedule.
• Weekly payments.
• Rolling onboarding with ongoing engagement opportunities.
• Compensation is determined by clinical experience, expertise, and seniority.

### Why Join

• Help shape the future of AI-powered clinical decision-making.
• Apply your Internal Medicine expertise to improve advanced healthcare AI systems.
• Collaborate with leading AI researchers and experienced clinicians.
• Contribute to high-impact projects that advance safe, accurate, and evidence-based medical AI.

### Equal Opportunity Statement
All qualified applicants will be considered without regard to legally protected characteristics. Reasonable accommodations are available upon request.
### Contract Information

• Independent contractor engagement.
• Fully remote work completed on your own schedule.
• Projects may be extended, modified, or concluded based on business needs and individual performance.
• Work does not involve access to confidential or proprietary information from any employer, client, or institution.
• Weekly payments are processed through supported payment platforms based on approved work completed.
• Please note that visa sponsorship is not available for this opportunity.

## Apply
[Apply at Weekday AI](https://apply.workable.com/weekday-1/j/E42A64D63A/apply)
---
Powered by [Workable](https://www.workable.com)