Roles · LLM Engineer · Mid-level
5 core skills, 80 in total. Expectations per skill, and what changes at the next level.
This page lists what a Mid-level } is expected to know and do, skill by skill. Core skills are the ones a manager and peers assess in a review cycle; the rest count only in self-assessment. Main areas: Machine Learning & AI, AI-Assisted Development.
Grouped by area. The label on the right is the expected depth: Awareness, Working, Advanced or Expert.
Independently designs evaluation pipelines: custom benchmarks, domain-specific eval sets, human evaluation protocols. Compares models across multiple metrics for production decision-making.
Independently conducts LLM fine-tuning: LoRA/QLoRA, instruction dataset preparation, hyperparameter tuning. Monitors training via W&B, evaluates results on held-out datasets.
Independently administers vector databases in production: Pinecone, Weaviate, Qdrant. Configures indexes (HNSW, IVF), optimizes recall vs latency, manages collections and metadata.
Independently integrates ChatGPT and Claude API into production pipelines. Configures system prompts, function calling, and streaming responses. Compares models by quality and cost for specific tasks.
Independently develops production-ready prompts: structured output parsing, error recovery prompts, multi-turn dialog management. Applies systematic approach to prompt design and testing.
Not assessed by the team, but part of the self-assessment and the development plan.
80 skills get a higher expectation or become core when moving from Mid-level to Senior. The biggest jumps first.
} in the open competency matrix: 80 skills across 5 levels. The matrix is free for individuals and stays free.