GenAI Platform Engineering

Design platform architecture with self-service portal, service catalog, and golden paths.

Implement branching strategies, code review workflows, and merge policies for AI projects.

Curate, version, and maintain evaluation datasets for hosted LLMs.

Implement Technical Requirements for EU AI Act Risk Categories with Current Enforcement Timeline

Build Pydantic models that represent platform service catalog entries including service metadata, configuration schemas, dependency declarations, and tier classifications for the AI developer platform.

Create a template registry that stores and validates golden path templates, including template versioning, step ordering, and parameter validation for AI workflow automation.

Build a catalog entry validator that uses JSON Schema to validate service configuration parameters, providing detailed error messages for invalid entries and supporting schema evolution.

Build cursor-based pagination for the service catalog API supporting configurable page sizes and stable traversal of large catalogs.

Implement text search across service catalog entries with relevance ranking, prefix matching for autocomplete, and combined search with structured filters.

Implement ETag-based conditional requests for the service catalog, supporting cache validation with If-None-Match and optimistic concurrency control with If-Match headers.

Add Prometheus counter and histogram instrumentation to platform API endpoints, tracking request counts, error rates, and latency distributions.

Create histogram-based latency tracking for provisioning operations with configurable bucket boundaries, percentile computation, and alert threshold evaluation.

Build Grafana dashboard configurations as code, defining panels for request rates, error rates, latency percentiles, and active service counts with PromQL queries.

Build a Python tool that initializes a Git repository with trunk-based development conventions. Create a standard AI project directory structure (prompts, model-configs, eval-datasets, k8s), configure Git settings for rebase-by-default and autostash, deterministically classify candidate branches aga

Build a Python validator that enforces feature branch naming conventions for AI projects. Validate branch names against patterns like feature/, fix/, config/, infra/, deploy/, eval/. Parse branch names to extract type and description components. Report invalid branches with helpful error messages su

Build a Python tool that manages the complete lifecycle of a feature branch: creation from main, commit tracking, squash merge simulation, and cleanup. Track branch metadata including age, commit count, and files changed. Detect stale branches that exceed the maximum age threshold.

Build a Python tool that deterministically generates and audits pre-commit hook configurations for AI projects. Render .pre-commit-config.yaml with hooks for ruff, mypy, yamllint, detect-secrets, and JSON validation, validate prompt template files against a Pydantic schema, and audit a rendered conf

Build a Python tool that generates Renovate configuration and Kubernetes CronJob manifests for automated dependency management. Create renovate.json with AI-specific package grouping rules. Generate the Kubernetes YAML for deploying Renovate as a GKE CronJob.

Build a Python tool that compares Renovate and Dependabot across multiple dimensions for AI project dependency management. Generate comparison matrices, score each tool on relevant criteria, and produce a recommendation based on project requirements.

Define a Pydantic-based schema for evaluation test cases, including enums for task categories and difficulty levels, and build a schema validator that loads JSONL datasets and reports validation errors with row-level detail.

Build a TestCaseGenerator that creates evaluation test cases spanning five task categories and three difficulty levels using template-based patterns, then analyze the resulting dataset for stratification balance and coverage gaps.

Build a DatasetBuilder class that loads JSONL evaluation datasets, validates entries with Pydantic, computes coverage statistics including category counts, difficulty distribution, and average token counts, then generates a CoverageReporter that formats results as tables and identifies balance issue

Build a ContaminationDetector class that analyzes recorded probe transcripts - the expected answer for each evaluation test case paired with the text a model returned for it - and decides deterministically which cases look contaminated. You implement text normalization, n-gram overlap scoring, per-p

Implement ROUGE-L scoring from scratch using the longest common subsequence algorithm to measure text overlap between model outputs and expected answers, then build a batch scorer that computes contamination rates across an evaluation dataset.

Build a QuarantineWorkflow that takes per-(case, model) contamination signals, isolates burned evaluation test cases with evidence records, and generates an aggregate contamination report with per-model and per-category rates - all with deterministic grouping, filtering, and arithmetic that runs ful

Build a ComplianceTracker over the four EU AI Act enforcement milestones (Feb 2025, Aug 2025, Feb 2026, Aug 2026). Compute the signed days remaining to each deadline relative to an explicit reference date, classify every milestone as complete, overdue, due-soon (within a configurable window), or upc

Build an AutomatedComplianceChecker that registers a verification predicate for each compliance requirement, runs one or all checks against a system-state snapshot, treats a predicate error as a failed check rather than a crash, and aggregates the pass/fail results into a compliance summary with a c

Build a GapReportGenerator that identifies non-compliant and partially compliant requirements, prioritizes them by combining enforcement deadline proximity with risk severity into a single priority score, generates deterministic remediation recommendations, and produces a formatted, ranked gap analy