Work
Projects
Autonomous RAG over documents, SQL, and logs with a built-in self-verification loop. A LangGraph cyclic graph (Researcher → Analyst → Auditor) where the Auditor returns a structured boolean and loops back for re-retrieval whenever an answer isn't faithful — with dual brakes: a business-rule max-revisions limit and an infra recursion limit. Co-lead — I own the agent graph, RAG core, security, and observability layers; built with a partner, not solo.
agent-graph · co-lead
Multi-agent self-verification graph
LangGraph Researcher/Analyst/Auditor loop; the Auditor uses structured output (a real bool, not parsed prose). Message history accumulates while retrieval results are overwritten, so every revision re-runs against fresh evidence.
rag-core
Hybrid retrieval + rerank
Qdrant dense + sparse (RRF) top-20 → cross-encoder rerank → top-5; a heading-aware chunker that carries a breadcrumb (A > B > C) on every chunk, plus a separate row-oriented serializer for tables.
security
MCP read-only SQL boundary
A DB-level sentinel_ro role that physically cannot write, plus an application-level ensure_read_only() guard: comment stripping, literal blanking, single-statement SELECT/WITH only, and a denied-keyword set.
product · security
Product surface & security
JWT RS256 auth, a server-injected tenant_id (the model can never choose its own tenant), typed citation DTOs, a /chat endpoint with thread continuity, and a Streamlit operator console.
observability
Observability & CI
Fixed a real bug where instrumentation ran before the tracer provider existed, so spans never reached Phoenix; CI publishes to GHCR with Trivy scanning, and a Ragas faithfulness score below 0.85 fails the build.
My contribution — agent graph, retrieval, and safety
As co-lead I own the agent graph, the RAG core, and the security and observability layers. This was built with a partner, not solo — my focus was the self-verification loop, hybrid retrieval, the read-only SQL boundary, and making the whole runtime traceable and faithful.
LangGraph
Qdrant
Cross-encoder rerank
MCP
FastAPI
JWT RS256
Ragas
Phoenix / OTel
GHCR / Trivy
Streamlit
An end-to-end agent that extracts a master profile from your CVs, scrapes and ranks job postings by a fit score, generates a tailored CV and cover letter per posting through a cv-tailor subagent, and sends applications only with human approval. Built on deepagents / LangGraph with interrupts. Solo — every commit is mine; published to PyPI with 11 GitHub stars.
agent · lead
Full use of deepagents primitives
Subagents, a FilesystemBackend, interrupt_on plus a checkpointer for human-in-the-loop approval, and planning — the agent leans on the framework's primitives end to end rather than reimplementing them.
resilience
Graceful degradation
Job sourcing falls back JSearch API → JobSpy scraper → bundled sample postings so it never hard-fails; the model provider chain degrades Anthropic → OpenRouter → OpenAI the same way.
memory
Persistent career memory
A career memory and application log persist across sessions so the agent never re-suggests a job it already proposed; per-call token and cost tracking is written to usage.jsonl.
safety
Anti-hallucination rule
Generated CVs and cover letters may only use facts drawn from the extracted profile — nothing is invented about the candidate.
release
Shipped to PyPI
Published to PyPI via GitHub Actions trusted publishing; 11 GitHub stars reflect real, unsolicited user interest.
My contribution — solo, shipped, and honest
I built and shipped this solo: a deepagents/LangGraph agent that goes from raw CVs to approved applications with human-in-the-loop interrupts, graceful provider and data-source fallbacks, and a strict anti-hallucination rule. Published to PyPI and earning 11 GitHub stars.
deepagents
LangGraph
Anthropic
PyPI
JSearch / JobSpy
Checkpointer
Human-in-the-loop
Cost tracking
A lightweight LLM-as-judge evaluation tool. It scores question/answer datasets across five dimensions — faithfulness, relevance, clarity, safety, and consistency — with a judge model, and compares two runs to show before/after deltas. Works with OpenAI, OpenRouter, or a local Ollama model. Solo — published to PyPI as openeval-llm (import and command name openeval).
cli · lead
run / compare CLI
A JSONL dataset goes in and a scored report comes out; compare prints a before/after delta table (e.g. faithfulness 0.71 → 0.88) so quality changes are visible at a glance.
judge
Crash-proof judge
Extracts JSON even when the model wraps it in fences or preamble, falls back to neutral defaults on missing or malformed dimensions, and retries transient errors — one bad answer can't take down a whole run.
reporting
Report provenance & cost
Every report records the judge model, dataset, and timestamp; local Ollama models are billed at $0 so cost accounting stays honest.
connectors
Pluggable connectors
The judge is decoupled from the subject under test, so you can grade a weak system with a stronger judge, and swap providers in a single line.
release
Shipped to PyPI
PyPI trusted publishing, 12 tests, and English docs (docs/ARCHITECTURE.md).
My contribution — a robust, shippable eval tool
I designed and shipped OpenEval solo: a provider-agnostic LLM-as-judge CLI with a crash-proof judge, run-to-run delta comparison, honest cost accounting, and 12 tests. Published to PyPI as openeval-llm.
Python
OpenAI
OpenRouter
Ollama
LLM-as-judge
JSONL
PyPI
pytest
An end-to-end personal proof of an enterprise analytics architecture: mock bank data → PostgreSQL FDW → dbt (staging / intermediate / marts / ai_layer) → a grounded AI insight engine → Superset. The AI never sees raw rows — dbt pre-aggregates each user into a one-row mart_ai_context, the single contract between the pipeline and the AI. Solo — built on a feature-branch + PR workflow.
ai-layer · lead
Grounding architecture
A mart_ai_context pre-aggregation feeds versioned Jinja2 prompt templates; Claude's structured output is validated with Pydantic, so every insight is reproducible and auditable.
dbt
Layered dbt pipeline
staging → intermediate → marts → ai_layer on deterministic, seeded synthetic data, with dbt tests wired in as a CI gate.
observability
Full observability
OpenTelemetry exports to Tempo / Prometheus / Grafana, so a failed nightly run is a traced event rather than a silent gap.
orchestration
Two AI trigger modes
Insights run after each dbt build and on a schedule for anomaly and recommendation passes.
engineering
Engineering discipline
A feature-branch + PR workflow even when working solo, with a README written as an interview-style Q&A.
My contribution — grounded AI on a real pipeline
I built this solo to prove an enterprise pattern end to end: the AI never touches raw rows — it reads a single pre-aggregated mart_ai_context contract, runs on versioned prompt templates with Pydantic-validated output, and is fully traced with OpenTelemetry.
dbt
PostgreSQL
FastAPI
Claude API
Apache Superset
OpenTelemetry
Tempo / Prometheus / Grafana
Pydantic
Jinja2
End-to-end analytics platform for talent management — ingesting psychometric assessment data (AON cognitive, Saville behavioural, Assessment Center) via PostgreSQL FDW, transforming it through a 3-layer dbt pipeline, and serving embedded Superset dashboards with per-tenant, per-user Row-Level Security. Primary owner of the dbt transformation layer (~109 models, 56 marts, 13 AI precompute tables).
ga-dbt · lead
AI analytics layer (13 marts)
Designed and built the full AI precompute layer: SWOT classification, 9-box matrix, risk shadow effect, Big Five / 4H aggregates, and EQ / sales / leadership impact-risk scores — migrating Analyzer features from in-memory Python to materialized SQL marts.
ga-dbt
Dashboard mart pipeline
Built behavioural & risk scoring models, sales-potential and leadership-impact marts, employee score facts, and canonical seed refactors across staging → intermediate → marts for 8+ embedded dashboards.
ga-dbt
Native RLS in dbt
Implemented PostgreSQL native RLS via the apply_native_rls post-hook macro, access_map_employee visibility table, and cross-tenant smoke tests — fixing RLS dependency issues that blocked the entire mart layer.
ga-workspace
Local analytics dev stack
Set up local DB provisioning, dbt quick-start scripts, schema-comparison tooling, and mock-source integration so the full FDW → dbt → Superset loop runs with a single docker compose up.
ga-analytics-api
Multi-tenant API & UAT deploy
Led the multi-tenancy refactor, UAT analytics-api configuration & Dockerfile, tenant-level GitLab CI security tests, and RLS alignment between the API and dbt layers.
ga-superset · ops
Production unblock & TR localisation
Fixed schema drift (candidate_user) blocking 130+ downstream models before a live demo; stabilised Superset Docker startup; updated Turkish UI translations and filter localisation (tr-TR).
Core ownership — dbt transformation layer
Primary engineer on the dbt pipeline (~109 models): from early local-dev setup and canonical score normalisation (AON, Saville, AC → STEN) through the full mart catalogue and AI precompute layer. Segment discovery and feature engineering started in early behavioral-clustering R&D (Aug 2025) before productionising scores as SQL marts. Platform scaffolding (K8s, CI, guest-token API) was built in parallel by the team; my focus was the data models, RLS, and making dashboards actually work end-to-end.
dbt Core 1.11
PostgreSQL 16
Apache Superset 6
FastAPI
Kubernetes
Argo Workflows
Redis / Celery
FDW
Native RLS
JWT
B2B corporate AI coaching platform — multi-agent orchestration (Companion, Coach, Advisor, Mentor) with LangGraph, modular POML prompts, Parquet-grounded personalization, real-time voice via OpenAI Realtime API, and HR analytics dashboards. Built the evaluation framework, early voice stack, and POML prompt foundation.
prompts · lead
Pattern system & POML migration
Implemented the advanced pattern engine (Chain-of-Thought, Tree-of-Thoughts, Planning, and 4 more) with mode-based selection; migrated all prompt files from Turkish to English and fixed naming/path inconsistencies across the 49-file POML library.
coach · lead
Referee Agent — LLM-as-Judge
Designed and built the end-to-end evaluation system: AI Client Simulator, multi-turn Coach integration, unified 5-dimension evaluator (role/style, techniques, principles, safety, flow), dynamic conversation closure, and OpenAI Evals export.
coach
Voice integration (OpenAI Realtime)
Pioneered real-time voice coaching: WebSocket voice server, RealtimeVoiceSession bridge, POML system prompts in voice path, voice_config.py, and agent-chat-ui React components with Web Audio API (24 kHz PCM16).
coach
Memory & state persistence
Fixed coach memory persistence (Context Messages: 0 bug), LangChain v0.3+ compatibility in coach submodule, and client-side memory optimization (last 6 messages, 40% token savings).
coach
Client simulator & eval tooling
Improved AI client simulator with turn-based progression guides and anti-repetition rules (49% fewer turns, score 7.88→8.2); added coaching evaluation reports and single-scenario test scripts.
coach · docs
Contributor onboarding
Wrote comprehensive setup guide for new contributors (submodule init, Python 3.13+, dependency fixes) and expanded Referee Agent README with scenarios, token costs, and result JSON examples.
Core ownership — evaluation & voice layer
Primary engineer on coach quality assurance and early product surfaces: built the Referee Agent to objectively score coaching sessions before production rollout, and shipped the first OpenAI Realtime voice path with POML-aware prompts. Analytics, company-wide chat, and performance-data grounding were developed in parallel by the team; my focus was making the coach testable, measurable, and voice-ready.
LangGraph
FastAPI
POML
OpenAI Realtime
Langfuse
Streamlit
Parquet / pandas
WebSocket
UV workspace
Corporate behavioral analytics platform — a LangChain ReAct agent that answers natural-language HR questions over psychometric data (Saville, 360°, 9-box, competency profiles) via dbt mart tables with native PostgreSQL RLS. Dual-service architecture: Analyzer (analysis engine) + Analyzer API (voice/text conversation layer). Lead engineer on security architecture, config bootstrap, and service integration.
analyzer · lead
4-layer security & RLS pipeline
Shipped the full RLS-security branch: JWT auth on HTTP + WebSocket, UserContext via ContextVar, execute_rls on all SQL paths, InputGuard/OutputGuard with YAML-driven threat patterns, cross-tenant isolation guard, and QueryValidationPipeline before every LLM call.
analyzer
Config bootstrap & schema layer
Introduced backend/core/bootstrap/ (ConfigPathResolver, agent_config, mod_loaders); consolidated application/schema; moved position_models and security types; eliminated duplicate config packages and hardcoded prompt strings.
analyzer
Master prompt & agent reliability
Fixed master_prompt.yaml rendering (dict sections → block strings), build_master_prompt() guards, PostgreSQL TCP keepalive + stale-connection retry, SQL alias ambiguity fixes, and chat history trimming for context overflow.
analyzer
Domain mods & analysis fixes
Fixed FourHProfileRiskMod runtime TypeError, resolved tools/__init__ circular import via PEP 562 lazy loading, unified SwotPatternType enums, and enhanced ProfileTransformer for employee_user_id identity validation.
ga-analyzer-api
Voice layer integration
Langfuse trace propagation across analyzer client and chat API caller; fail-fast startup validation for analyzer service URL; global 500 error logging; improved employee ID extraction from nested survey payloads.
analyzer · observability
Langfuse tracing
Integrated Langfuse callbacks in AnalizciAgent and WebSocket endpoints for end-to-end LLM trace propagation; docker-compose config and utility module for observability across the analysis pipeline.
Core ownership — security & platform foundation
Primary engineer on the security and configuration foundation that made production deployment possible: merged JWT + RLS + guard pipeline into main, built the bootstrap config system agents run on, and wired observability across both analyzer engine and conversation API. ReAct agent skills, company analytics chat, and enum refactors were developed in parallel by the team; my focus was tenant-safe data access, prompt architecture, and making the dual-service stack production-ready.
LangChain
LangGraph
FastAPI
PostgreSQL RLS
OpenAI Realtime
Langfuse
POML / YAML prompts
JWT auth
WebSocket streaming
Two production B2B AI microservices behind the corporate agent gateway. The KPI Advisor recommends SMART-aligned KPIs for any position, grounded in the APQC Process Classification Framework (lexical + category-prior retrieval, no vector DB) and validated by an offline LLM-as-judge pipeline. The Skills Advisor selects exactly 12 competency dimensions from a closed dictionary for any job title, with Turkish rationales and a DSPy 5-expert ensemble. Primary engineer on both — generation engines, retrieval, the eval flywheel, caching, and production integration.
kpi-advisor · lead
KPI generation service & APQC retrieval
Built the core FastAPI service: OpenAI JSON-mode structured output, hybrid lexical + category-prior retrieval over 1631 PCF nodes and 2680 metrics, diversity selection, confidence-aware fallback, and prompt assembly with a domain-expert persona and corporate rules.
kpi-advisor
Offline quality flywheel
Implemented the step2 LLM-as-judge eval (SMART ×0.35 + Relevance ×0.40 + Clarity ×0.25), step3 Excel export for human review, and step4 feedback import into the golden dataset — closing the synthetic-data → eval → human flywheel.
skills-advisor · lead
Founding competency assessment system
Built the founding AI competency-suggestion flow: Ocean/Saville test types, dimension scoring with Turkish rationales, and a Streamlit two-column UI — establishing the product before the modular FastAPI refactor.
skills-advisor
Closed-dictionary constrained generation
Moved dimensions and mappings into a ~900-line dimensions.json with /dimensions endpoints; per-request shuffle to reduce positional bias; whitelist validation, dedupe, and a hard cap of 12 on every response.
skills-advisor
DSPy SME expert ensemble
A dynamic SME weight system with an arithmetic-mean adjudicator: five parallel expert personas merge into a consensus list with category balance — an alternative to single-pass LangChain for higher-stakes roles.
both · infra
Caching & production integration
SHA-256 LLM-level and request-level caches so identical requests skip the model entirely; JWT + tenant whitelist via grid-auth; and gateway invoke wiring (/api/v1/agents/…/invoke) for downstream HR workflows.
My contribution — two recommendation engines, end to end
Primary engineer on both microservices, from the founding competency UI and the APQC-grounded KPI generator through DSPy consensus, embedding-free retrieval, multi-layer caching, and the offline eval flywheel. Observability, UI polish, and O*NET / LangChain-v1 migrations were developed in parallel by the team; my focus was structured-output quality, constrained generation, and shipping both behind one gateway contract.
FastAPI
OpenAI JSON mode
Pydantic v2
DSPy
LangChain
Redis
APQC PCF
LLM-as-judge
Few-shot RAG
grid-auth
Corporate AI agent gateway — a FastAPI monorepo that exposes multiple downstream AI microservices through one standard invoke API: shared JWT auth, request/response envelope, structured logging, and OpenTelemetry hooks. Teams ship thin agent handlers + httpx proxies; domain logic stays in separate services (KPI Advisor, Analyzer, etc.). Primary engineer on platform bootstrap, auth integration, Docker topology, dynamic CI, and the Cookiecutter-based agent scaffold.
ai-api-monorepo · lead
Platform bootstrap & gateway core
Created the initial monorepo structure: core router factory, InvokeRequest/InvokeResponse schemas, agent registration in main.py, a Cookiecutter template for new agents, and an API.md integration guide so teams could add services with the same contract and deployment pattern.
core
JWT auth & invoke pipeline
Integrated grid-auth for token verification on every /invoke call; dev bypass vs production auth paths; streamlined API responses (operational metadata logged, not returned); enhanced agent invoke logging and error handling.
infra
Per-agent Docker Compose topology
Restructured compose to per-agent includes (agents/ga_kpi_advisor/docker-compose.yml); dev overlay with external KPI Advisor path; multi-stage Dockerfiles with non-root user, OCI labels, HEALTHCHECK, and build-arg versioning.
ci
Dynamic GitLab child pipelines
Built generate_agents_pipeline.py to scan agents/ for Dockerfiles and emit per-agent CI child pipelines; added GitLab CI + pre-commit (Ruff, MyPy); iterated trigger-agents, Docker host, and job dependency rules for reliable builds.
kpi-advisor agent
First production agent integration
Wired ga-kpi-advisor as the reference agent: httpx proxy to downstream KPI service, env-based URL config, validation error logging, and end-to-end invoke flow from Streamlit through the gateway.
kpi-advisor agent
Cross-service contract alignment
Kept gateway payloads in sync with KPI Advisor schema evolution: leadershipLevel, positionGroup, name/definition fields, optional id — ensuring monorepo invoke examples match the downstream Pydantic models.
Core ownership — agent gateway platform
Primary engineer from initial commit through production-ready gateway: monorepo layout, first agent wiring, JWT auth, per-agent Docker/compose pattern, dynamic CI pipeline generation, and the Cookiecutter scaffold for new agents. create_gateway_app refactor, CORS settings, and scaffolding improvements were developed in parallel by the team; grid-observability adoption was led by another engineer. My focus was the platform contract, deployment topology, and making new agents plug in with minimal boilerplate.
FastAPI
httpx
Pydantic v2
Cookiecutter
JWT / grid-auth
OpenTelemetry
Docker Compose
GitLab CI
Gateway pattern
bal. acc.
comparison metric
My MSc thesis. I built a Random Forest classifier that predicts which payment instrument a consumer will choose, and optimized its hyperparameters with three swarm-intelligence algorithms — PSO, ABC, and a Hybrid ABC-PSO — comparing them on balanced accuracy. Solo, end to end: data preparation, modeling, hyperparameter optimization, and evaluation.
optimization
Hybrid ABC-PSO hyperparameter search
Combined Particle Swarm Optimization (PSO) and Artificial Bee Colony (ABC) into a Hybrid ABC-PSO to tune the Random Forest's hyperparameters, and compared it against PSO and ABC on their own.
modeling
Random Forest classifier
A Random Forest that predicts which payment instrument a consumer will choose from consumer payment-behaviour data.
evaluation
Comparison on balanced accuracy
Evaluated PSO vs ABC vs Hybrid ABC-PSO on balanced accuracy to test whether the hybrid actually improves on single-algorithm hyperparameter tuning.
My contribution — the full research pipeline
Sole author, end to end: data preparation, the Random Forest model, hyperparameter optimization with a Hybrid ABC-PSO, and an honest comparison on balanced accuracy. Supervised by Dr. Elif Deniz Yelmenoğlu.
Python
scikit-learn
NiaPy
Random Forest
PSO
ABC
pandas
