PhD, Industrial & Systems Engineering
University of Washington
AI research · Real systems · Real impact
Hi, I am Zhanlin Liu
From multimodal retrieval to agentic automation, I turn ambitious questions into real-world systems used at scale.
What problem should AI solve next?
Click a topic to see the problems I work on.
Four research worlds, one connected vision
From understanding to action, with evaluation at the core.
My work spans four connected areas, from multimodal understanding to agentic systems, all grounded in real-world applications and evaluation.
Explore all projectsCareer journey
A path from research to real-world impact.
Working across startups and tech companies to bring AI from research to products at scale.
2019
Wyze
Audio AI
2021
Warner Music
Music AI
2022
Microsoft
LLM & protein models
2023
AstrumU
Skill AI
Now
ProRata
Agentic AI
-
Publications
20+ papers
NLP, audio intelligence, multimodal learning and ML systems
-
Patents
Four filed, one granted
Document intelligence, multimodal attribution and content discovery
-
Awards
Hackathon and competition placings
Databricks Data+AI Summit and the OpenAtom Foundation
What would you like to know?
Ask me anything about my work, research, or experience.
or
Get in touchMultimodal Retrieval & RAG
Retrieval quality starts with content quality and understanding.
Agentic Automation
The goal of AI is to create simplicity and automation — not to expose more AI to the user.
System Evaluation
Complex AI systems need evaluation beyond either humans or LLM judges alone.
Other Intelligences
Deep models over audio, protein sequences and user graphs — turning raw signals into predictions.
// Experience
Experience
Teams and problems I’ve worked on — from internships through research to production AI.
View full résuméEducation
MSc, Statistics
University of Washington
BS, Statistics — with Distinction
University of Iowa
// Projects
Hard problems, and what I built to solve them
Grouped by the kind of problem. Pick a question to see the system behind it.
1 / Pick a problem
Multimodal Retrieval & RAG
Agentic Automation
System Evaluation
Other Intelligences
2 / How I solved it
-
ProRata.ai
Knowledge Graph / OntoKG
There are thousands of Donald Trumps. Are you referring to the president?
Design the taxonomy first, then extraction pipelines that scale to all of them.
ProRata.ai
Knowledge Graph / OntoKG
There are thousands of Donald Trumps. Are you referring to the president?
Knowledge Systems
Design the taxonomy first, then extraction pipelines that scale to all of them.
Entity extraction, linking and property-graph construction over publisher content, grounded against a canonical entity index.
- Webpage HTML to markdownRaw pages converted to clean, structured textHTMLMarkdownWeb pages, articles, blogs and documentation, made readable for an LLM.
- Schema-Guided LLM ExtractionExtract entities and relations with schema guidanceEntity schematype · subtypeRelation schematopic · subtopicExtraction EvaluationMeasure extraction quality
- Entity LinkingLink extracted entities to a canonical indexOpenAIOpen AIOpenAI Inc.OpenAICanonical entityLinking EvaluationMeasure linking accuracy
- Entity Index ExpansionUpdate and expand the entity indexAdd new entitiesMerge duplicatesEnrich with attributesGrow over time
- Knowledge Graph ConstructionEntities become nodes. Relation topics become edge families with specific relations under each topic.Entitydeveloped_bycreated_bybuilt_byowned_by
- ApplicationsTurn connected knowledge into real valueGist GEOExplore entities, relations, and insights across your knowledge.Gist AnswerGet grounded answers powered by your knowledge graph.
Key capabilities
- Schema-guided extractionExtract consistent, high-quality entities and relations.
- Entity linkingConnect mentions to a unified entity index.
- Question answeringTurn your knowledge graph into reliable, grounded answers.
Tech stack
- Python
- LLM / OpenAI
- Graph database
- Information extraction
- Entity linking
- Evaluation
Impact
Powers Gist GEO and Gist Answer
- ~34Mcanonical entities
- 100M+entities & mentions
-
ProRata.ai
Discovery Questions
What if I already knew what you’d ask next?
Generate the questions from the content, and retrieve them at scale.
ProRata.ai
Discovery Questions
What if I already knew what you’d ask next?
User Understanding
Generate the questions from the content, and retrieve them at scale.
LLM-generated contextual questions from publisher content, ranked and served in the Gist Widget for search and discovery.
- Publisher contentArticles across the 30+ publishers the Gist Widget runs on
Question generationLLM, per article
- Read the articlepublisher content, page by page
- Generate contextual questionswhat a reader would ask next
- Quality gateevaluation datasets and metrics name the failure modes
- Questions for the pagethe ones worth asking about that article
Search and discovery20M+ monthly widget views
- Gist Widget on the pagelive across 30+ publishers
- A reader picks oneapproximately 0.5% engagement
- Gist search and discovery65K+ monthly Gist queries
- Iterative quality improvementLLM evaluation, human-in-the-loop validation and online CTR feedback improve the questions the next generation produces
- 20M+monthly widget views
- 30+publishers
- 65K+monthly Gist queries
-
ProRata.ai
Multimodal & Music Attribution
How should Suno pay UMG and music artists?
Understanding and matching across text, tables, images, audio and video.
- Music Business Worldwide — Universal Music × ProRata
- Interactive demo coming soon.
ProRata.ai
Multimodal & Music Attribution
How should Suno pay UMG and music artists?
Creative AI
Understanding and matching across text, tables, images, audio and video.
Retrieval-based music and multimodal attribution, matching generative AI output to reference catalogs across text, images, audio and video.
- Generative outputWhat a generative-AI system producedtextimagesaudiovideo
- Multimodal understandingEach modality understood so that it can be retrieved and matched
- Retrieval against reference catalogsCandidates retrieved at scaleMusic-industry reference catalogsthe references music is attributed to
- Matching and comparisonThe output compared against the candidates, at scale
- Source and provenance attributionThe reference music or source content behind the output, identified and attributedSubmitted patenton multimodal attribution
- In productPublisher-facing and search-oriented AI applicationsLower latency and costarchitecture optimised alongside retrieval quality
- Multi-viewretrieval
- Patentsubmitted
-
ProRata.ai
PACE
Your RAG system just quoted the cookie banner. Now what?
An agent learns each publisher’s layout once, then extracts with zero LLM cost.
ProRata.ai
PACE — Publisher-Adaptive Content Extractor
Your RAG system just quoted the cookie banner. Now what?
Web Intelligence
An agent learns each publisher’s layout once, then extracts with zero LLM cost.
Publisher-adaptive content extraction that learns site-specific patterns and produces clean multimodal markdown at zero inference cost.
- Agentic orchestratorUnderstands the requirements · plans and adapts the workflow · monitors, evaluates and refines
Adaptive trainingLLM-guided, once per publisher
- Reference pagesrepresentative of the publisher
- HTML preprocessingclean DOM, noise removed
- Prompt constructiondefault instructions + user requirements
- LLM analyzerreads each page’s DOM
- Publisher-level configurationpatterns and rules, aggregated
- Human-in-the-loop validationverifies, approves or refines the labels
- Instantiate the extractoras Python or JavaScript
- Configured extractor templatetraversal · fallback · deduplication · formatting
Scalable inferenceno LLM calls
- New or held-out pagesfrom the same publisher
- Run the extractordeterministic, low latency
- Flexible outputsplain text · markdown · JSON · multimodal
- Evaluation and quality checkagainst validated truth on held-out pages
- Evaluation-driven refinementfeedback adapts the prompts, the configuration and the workflow
- 1,500+publishers
- 0-costinference
-
ProRata.ai
EPOCH
Are you still tuning prompts by hand?
One evaluation-driven loop over prompts, rules, fine-tuning and code.
- GitHub repository — EPOCH
- arXiv 2603.09049 — EPOCH: An Agentic Protocol for Multi-Round System Optimization
ProRata.ai
EPOCH
Are you still tuning prompts by hand?
AI Evaluation
One evaluation-driven loop over prompts, rules, fine-tuning and code.
An agentic optimization platform that rewrites prompts, rules, configs and code, runs the evaluations, and keeps only what improves the metric.
- BaselineMeasure the task as it standsTask-specific evaluationthe metric the loop optimises
- Propose a changeModify prompts, rules, configuration or codepromptsrulesconfigurationcode
- EvaluateRun the evaluation against the baselineAutomated experimentno manual prompt iteration
- Keep what winsRetain only changes that improve the target metricQuery understanding
The retained change becomes the next baseline.
- Multi-roundoptimization loop
- Papersubmitted to COLM 2026
-
ProRata.ai
Gist Answer + LLM Eval
Who checks the AI that’s checking the AI?
Score it against human-feedback benchmarks, not just another model.
ProRata.ai
Gist Answer + LLM Eval
Who checks the AI that’s checking the AI?
NLP Evaluation
Score it against human-feedback benchmarks, not just another model.
RAG answer generation with LLM-as-a-Judge and human-in-the-loop evaluation, validating automated scores against human judgement.
- GenerateThe system produces an output for the task under evaluationtext denoisingchunkingquestion generationanswer generation
- LLM-as-a-JudgeAn automated evaluator scores itAutomated scorethe metric under test, not the ground truth
- Human annotationThe same output goes in front of peopleBinary preferencethis one or that oneMultiple choicewhich of these is right
- Agreement analysisHuman agreement, disagreement and confidence, against what the judge said
- Recalibrate the judgeRobustness, calibration and reliability, improved from the human signal
The recalibrated judge scores the next round.
- 4task families evaluated
- Human-validatedjudge calibration
-
AstrumU
SkillLLM & Skill Taxonomy
Your résumé says PyTorch. The job asks for deep learning. Match?
Fine-tuned skill extraction on top of a comprehensive skill taxonomy.
- arXiv 2410.12052 — Skill-LLM: Repurposing General-Purpose LLMs for Skill Extraction
- Press release — ETS, JFFLabs & AstrumU partnership
- AstrumU — Enterprise Organizations overview (PDF)
AstrumU
SkillLLM & Skill Taxonomy
Your résumé says PyTorch. The job asks for deep learning. Match?
Career Intelligence
Fine-tuned skill extraction on top of a comprehensive skill taxonomy.
Fine-tuned LLMs for contextual skill extraction, the skill taxonomy and knowledge graph beneath them, and the matching that turns both into recommendations.
- AI team and platformScaled from 2 to 7+ · annotation, training, evaluation, deployment and production inference, owned end to end
ExtractionSkillLLM, fine-tuned
- Job descriptions and profilesmillions of postings and profiles
- Contextual skill extractionLLMs fine-tuned for skills in context
- Normalized skill representationsunstructured text, made structured
- Skill taxonomy and knowledge graphtechnical and soft skills · normalized · deduplicated · linked · expanded
Matching and recommendationretrieval and embeddings
- Normalize and disambiguateone skill, however it was written
- Semantic matchingretrieval- and embedding-based
- In productionjob recommendation, skill-gap analysis, career intelligence, workforce analytics
- Continuous taxonomy expansionnewly extracted skills and the semantic relationships between them expand the taxonomy, which changes what the next extraction resolves to
- Millionsof postings & profiles
- 2 to 7+AI team scaled
-
Microsoft Research
Science Engine / Antigen LM
A language model that reads proteins, not prose.
A protein language model over 2B+ sequences, scoring how natural it looks.
Microsoft Research
Science Engine / Antigen LM
A language model that reads proteins, not prose.
Scientific AI
A protein language model over 2B+ sequences, scoring how natural it looks.
BERT-based protein language models over billions of antigen sequences, scoring how natural a sequence looks, plus scientific entity extraction.
- Antigen language modelBERT-based models on 2B+ multi-species sequences, trained across multiple GPUsPerplexity 1.7 → 1.4beats AntiBERTyNaturalness scoringfinds atypical antigen sequences
- Scientific entity extractionTransformer extraction over papers and patentsBenchmarkedagainst GPT-2 and GPT-3 few-shot
- GPT document intelligenceSummarisation, entity-relation extraction, question answering and document generationFine-tunedon European Medicines Agency data
- 2B+antigen sequences
- 1.7 → 1.4perplexity
-
Warner Music Group
Music Recognition & Music AI
Is there a new trend of music?
String matching plus learned embeddings — and genres never seen before.
Warner Music Group
Music Recognition & Music AI
Is there a new trend of music?
Music AI
String matching plus learned embeddings — and genres never seen before.
Music copyright identification and metadata reconciliation, plus open-set genre recognition and lyric recovery from historical recordings.
- Copyright and entity matchingLarge-scale copyright identification and metadata reconciliationDeterministic + fuzzy matchingstring matching over noisy metadataEntity embeddingsartist, recording and catalog, resolved
- Open-set music recognitionGenre classification and trend analysisOutside the taxonomyemerging and previously unseen genres
- Lyric recognitionWith Warner RecordsRecognition and restorationlyrics recovered from historical recordings
- Deterministic + learnedmatching
- Open-setgenre recognition
-
Wyze Labs
Audio Event AI
Is your baby crying when you are not in the room?
Deep audio detection in the cloud, plus efficient C++ on the device.
Wyze Labs
Audio Event AI
Is your baby crying when you are not in the room?
Edge AI
Deep audio detection in the cloud, plus efficient C++ on the device.
Sound-event annotation at scale, cloud audio classification for Wyze cameras, and an efficient on-device C++ detector beneath it.
- Sound-event datasetPublic audio and customer feedback · annotation guidelines, tutorials, quality metrics and evaluation processes
On the deviceefficient C++
- Camera audiocontinuous, on a Wyze camera
- On-device event detectionan efficient C++ algorithm
- Noise filtered out95%+ of irrelevant events removed in testing
- Events worth sendingall that leaves the device
In the cloudscalable services
- Deep audio classificationdeep-learning models for complex sound events
- Deployed as cloud serviceswith engineering teams, for Wyze cameras
- Baby cries · dog barks · gunshotsthe events a camera owner is told about
- Customer feedback returns to the datasetone of the two sources the annotated dataset is built from, so what the deployed system gets wrong becomes training data
- 95%+noise events filtered
- Cloud + edgedeployed
-
Alibaba Group
GCN Recommendation
How does Alibaba know what you’ll like before you do?
Treat it as a graph and let multi-view convolutions fill the gaps.
Alibaba Group
GCN Recommendation
How does Alibaba know what you’ll like before you do?
Recommender Systems
Treat it as a graph and let multi-view convolutions fill the gaps.
MV-GCN, a multi-view graph convolutional network for link prediction and recommendation, modelling interaction and content as separate graph views.
- Graph matrix completionRecommendation as link prediction over a user–item graph
Interaction viewobserved links
- User–item interactionswho interacted with what, as a graph
- Graph convolutionover the interaction view alone
- Cross-view consistencywhat one view learns about a node, the other has to agree with
Content viewcontent information
- Content as its own viewnot concatenated into the interaction features
- Graph convolutionover the content view alone
- Predicted linksthe two views complete the matrix together — evaluated on homogeneous and heterogeneous datasets, outperforming contemporary graph-based baselines
- 2graph views
- BeatsGCN baselines
// Blog
Blog
Notes on AI, engineering, and things I’m learning.
// Recognition
Recognition
Awards, patents, and publications that mark milestones in my work.
Award
Best Refereed Paper, Finalist
2019
For “Identifying the Influential Inputs for Network Output Variance Using Sparse Polynomial Chaos Expansion”
Granted
Cross-Document Intelligent Authoring And Processing, Including Format For Semantically-Annotated Documents
US20210081601A1
Submitted
Multimodal Attribution
Submitted
Systems and Methods for Enhancing Content Navigation and Discovery
Draft
Agentic Content Understanding for RAG and GEO
Rendered from data/publications.js — the complete record is also on
Google Scholar.
// Contact
Get in touch
Have a question, an opportunity, or just want to say hi? I’d love to hear from you.