Seltz tops Exa’s People Search benchmark

Exa built the best people search benchmark anyone has published, then open-sourced it. So we took it head to head against them.

We match their quality and return the results four times faster.

tldr;

• We tested Seltz on Exa's 1,400-query people search benchmark. We added per-query timing to our harness so latency gets measured alongside quality.
  
  

• The top result is a person who actually fits the query nine times out of ten (0.91). Almost every query turns up a correct person somewhere in the top ten (0.98). Across the full top ten, better than eight in ten profiles hold up (0.83). No one else on the benchmark clears this bar except Exa.
  
  

• We return results in 363ms at median, faster than everyone else we ran the benchmark against.

The results

A 0.01 spread at this sample size is noise. On the benchmark Exa built to demonstrate the product they spent a year training, we matched them, and we did it at one fourth the latency.

We used us-west-2 to run the evaluation. We were surprised that tail latency widens the gap between us and our competitors. Our p99 is 1.5x our p50. Exa is 1.8x, Parallel 1.9x, Perplexity 2.4x. An agent making six sequential profile lookups never experiences the median. It experiences the tail.

One note for anyone comparing this table to Exa’s own comparison pages, which report 75.5% rank-one accuracy on people. That figure comes from targeted lookup queries, which are not part of the released dataset. The open set is entirely role-based discovery, and it is scored differently.

How the People Benchmark works

Every one of the 1,400 queries composes a role, a seniority level, and a location, and some add an experience floor. They are phrased the way people type them rather than as clean structured filters: "policy manager based in sf at gaming", "10+ years ny senior staff engineers", "head of security at companies in berlin".

The set spans 280 role titles across 13 functions and 234 distinct locations, tested at five levels of geographic specificity from city up to region. Engineering is the largest function at 365 queries. Individual contributors are the largest seniority band at 480.

Discovery queries have no single right answer, which is what makes them hard to game. A query like "director of marketing in california senior level" matches many real people and names none of them, so a system cannot fall back on a memorized executive. Exa sampled people with verifiable public profiles who are not prominent enough for a language model to recall from pre-training.

Scoring runs per result. For each profile returned, an LLM judge checks whether that person satisfies every constraint in the query: the Exa grader mostly checks for role and location. HR@1 is the fraction of queries whose top profile passes all of them. P@10 is the fraction of the top ten that pass. Matching one attribute is easy. Matching the full set against fresh profile data is where systems separate.

Why we chose Exa’s benchmark

Exa is widely considered the leader in People search among Search providers. Their benchmark is the closest thing to a high quality, third party, verification of the baseline quality we can achieve.

That said, their benchmark measures quality. It does not measure latency, so we extended the harness to record per-query response time for every provider under identical conditions.

Quality-only benchmarks measure the wrong thing for agents. Two seconds is fine for a recruiter typing a query. It is twelve seconds of dead air inside a six-call sourcing loop.

Every provider ran in its default general-purpose tier, using the configuration its own documentation recommends. Faster low-quality modes exist across several of these APIs including ours, and comparing across tiers tells you nothing.

Our rows return full profile documents where the rest of the table returns snippets or highlights, so we answer four times faster while returning more per call.

How we achieve high quality and low latency

We built the models instead of fine-tuning a general-purpose one.

Retrieval runs in two stages. The first is a hybrid retriever: using impact-weighted terms. It keeps the precision of keyword search while learning which terms matter, running alongside a dense bi-encoder that matches on meaning.

Our re-ranker then scores the top candidates jointly for final ordering. We also train a large teacher, the most accurate and most expensive way to judge a query against a profile, and distill its judgment into the small first-stage retrievers we actually serve. We don't own a GPU cluster. Not because we don't want to. We just kept failing to find a reason to buy one.

Those retrievers inherit reranker-quality judgment without paying reranker cost on every document in the index. That step is what the 0.83 P@10 and the 363ms median are both made of.

People search is a balancing act between a role, a location, and an entity used to disambiguate. 

Our synthetic training data exercises all three together, including near-miss cases: the right role in the wrong city, or the right name attached to a thinner profile. Those are the distinctions that separate a good hit from a plausible miss, and they are exactly what the benchmark's multi-constraint queries test.

Speed comes from both halves at once. Better models alone don't get you to 363 milliseconds, and neither does a fast serving layer running someone else's embeddings. Parallel's 5,102ms median is what happens when quality is bought without the second half.

Run it against your own queries

While we’re proud of our results, individual benchmarks don’t tell the entire story.

Our recommendation: Take three hundred queries from your own product, run them against all eight providers in this table, and score them on what your customers or employees actually need. 

Ready to try Seltz? Try the API today and get 20,000 searches on us.