Searching by meaning with MySQL, PHP and OpenAI: an intelligent database in five steps

Picture a recruitment agency with 100,000 candidates in MySQL. A recruiter is looking for an experienced Laravel developer and types what everyone types:

SELECT reference FROM candidates WHERE profile_text LIKE '%Laravel%';

I rebuilt that with ten made-up profiles and actually ran it. The query found two. The first is someone with a year and a half of bootcamp and internship experience, with "Laravel" four times in the profile. The second is an engineer with nine years of experience who wrote "later partly Laravel". The candidate who designed the public API of a payment service, with OAuth2, rate limiting and versioning, did not come up. That person wrote "PHP" and "API platforms", not "Laravel".

That is the problem with searching on words: people describe the same work in a hundred ways. One is a "PHP backend developer", another a "software engineer with experience in web applications", a third a "developer of complex API platforms". In this article we build a database that searches on meaning, with MySQL, PHP and the OpenAI API. All the code is included and has been tested: against MySQL 8.4, with real embeddings from text-embedding-3-small and a real analysis by gpt-6-luna. The scores in this article were measured, not made up. They also show why the highest score does not have to be the best candidate.

Short answer. A database becomes "intelligent" through three layers. Embeddings turn every text into a series of numbers, so you can search on meaning. A language model weighs only the dossiers that were found against the requirements and names what is missing. The human decides. Embeddings decide nothing, and neither does the language model.

What is an embedding? A map where meaning has a place

Imagine a huge map. Every text gets a point on it. Texts about the same subject lie close together: "PHP backend developer" next to "software engineer for web applications", and "ICU nurse" somewhere else entirely. An embedding is the address of such a point: a row of numbers.

A small example with three numbers. For the sake of the explanation, pretend the first number measures "programming", the second "web interfaces" and the third "healthcare":

A and B do not share a single word, but their arrows point almost the same way. You measure that with cosine similarity: the cosine of the angle between two arrows. You multiply the numbers pairwise and add them up (the dot product), and divide by the lengths of both arrows:

A·B = 0.9×0.8 + 0.2×0.4 + 0.1×0.1 = 0.81
|A| = √(0.81 + 0.04 + 0.01) = 0.927
|B| = √(0.64 + 0.16 + 0.01) = 0.900
cos(A, B) = 0.81 / (0.927 × 0.900) = 0.97

cos(A, C) = 0.20 / (0.927 × 0.911) = 0.24

A 1 means: exactly the same direction. A 0 means: nothing to do with each other. A and B score 0.97, A and C 0.24.

A real model works with many more numbers. text-embedding-3-small returns 1,536 per text by default. And those numbers have no names like "programming" or "healthcare". The model learned the positions from huge amounts of text, and nobody can tell you what number 812 means. The calculation stays the same, though. OpenAI also delivers the vectors already at length 1. That makes the denominators drop out, and cosine similarity equals the dot product.

Important to hold on to: this is geometry, not understanding. The model does not know what a job application is. It knows that certain words and sentences occur in similar contexts. Further down is a measured example where exactly that goes wrong: a candidate who writes that they want to become senior scores as if they already are.

How does an intelligent database work? Four phases, two moments

The system does its work at two different moments. Preparation happens in advance, once per profile and again whenever a profile changes. Searching, analysing and deciding only happen when a recruiter asks a question.

Phase A, preparing the data. PHP reads a candidate profile from MySQL and sends the relevant text to the OpenAI Embeddings API. That returns a vector of 1,536 numbers, and PHP stores it with the profile.

Phase B, searching. The recruiter asks a question. PHP has an embedding made of that question too. Then SQL first filters the hard requirements, such as at least five years of experience. PHP compares the vector of the question with the stored vectors and keeps the best-matching profiles.

Phase C, analysis. Only those few profiles go to a language model, together with the job requirements. The model puts the requirements next to the evidence, quotes from the dossiers and says what is missing.

Phase D, deciding. The recruiter reads the reasoning, checks the sources and decides. That is not a courtesy step. In recruitment it is a legal requirement, more on that below.

The example in PHP and MySQL, in five steps

You need: PHP 8.2 or higher with PDO and cURL, MySQL 8.4 and an OpenAI API key. I use MySQL 8.4 because it is the LTS version; MySQL 8.0 has been end-of-life since 30 April 2026. The code also works on 9.7. The key and the database credentials live in environment variables, never in the code and never in Git.

Step 1: a table for candidates, profiles and embeddings

schema.sql

CREATE TABLE candidates (
    id               INT UNSIGNED AUTO_INCREMENT PRIMARY KEY,
    reference        VARCHAR(20)  NOT NULL UNIQUE,
    profile_text     TEXT         NOT NULL,
    years_experience TINYINT UNSIGNED NULL,
    region           VARCHAR(60)  NULL,
    available_from   DATE         NULL,
    embedding        JSON         NULL,
    embedding_model  VARCHAR(60)  NULL,
    embedding_hash   CHAR(64)     NULL,
    updated_at       TIMESTAMP    NOT NULL DEFAULT CURRENT_TIMESTAMP
                                  ON UPDATE CURRENT_TIMESTAMP
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4;

A few choices stand out. There is no name, email address or date of birth in it. You do not need them to search on skills, and what is not in there cannot leak to OpenAI. The reference number (K-2207) points back to the real dossier in your ATS. The hard data, such as years of experience, region and availability, sit in separate columns. You want to filter on those with plain SQL, not with a vector.

The embedding sits in a JSON column. MySQL 8.4 has no vector type, and a JSON array of 1,536 numbers works everywhere. In my test such a column was about 31 KB per profile. I also keep two columns you will be grateful for later. embedding_model records which model made the vector, because you cannot compare vectors from different models with each other. embedding_hash is a fingerprint of the text the vector was made from.

Step 2: making an embedding with PHP

openai.php

<?php
declare(strict_types=1);

function openai(string $endpoint, array $payload): array
{
    $key = getenv('OPENAI_API_KEY');
    if ($key === false || $key === '') {
        throw new RuntimeException('OPENAI_API_KEY is not set');
    }

    $ch = curl_init('https://api.openai.com/v1/' . $endpoint);
    curl_setopt_array($ch, [
        CURLOPT_POST           => true,
        CURLOPT_RETURNTRANSFER => true,
        CURLOPT_HTTPHEADER     => [
            'Authorization: Bearer ' . $key,
            'Content-Type: application/json',
        ],
        CURLOPT_POSTFIELDS     => json_encode($payload, JSON_THROW_ON_ERROR),
        CURLOPT_CONNECTTIMEOUT => 10,
        CURLOPT_TIMEOUT        => 120,
    ]);

    $body = curl_exec($ch);
    if ($body === false) {
        throw new RuntimeException('Network error: ' . curl_error($ch));
    }
    $status = curl_getinfo($ch, CURLINFO_RESPONSE_CODE);

    $data = json_decode($body, true);
    if ($status >= 400 || !is_array($data)) {
        $message = $data['error']['message'] ?? substr($body, 0, 200);
        throw new RuntimeException("OpenAI returned HTTP $status: $message");
    }
    return $data;
}

function embed(array $texts): array
{
    $data = openai('embeddings', [
        'model' => 'text-embedding-3-small',
        'input' => array_values($texts),
    ]);

    $vectors = [];
    foreach ($data['data'] as $item) {
        $vectors[$item['index']] = $item['embedding'];
    }
    ksort($vectors);
    return $vectors;
}

openai() is the only place where the network comes into the picture. The key comes from getenv(); if it is missing, the function stops right away with a clear error. There are two timeouts: ten seconds to connect and 120 for the whole response, because an analysis by a language model can take a long time (more on that shortly). A network error throws an exception. So does an HTTP status of 400 or higher, with OpenAI's error message in it; with an invalid key you get, for example, OpenAI returned HTTP 401: Incorrect API key provided. If no JSON comes back, for example an HTML error page from a proxy, the first 200 characters of it go into the message. There is no curl_close(): since PHP 8.0 that function does nothing, and in PHP 8.5 it was deprecated.

embed() sends a whole list of texts in one request. The API accepts up to 2,048 texts per request, with at most 300,000 tokens in total, and at most 8,192 tokens per text. The response contains an index for each text. I sort on that, so vector 3 is guaranteed to belong to text 3.

A practical pitfall from my test. On a Windows laptop, cURL first returned unable to get local issuer certificate. The cause was the virus scanner (Norton), which breaks open HTTPS traffic and re-encrypts it with its own certificate. PHP did not know that certificate. Do not switch the check off with CURLOPT_SSL_VERIFYPEER = false. Point curl.cainfo and openssl.cafile in php.ini to a bundle that contains that certificate, or in PHP 8.2+ use CURLOPT_SSL_OPTIONS => CURLSSLOPT_NATIVE_CA, so cURL trusts the Windows certificates itself.

Step 3: storing, and only redoing it for a changed profile

db.php

<?php
declare(strict_types=1);

function db(): PDO
{
    return new PDO(
        getenv('DB_DSN'),   // e.g. mysql:host=127.0.0.1;dbname=recruit;charset=utf8mb4
        getenv('DB_USER'),
        getenv('DB_PASS'),
        [
            PDO::ATTR_ERRMODE            => PDO::ERRMODE_EXCEPTION,
            PDO::ATTR_DEFAULT_FETCH_MODE => PDO::FETCH_ASSOC,
        ]
    );
}

index_embeddings.php

<?php
declare(strict_types=1);
require __DIR__ . '/db.php';
require __DIR__ . '/openai.php';

$pdo  = db();
$rows = $pdo->query('SELECT id, profile_text, embedding_hash FROM candidates')->fetchAll();

// Only profiles that are new or changed
$todo = array_filter($rows, fn ($r) => $r['embedding_hash'] !== hash('sha256', $r['profile_text']));

$update = $pdo->prepare(
    'UPDATE candidates
        SET embedding = ?, embedding_model = ?, embedding_hash = ?
      WHERE id = ?'
);

foreach (array_chunk($todo, 100) as $batch) {
    $vectors = embed(array_column($batch, 'profile_text'));

    $pdo->beginTransaction();
    foreach ($batch as $i => $row) {
        $update->execute([
            json_encode($vectors[$i]),
            'text-embedding-3-small',
            hash('sha256', $row['profile_text']),
            $row['id'],
        ]);
    }
    $pdo->commit();
}
echo count($todo) . " profiles updated\n";

For each profile, the script compares the SHA-256 hash of the current text with the stored hash. Only new and changed profiles go to OpenAI, in blocks of a hundred. Each block is saved in one transaction. The first time it reported "10 profiles updated", the second time "0 profiles updated". So you can run it every hour via cron without paying twice for the same text.

What does that cost? A worked example. text-embedding-3-small costs $0.02 per million tokens. If you give 100,000 profiles an average of 400 tokens, that is 40 million tokens: $0.80 for the whole database. Through the Batch API, which gives a 50% discount and processes up to 50,000 texts per batch, it comes to about $0.40. The embeddings are rarely the expensive side of a system like this.

Step 4: turning the question into a vector and comparing

search.php

<?php
declare(strict_types=1);
require_once __DIR__ . '/db.php';
require_once __DIR__ . '/openai.php';

function cosine(array $a, array $b): float
{
    $dot = $normA = $normB = 0.0;
    foreach ($a as $i => $value) {
        $dot   += $value * $b[$i];
        $normA += $value * $value;
        $normB += $b[$i] * $b[$i];
    }
    return $dot / (sqrt($normA) * sqrt($normB));
}

function search(PDO $pdo, string $question, int $minYears = 0, int $limit = 5): array
{
    $queryVector = embed([$question])[0];

    // Hard requirements first in SQL, meaning afterwards in PHP
    $stmt = $pdo->prepare(
        'SELECT id, reference, profile_text, years_experience, region, embedding
           FROM candidates
          WHERE embedding IS NOT NULL
            AND embedding_model = :model
            AND years_experience >= :min_years'
    );
    $stmt->execute(['model' => 'text-embedding-3-small', 'min_years' => $minYears]);

    $results = [];
    foreach ($stmt as $row) {
        $vector = json_decode($row['embedding'], true, 512, JSON_THROW_ON_ERROR);
        unset($row['embedding']);
        $row['score'] = cosine($queryVector, $vector);
        $results[] = $row;
    }

    usort($results, fn ($a, $b) => $b['score'] <=> $a['score']);
    return array_slice($results, 0, $limit);
}

cosine() is the formula from the three-number example, now for 1,536 numbers. Because OpenAI vectors already have length 1, the dot product ($dot) would be enough. I keep the full formula, so the function is also correct for models that do not normalise.

search() does three things in a deliberate order. First the question becomes a vector. Then SQL fetches only the profiles that pass the hard filters, with a prepared statement, and only vectors from the same model. Only after that does PHP calculate the score per profile and sort from high to low. Note one detail: years_experience >= :min_years leaves out profiles where the number of years is missing, because NULL >= 5 is not true in SQL. That may be exactly right, but decide it deliberately.

Step 5: having GPT analyse the best results

analyse.php

<?php
declare(strict_types=1);
require_once __DIR__ . '/search.php';

function analyse(string $vacancy, array $candidates): array
{
    $dossiers = array_map(fn ($c) => [
        'candidate_id'     => $c['reference'],
        'profile_text'     => $c['profile_text'],
        'years_experience' => $c['years_experience'],
        'region'           => $c['region'],
    ], $candidates);

    $schema = [
        'type' => 'object',
        'additionalProperties' => false,
        'required' => ['candidates'],
        'properties' => [
            'candidates' => [
                'type'  => 'array',
                'items' => [
                    'type' => 'object',
                    'additionalProperties' => false,
                    'required' => ['candidate_id', 'relevant_skills', 'evidence',
                                   'missing_information', 'recommendation', 'sources'],
                    'properties' => [
                        'candidate_id'        => ['type' => 'string'],
                        'relevant_skills'     => ['type' => 'array', 'items' => ['type' => 'string']],
                        'evidence'            => ['type' => 'array', 'items' => ['type' => 'string']],
                        'missing_information' => ['type' => 'array', 'items' => ['type' => 'string']],
                        'recommendation'      => ['type' => 'string'],
                        'sources'             => ['type' => 'array', 'items' => ['type' => 'string']],
                    ],
                ],
            ],
        ],
    ];

    $response = openai('responses', [
        'model' => 'gpt-6-luna',
        'input' => [
            ['role' => 'system', 'content' =>
                'You support a recruiter. Use only the supplied dossiers. ' .
                'In evidence, quote profile_text literally, and in sources name the ' .
                'candidate_id and the fields you used. If something is not there, put it ' .
                'under missing_information and do not guess it. You do not decide: you only ' .
                'advise which next step the recruiter can take.'],
            ['role' => 'user', 'content' => json_encode(
                ['vacancy' => $vacancy, 'dossiers' => $dossiers],
                JSON_THROW_ON_ERROR | JSON_UNESCAPED_UNICODE
            )],
        ],
        'text' => ['format' => [
            'type'   => 'json_schema',
            'name'   => 'candidate_advice',
            'strict' => true,
            'schema' => $schema,
        ]],
    ]);

    if (($response['status'] ?? '') !== 'completed') {
        throw new RuntimeException('Analysis not completed: ' . ($response['status'] ?? 'unknown'));
    }
    foreach ($response['output'] as $item) {
        if ($item['type'] !== 'message') {
            continue;
        }
        foreach ($item['content'] as $part) {
            if ($part['type'] === 'refusal') {
                throw new RuntimeException('Model refused: ' . $part['refusal']);
            }
            if ($part['type'] === 'output_text') {
                return json_decode($part['text'], true, 512, JSON_THROW_ON_ERROR);
            }
        }
    }
    throw new RuntimeException('No text in the response');
}

This is the Responses API (POST /v1/responses), which OpenAI recommends for new projects. Chat Completions remains available. gpt-6-luna is OpenAI's cheapest model for well-defined tasks: $0.10 per million input tokens and $0.50 per million output tokens. Analysing three dossiers therefore costs fractions of a cent.

Three things make the answer verifiable. The instruction forbids guessing and requires literal quotes with source references. The JSON schema with strict: true enforces the shape: every field is required, and extra fields are not allowed. And the code reads the response as it actually arrives. output_text only exists in OpenAI's SDKs. In the raw response you look in output for the item of type message, and in it for the part of type output_text. Do not blindly take output[0]: a reasoning step can come first there. A refusal comes back as refusal, and a cut-off response has the status incomplete.

One measurement to keep in mind: the analysis of three dossiers took 77 seconds in my test. The search step took 3.9 seconds, almost all of it waiting for the embedding of the question; the comparison itself takes 40 ms for a hundred profiles. So put such an analysis in a queue, or show the recruiter the search results first and the advice later. A screen with a spinning wheel is not the right solution here.

Why JSON in MySQL is a teaching model, not a vector index

The code above reads all vectors from the database on every search and calculates them one by one. That is called brute force. It is exact: you never miss the best match. But it scales linearly. Ten times more profiles means ten times more work. How vectors are handled in the different databases:

So MySQL 9.7 Community can store a vector as VECTOR(1536), with STRING_TO_VECTOR() and VECTOR_DIM(). Anyone who then calls DISTANCE() gets an error, because according to the manual that function is only available in HeatWave on OCI and in MySQL AI. On a regular MySQL server the calculation therefore stays in PHP, whichever you choose. What happens when you do the same in MariaDB, PostgreSQL and SQLite, I described in What AI did to databases for web applications.

Rules, search and analysis: three techniques for three questions

Traditional software works with rules that a programmer wrote out:

IF experience >= 5 AND knows_laravel = true THEN invite

That is predictable and explainable. It fails as soon as reality does not fit your columns. Someone who wrote "Symfony, later partly Laravel": is knows_laravel true or false?

Semantic search finds texts that are similar in content to the question, even when the words differ. It gives a ranking. It does not give a judgement.

AI-assisted analysis has a language model put the information found next to the criteria. The model names what is proven and what is missing, and proposes a next step.

You can see how they work together in the real scores. This is what search() returned for the query "Senior PHP/Laravel developer with demonstrable API experience", across all ten profiles. The test ran in Dutch, so the query and the profiles were Dutch text:

Three things stand out.

There is no "good" score. All ten profiles score between 0.46 and 0.68, including the project manager and the data engineer. A score of 0.5 says nothing with this model, and a fixed threshold such as "everything above 0.8 is a match" yields zero results here. OpenAI does not give threshold values. Determine them on your own data, with examples where you know the correct outcome.

Ambition scores like experience. K-1041 has two years of experience and sits at 0.671, just below the engineer with nine years. The profile ends with "Wants to grow into a senior Laravel developer". For the embedding, "senior" and "Laravel" are simply in there. The vector cannot tell that it is a wish, not a fact. I measured it again: without that one sentence the score drops to 0.653, and then K-3315 does rank above it. The frequently repeated word "Laravel" does most of the work, the wish does the rest.

The best candidate may be number three. K-3315 designed the public API of a payment service, with OAuth2, rate limiting and versioning. That is the strongest API evidence of the three, but this profile scores lowest: mostly Go in the last two years, and Laravel is not mentioned. Whether that is a drawback is a question for the interview. A vector cannot answer that question.

This is where the layers together do what they cannot do apart. SQL removes K-1041 with one line (min_years = 5). The search finds K-3315, whom LIKE would have missed. And the language model saw what neither of them saw. About K-2207 it wrote: "Whether the candidate designed and built APIs themselves; it says the candidate built integrations." That is exactly the question a good recruiter asks in the first interview.

At the same time, the limits remain. Embeddings make no decisions, they sort. A language model can be wrong, even with quotes. And the advice is never better than the dossier: if someone's API work is not in the profile, it does not exist for this whole system.

What is RAG? First the dossiers, then the advice

What we built in steps 4 and 5 is called Retrieval-Augmented Generation, RAG. The idea is simple. The language model does not need to know your database. For each question it gets only the information the search step found, and writes based on that.

Compare it with a good adviser. They do not tell you from memory who is suitable, but first take the three relevant dossiers out of the cabinet, put them next to the vacancy and then write an advice, with the dossier and the page next to every claim.

That gives you four things:

• Up to date. A profile updated this morning counts this afternoon. The model does not need retraining, only that one embedding has to be redone.
• Verifiable. Every quote refers to a candidate ID and a field. A recruiter can check in ten seconds whether it is really there.
• Cost. Worked example: 100,000 profiles of 400 tokens are 40 million tokens together. Even if that fitted in one prompt, it would cost at least $4 per question with gpt-6-luna, and above 272,000 tokens per prompt OpenAI charges double the rate. Three dossiers are a few hundred tokens.
• Less fabrication. A model that only has to choose from what it is given, and is instructed to report empty spots as "missing", makes up less than a model that has to draw on its memory. Less is not zero. That is why the quotes are there.

Searching, classifying, predicting and deciding are four different things

The same pattern works wherever knowledge sits in text that people write down differently:

• Customer service: for a new complaint, show the five most similar resolved tickets, with their solution.
• Legal team: find clauses that are similar in content to a limitation of liability, even when the word "liability" is not in them.
• Development team: search the technical documentation with the question as you would ask it ("why does the import fall over at night?"), not with the term the writer happened to choose.
• Labour market analysis: group job ad texts into clusters of skills, and see which clusters grow per quarter.
• Recruitment: put candidate profiles next to job requirements, as in this article.

Keep the verbs apart. Searching is: what is similar to this? Classifying is: which box does this belong in, for example "invoice question" or "outage"? That can be done with embeddings plus a few hundred labelled examples. Predicting is: what is going to happen? Deciding is: what do we do? That last one stays with a human.

Predicting is the pitfall. That job ads about "Laravel" and "API" lie close together in vector space says nothing about the demand for Laravel developers next year. For that you need time series, historical counts, a statistical model and a test on data the model had not yet seen. Vectors can help order text before you start counting. They do not make a forecast.

Reliability, privacy and the law: where it goes wrong

Advice without a source is an opinion

The schema in step 5 requires evidence (literal quotes) and sources (candidate ID and field name). That lets a human check the advice. You can also check this automatically in PHP: does every quote appear literally in that candidate's profile_text? A quote that cannot be found is a red flag. Then do not show that advice, or mark it.

Hard requirements belong in SQL

A legal work permit, a required diploma, availability from a certain date: those are yes/no questions. Ask them of the database, not of a vector or a language model. SQL does not get "at least five years" wrong, and afterwards you can show exactly why someone fell outside the selection.

Less data, less risk

Send OpenAI only what is needed to search on skills. Leave out name, address, date of birth, photo and nationality. Also remove sentences that reveal health, religion or origin ("after a burnout", "volunteer at the mosque"). Such data falls under Article 9 of the GDPR, the special categories of personal data, and in principle you may not process it.

OpenAI's API does not train on what you send by default. Data for abuse monitoring is kept for up to thirty days, though. Zero Data Retention is only possible after approval by OpenAI. Processing in Europe is possible via eu.api.openai.com, but that also requires such a retention agreement, and for models released from 5 March 2026 OpenAI charges 10% more for it.

An embedding is not anonymous

It feels as if 1,536 bare numbers no longer say anything about a person. That is not true. Researchers at Cornell showed in 2023 that their method Vec2Text could recover 92% of 32-token texts exactly from the embedding, and recovered full names from embeddings of medical notes. That was with an open embedding model, not with text-embedding-3-small, but the principle stands: an embedding of a personal profile is personal data. Secure the column as you would secure the text, and delete it together with the profile.

What the law requires for AI in recruitment

GDPR, Article 22: no one may be subjected to a decision based solely on automated processing that significantly affects them. In the SCHUFA case (C-634/21, 7 December 2023), the Court of Justice ruled that an automatically calculated score can itself be such a decision, if the person deciding relies on it decisively. A recruiter who only looks at the top 3 of a vector score and never sees the rest comes dangerously close.

The AI Act explicitly lists AI for recruitment and selection, such as filtering applications and evaluating candidates, as high-risk in Annex III. Because of Regulation (EU) 2026/1744, the "Digital Omnibus", those obligations apply from 2 December 2027, no longer from August 2026. Anyone using such a system must then, among other things, ensure human oversight by people with the necessary knowledge and authority. The logs must be kept for at least six months, and employees and their representatives must be informed in advance. The ban on emotion recognition in the workplace has applied since 2 February 2025. The Dutch Data Protection Authority (Autoriteit Persoonsgegevens) wrote in March 2026 that AI in recruitment is growing fast and brings major risks, and that transparency is often lacking.

The human holds the pen

In practice that means the following. The system orders and advises, and a recruiter decides and signs. The recruiter can see every candidate, including number 40. Every piece of advice has sources. Every decision is recorded with who, when and why. And someone who was rejected can ask how that decision came about and gets an answer a human can explain.

When is PHP fast enough? Measured at 100, 1,000 and 10,000 profiles

I measured the search step with random vectors of 1,536 dimensions, on a laptop with MySQL 8.4 in Docker and PHP 8.2 without JIT. Each measurement is the median of three runs.

The surprise is not in the maths. All the cosine calculations together took 491 ms at 10,000 profiles. Most of the work was transport: MySQL stores JSON in binary form, but converts it to text of 31 KB per profile on every SELECT, and PHP has to parse that text again. At 10,000 profiles that alone took 199 MB of memory, and 551 MB after json_decode.

If you store the same vector as a BLOB of 6,144 bytes (pack('g*', ...$vector) when storing, unpack('g*', $blob) when reading), it drops to 957 ms. That is six times faster. The difference in precision from the JSON version was at most 0.000000004. For ranking that makes no difference.

What does that mean in practice? Treat it as a rule of thumb, because the limits depend on your server, the number of concurrent searches, the number of dimensions and how fast an answer has to be there:

• Around 100 candidates: the JSON version from this article is enough. Nobody notices 40 ms.
• Around 10,000 candidates: measure. Almost six seconds per search is too slow for a search field. Switch to a BLOB, filter out as much as possible with SQL beforehand, and consider dimensions: 512 when creating the embeddings. That gives shorter vectors, which are somewhat less accurate. Measure that too.
• Around 1,000,000 candidates: this is where brute force stops. Extrapolated from the BLOB measurement, a single search would take well over a minute and a half, with gigabytes of vectors per request. Then you need a vector index, such as HNSW in MariaDB or pgvector, or a separate vector database. Such an index searches approximately: much faster, but it can occasionally miss the best match.

Four more things that pay for themselves quickly. Store the embedding of frequent search queries, so the same question does not keep going to OpenAI. For a large initial load, create the embeddings via the Batch API, for half the price. Run the index script after every profile change, or via cron; the hash makes sure only changed profiles go again. And if you ever switch embedding models, recreate all vectors. That is why the embedding_model column exists, and why search() filters on it.

The demonstration: one question in plain language, from sentence to advice

The recruiter types:

"Which candidates are suitable for a senior PHP/Laravel role, have demonstrable API experience, and what information is still missing to make a good selection?"

A lesson from the test: do not turn this whole sentence into a vector in one go. The second part ("what information is still missing") is an instruction to the language model, not a search term. With the full sentence, K-2207 scored 0.614. With only the search part ("Senior PHP/Laravel developer with demonstrable API experience") that rose to 0.681, and the distance to the irrelevant profiles grew. So split the question. The application can do that with a fixed form, or with a small first model call.

After that it runs like this:

1. The search part becomes an embedding.
2. SQL filters the hard requirements.
3. search() ranks what is left.
4. The three best dossiers go to analyse() together with the job ad text.
5. The advice comes back as JSON with quotes and sources.
6. The recruiter checks and decides.

This is the real answer from gpt-6-luna in the test for K-3315, translated from the model's Dutch answer. The vacancy asked for at least five years of backend experience, API design and hybrid work in Utrecht.

{
  "candidate_id": "K-3315",
  "relevant_skills": ["Seven years of backend experience", "PHP", "Go",
    "Design of public APIs", "OAuth2", "Rate limiting", "Versioning"],
  "evidence": [
    "“Seven years of backend experience in Go and PHP; designed the public API of a payment service with OAuth2, rate limiting and versioning.”",
    "“Mostly Go in the last two years.”"
  ],
  "missing_information": [
    "Laravel experience is not mentioned.",
    "How much PHP experience there is and how recent it is, is not mentioned.",
    "Whether the candidate is willing to work at the office in Utrecht two days a week is not mentioned."
  ],
  "recommendation": "Invite the candidate for a technical interview about PHP and Laravel experience and check whether that experience fits the role; also discuss the hybrid work location.",
  "sources": ["K-3315: profile_text", "K-3315: region", "K-3315: years_experience"]
}

About K-1041, the candidate with the ambition, the model wrote plainly: "At least five years of experience as a backend developer is not mentioned; the dossier mentions two years of total experience" and "Experience as a senior developer is not mentioned; the candidate wants to grow into a senior Laravel developer." What the vector read as "senior", the language model did pick out. It did not advise rejection. It proposed questions, and left the judgement to the recruiter.

That is exactly how the division of roles should be. An intelligent database does not know everything by itself, and it does not decide by itself either. It is a system that finds the relevant information, analyses it in context and helps people make better-founded decisions. If you want to try it yourself: start with a hundred real, anonymised profiles and ten vacancies where you know who was hired at the time. Check whether that person is in the top ten. That one number says more about your system than any demo.

Frequently asked questions

Do I need MySQL 9 for vector search?

No. MySQL 9.7 Community has a VECTOR type, but no distance function and no vector index. The calculation then still happens in PHP, just as with a JSON or BLOB column in MySQL 8.4. If you want the database itself to calculate distances, look at MySQL HeatWave, MariaDB 11.8 or PostgreSQL with pgvector.

What is a good threshold for cosine similarity?

In general there is none. With text-embedding-3-small, all ten profiles in my test scored between 0.46 and 0.68, including the ones that did not fit. Use the ranking instead of a threshold, or set a threshold with examples from your own data where you know the correct outcome.

Can I build this without OpenAI?

Yes. The architecture stays the same; only embed() and analyse() call a different model, for example a local embedding model and a local language model. Vectors from different models cannot be compared with each other. So choose one model per database, and recreate everything when you switch.

May I let AI select candidates automatically?

Not fully automatically. Article 22 of the GDPR in principle prohibits decisions based solely on automated processing that significantly affect someone. According to the SCHUFA judgment, a decisive score can already fall under that. From 2 December 2027, the obligations for high-risk AI from the AI Act also apply. Let the system order and advise, and let a human decide.

Further reading on this site

• What AI did to databases for web applications, from Oracle to SQLite
• Building an AI agent in PHP: a complete example with Composer, Claude and MCP
• The Jev principle explained: AI that ticks boxes instead of writing
• What is MCP? How to connect AI safely to tools, data and business software

Sources

• OpenAI, Embeddings guide (1,536 and 3,072 dimensions, 8,192 tokens, normalisation to length 1, dimensions parameter) and Embeddings API reference (2,048 texts, 300,000 tokens per request); viewed 2 November 2026
• OpenAI, Pricing, Batch API and gpt-6-luna; viewed 2 November 2026
• OpenAI, Migrate to the Responses API and Structured Outputs
• OpenAI, Your data (training, retention period, Zero Data Retention, European data residency)
• MySQL, Data Types 8.4, The VECTOR Type and Vector Functions (9.7); release notes 8.4
• MariaDB, Vector Overview; pgvector
• Morris, Kuleshov, Shmatikov and Rush, Text Embeddings Reveal (Almost) As Much As Text, EMNLP 2023
• Court of Justice of the EU, judgment C-634/21 (SCHUFA), 7 December 2023; GDPR, Articles 9 and 22
• AI Act (EU) 2024/1689, Annex III point 4 and Article 26; Council of the EU, final approval of the simplification (29 June 2026), published as Regulation (EU) 2026/1744
• Dutch Data Protection Authority (Autoriteit Persoonsgegevens), AI and algorithmic risks report Netherlands, March 2026 and rules for automated decision-making in recruitment and selection
• Own test, 2 November 2026: MySQL 8.4.13 in Docker, PHP 8.2.35, text-embedding-3-small and gpt-6-luna; ten made-up profiles, no real candidates