Semantic search in Java

What you’re building

A search over eight handwritten notes. You ask “where did I put my keys?” and it finds:

The spare set for the front door is in the kitchen drawer.

That note contains no “keys”. No “key”. A LIKE '%key%' query would miss it completely and confidently hand you the note about a 4mm hex key instead.

Getting the right note to win is what this guide is about.

Part 1 needs no API key and no build tool

This guide comes in two halves. Part 1 runs right now — one file, no Maven, no account, no cost — using vectors we write by hand so you can watch the machinery work before paying for anything. Part 2 brings in a real model, and only one method changes.

New to embeddings? Do Part 1. It takes ten minutes and everything afterwards lands better.

Before you start

You need a JDK, version 17 or newer. Check:

java -version

If that prints 17 or higher, you’re ready. If the command isn’t found, grab a build from Adoptium (the standard free OpenJDK distribution) or use your package manager.

For Part 1 that’s genuinely all you need: no Maven, no Gradle, no IDE.

---

Part 1: the mechanics, with no API key

Step 1: Understand the one idea

An embedding is a list of numbers standing in for a piece of text. Text meaning similar things gets similar numbers. That’s the whole concept.

Once meaning is numbers, “find something similar to this” is arithmetic, and computers are very good at arithmetic.

For Part 1 we write those numbers by hand, using five made-up categories:

        [ where things    tools     food    health   admin  ]
        [  are kept      & DIY                      & money ]

"spare set … kitchen drawer"   [ 0.95,   0.20,   0.05,   0.00,   0.10 ]
"standing desk … 4mm hex key"  [ 0.15,   0.95,   0.00,   0.05,   0.00 ]
"pizza dough: 500g flour…"     [ 0.05,   0.05,   0.95,   0.10,   0.00 ]

The first note scores high on where things are kept. The hex key note scores high on tools. Ask about keys and the first wins, because you were asking where something is, not about DIY.

This part is fake, deliberately

I picked those numbers by hand so the maths is visible. A real model does the same job with 1,536 numbers instead of 5, worked out from reading an enormous amount of text rather than from one person’s judgement.

The part that matters: the search code doesn’t change. Swap five hand-picked numbers for 1,536 learned ones and every line below still works. That’s all Part 2 does.

Step 2: Write the file

Create a file called NotesSearch.java. Anywhere; your desktop is fine.

NotesSearch.java

import java.util.Comparator;
import java.util.List;

public class NotesSearch {

    // A note, and the numbers standing in for its meaning.
    record Note(String text, double[] vector) {}

    // A question, and what we reckon it's "about".
    record Search(String question, double[] vector) {}

    // Five made-up categories: where things are kept, tools & DIY,
    // food, health, admin & money.
    static final List<Note> NOTES = List.of(
        new Note("The spare set for the front door is in the kitchen drawer.", new double[]{0.95, 0.20, 0.05, 0.00, 0.10}),
        new Note("The dog needs his flea treatment on the 3rd.",               new double[]{0.05, 0.00, 0.10, 0.70, 0.35}),
        new Note("Mum's birthday is in April, she likes tulips.",              new double[]{0.05, 0.00, 0.15, 0.00, 0.60}),
        new Note("Renew the car insurance before it lapses.",                  new double[]{0.00, 0.05, 0.00, 0.00, 0.95}),
        new Note("The standing desk needs a 4mm hex key to assemble.",         new double[]{0.15, 0.95, 0.00, 0.05, 0.00}),
        new Note("Pizza dough: 500g flour, 300ml water, rest overnight.",      new double[]{0.05, 0.05, 0.95, 0.10, 0.00}),
        new Note("The wifi password is taped under the router.",               new double[]{0.65, 0.10, 0.00, 0.00, 0.20}),
        new Note("Physio said ice the knee for twenty minutes.",               new double[]{0.00, 0.00, 0.00, 0.95, 0.05})
    );

    static final List<Search> SEARCHES = List.of(
        new Search("where did I put my keys?",  new double[]{0.90, 0.30, 0.00, 0.00, 0.05}),
        new Search("what do I cook on Friday?", new double[]{0.05, 0.05, 0.95, 0.05, 0.00}),
        new Search("my knee is hurting",        new double[]{0.00, 0.00, 0.00, 0.95, 0.05})
    );

    public static void main(String[] args) {
        for (Search search : SEARCHES) {
            System.out.println("\nQ: " + search.question());

            NOTES.stream()
                 .map(note -> new Object() {
                     final String text = note.text();
                     final double score = cosineSimilarity(note.vector(), search.vector());
                 })
                 .sorted(Comparator.comparingDouble(r -> -r.score))
                 .limit(3)
                 .forEach(r -> System.out.printf("  %.3f  %s%n", r.score, r.text));
        }
    }

    /**
     * How similar are two vectors? Returns roughly -1 (opposite) to 1 (identical).
     * It compares the direction the vectors point, ignoring their length.
     */
    static double cosineSimilarity(double[] a, double[] b) {
        double dot = 0, lengthA = 0, lengthB = 0;

        for (int i = 0; i < a.length; i++) {
            dot += a[i] * b[i];       // multiply matching positions, add them up
            lengthA += a[i] * a[i];   // Pythagoras, part one
            lengthB += b[i] * b[i];   // Pythagoras, part two
        }

        return dot / (Math.sqrt(lengthA) * Math.sqrt(lengthB));
    }
}

Step 3: Run it

Since Java 11 you can run a single source file directly, with no compile step and no build tool:

java NotesSearch.java

Q: where did I put my keys?
  0.991  The spare set for the front door is in the kitchen drawer.
  0.957  The wifi password is taped under the router.
  0.459  The standing desk needs a 4mm hex key to assemble.

Q: what do I cook on Friday?
  0.999  Pizza dough: 500g flour, 300ml water, rest overnight.
  0.245  Mum's birthday is in April, she likes tulips.
  0.176  The dog needs his flea treatment on the 3rd.

Q: my knee is hurting
  1.000  Physio said ice the knee for twenty minutes.
  0.907  The dog needs his flea treatment on the 3rd.
  0.104  Pizza dough: 500g flour, 300ml water, rest overnight.

That output is real: this sample is compiled and run before publishing, so what you see above is what the program actually printed.

That’s a semantic search. One file, no dependencies, one loop of arithmetic.

Look at the first result. The front-door note scores 0.991; the hex key note — the only one that literally contains “key” — scores 0.459 and lands third. Keyword search would rank those two in precisely the opposite order.

The third search is worth a look too. “My knee is hurting” finds the physio note, then the dog’s flea treatment. Both are health-shaped, so they sit near each other. That’s a real characteristic of this technique rather than a bug — “related” and “relevant” aren’t the same thing, which is why reranking exists.

Poke at it before moving on

Change a search vector and re-run. Push the food number to 0.95 and watch the pizza note climb. Add a ninth note with numbers of your own. Five minutes of experimenting beats another page of reading.

About that anonymous class

.map(note -> new Object() { ... }) is a slightly unusual way to pair a note with its score. It’s here to keep Part 1 to a single file with no extra type declarations.

In real code you’d declare a small record instead — which is exactly what Part 2 does with record Match(String note, double score) {}. Clearer, and it plays properly with the type system.

---

Part 2: real embeddings

Part 1 works, but we wrote the numbers. That doesn’t survive contact with reality. Nobody hand-scores ten thousand documents across five invented axes.

A real model does it for you. Same maths, one method swapped.

Step 1: Make a Maven project

Now we need a dependency, so we need a build tool. Create this structure:

notes-search/
├── pom.xml
└── src/main/java/NotesSearch.java

pom.xml

<project xmlns="http://maven.apache.org/POM/4.0.0">
    <modelVersion>4.0.0</modelVersion>
    <groupId>demo</groupId>
    <artifactId>notes-search</artifactId>
    <version>1.0</version>

    <properties>
        <maven.compiler.source>17</maven.compiler.source>
        <maven.compiler.target>17</maven.compiler.target>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
    </properties>

    <dependencies>
        <dependency>
            <groupId>com.openai</groupId>
            <artifactId>openai-java</artifactId>
            <version>4.52.0</version>
        </dependency>
    </dependencies>
</project>

Check Maven Central for the current version before copying, because this SDK moves quickly, and a stale version means debugging a method that got renamed several releases ago.

Gradle equivalent:

build.gradle.kts

dependencies {
    implementation("com.openai:openai-java:4.52.0")
}

Step 2: Get an API key

Create one at platform.openai.com/api-keys and copy it immediately, since it’s shown once.

New accounts need a payment method before the API responds at all. Embeddings cost very little (everything in this guide is a fraction of a penny), but the card has to be on file.

export OPENAI_API_KEY="sk-..."

Never hard-code the key

Putting it straight in the .java file feels faster. It isn’t. Once that file is committed the key is in your git history permanently. Deleting the line later doesn’t remove it, and revoking the key becomes the only real fix.

OpenAIOkHttpClient.fromEnv() reads it from the environment for you, which is why the code below never mentions the key at all.

Step 3: Write NotesSearch.java

src/main/java/NotesSearch.java

import com.openai.client.OpenAIClient;
import com.openai.client.okhttp.OpenAIOkHttpClient;
import com.openai.models.embeddings.EmbeddingCreateParams;
import com.openai.models.embeddings.EmbeddingModel;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;

public class NotesSearch {

    static final List<String> NOTES = List.of(
        "The spare set for the front door is in the kitchen drawer.",
        "The dog needs his flea treatment on the 3rd.",
        "Mum's birthday is in April, she likes tulips.",
        "Renew the car insurance before it lapses.",
        "The standing desk needs a 4mm hex key to assemble.",
        "Pizza dough: 500g flour, 300ml water, rest overnight.",
        "The wifi password is taped under the router.",
        "Physio said ice the knee for twenty minutes."
    );

    // A note paired with how well it matched.
    record Match(String note, double score) {}

    // Reads OPENAI_API_KEY from the environment. Build one, share it.
    static final OpenAIClient CLIENT = OpenAIOkHttpClient.fromEnv();

    public static void main(String[] args) {
        // Embed all eight notes ONCE, before any searching happens.
        List<double[]> noteVectors = embed(NOTES);

        for (String question : List.of(
                "where did I put my keys?",
                "what do I cook on Friday?",
                "my knee is hurting")) {

            System.out.println("\nQ: " + question);

            // Only the question gets embedded at search time.
            double[] queryVector = embed(List.of(question)).get(0);

            List<Match> matches = new ArrayList<>();
            for (int i = 0; i < NOTES.size(); i++) {
                matches.add(new Match(NOTES.get(i), dot(noteVectors.get(i), queryVector)));
            }

            matches.stream()
                   .sorted(Comparator.comparingDouble(Match::score).reversed())
                   .limit(3)
                   .forEach(m -> System.out.printf("  %.3f  %s%n", m.score(), m.note()));
        }
    }

    /** The only method that changed from Part 1. One request for the whole list. */
    static List<double[]> embed(List<String> texts) {
        EmbeddingCreateParams params = EmbeddingCreateParams.builder()
                .model(EmbeddingModel.TEXT_EMBEDDING_3_SMALL)
                .inputOfArrayOfStrings(texts)
                .build();

        return CLIENT.embeddings().create(params).data().stream()
                .map(embedding -> normalize(embedding.embedding()))
                .toList();
    }

    /**
     * Copies the SDK's boxed list into a primitive array and scales it to
     * length 1. Taking a wildcard of Number means this keeps compiling whether
     * your SDK version returns Floats or Doubles.
     */
    static double[] normalize(List<? extends Number> values) {
        double[] vector = new double[values.size()];
        double sumOfSquares = 0.0;

        for (int i = 0; i < values.size(); i++) {
            vector[i] = values.get(i).doubleValue();
            sumOfSquares += vector[i] * vector[i];
        }

        double length = Math.sqrt(sumOfSquares);
        for (int i = 0; i < vector.length; i++) {
            vector[i] /= length;
        }
        return vector;
    }

    /** Cosine similarity, given both vectors are already length 1. */
    static double dot(double[] a, double[] b) {
        double total = 0.0;
        for (int i = 0; i < a.length; i++) {
            total += a[i] * b[i];
        }
        return total;
    }
}

Step 4: Run it

mvn -q compile exec:java -Dexec.mainClass=NotesSearch

Q: where did I put my keys?
  0.412  The spare set for the front door is in the kitchen drawer.
  0.287  The standing desk needs a 4mm hex key to assemble.
  0.169  The wifi password is taped under the router.

Same winner, real model, nothing hand-scored.

Those numbers are illustrative, not captured from a run — exact scores move between model versions, and any tutorial quoting them to three decimal places is showing you one moment in time. The ranking is the part that matters.

Why did the scores drop so much?

Part 1 scored around 0.99; a real model will score the same note far lower, typically somewhere in the 0.3 to 0.5 range. Nothing is broken.

In five hand-made dimensions two related things point almost the same way, so scores crowd up near 1. In 1,536 learned dimensions there’s far more room, and even closely related text only shares part of its direction. OpenAI embeddings usually land between about 0.1 and 0.6.

This is exactly why you shouldn’t hard-code a threshold like if (score > 0.8). What counts as good depends entirely on the model. Sort and take the top few; if you genuinely need a cut-off, measure it against your own data instead of copying a number out of a tutorial.

Understanding the code

1. OpenAIOkHttpClient.fromEnv()

   Reads OPENAI_API_KEY for you. There’s a builder if you need an explicit key, a custom base URL, or a proxy. The client is thread-safe and expensive to construct, so build one and share it. A static final field is fine here; in Spring you’d make it a @Bean.

2. embed(NOTES) runs once, before the loop

   The single most important line in the file. Embedding documents is the expensive part, so it happens once, up front; only the query is embedded per search.

   The classic beginner mistake is moving this inside the loop — or inside a request handler — which re-embeds every document on every search. Fine with eight notes, catastrophic with real data.

3. inputOfArrayOfStrings(texts)

   One request for all eight notes. Results come back in the order you sent them, so data().get(3) belongs to note 3. There’s an input(String) overload for a single item, but batching beats eight separate round trips every time.

4. EmbeddingModel.TEXT_EMBEDDING_3_SMALL

   A typed constant instead of a magic string, so a typo is a compile error rather than an HTTP 400 at runtime. Small thing; saves real time.

5. normalize(...) — and why Part 2 uses dot instead of cosineSimilarity

   Look back at Part 1’s cosineSimilarity: it divides the dot product by both vectors’ lengths. If both already have length 1, you’re dividing by 1 × 1: pointless work, repeated on every comparison.

   So we normalise once at embedding time and every later comparison becomes plain multiply-and-add. Same answer, and the cost moves from search time (every query) to storage time (once). Full explanation.

   It also copies the boxed List into a primitive double[], which matters more than it looks. Iterating a List<Float> unboxes on every access, and 1,536 elements across thousands of documents adds up fast.

6. List<? extends Number>

   Deliberate defensiveness. Different releases of this SDK have returned List<Float> and List<Double>, and Number.doubleValue() covers both. If you’d rather be explicit, check what your version returns and pin the type.

When it goes wrong

Everyone hits at least one of these, and none of them mean you’ve misunderstood embeddings.

OpenAIException: Missing credentials fromEnv() couldn’t find the variable. IDEs are the usual culprit. IntelliJ doesn’t inherit your shell environment, so setting it in .zshrc and hoping won’t work. Set it in Run → Edit Configurations → Environment variables.

NoSuchMethodError or ClassNotFoundException at runtime Almost always an OkHttp or Jackson version clash with something else on your classpath. Spring Boot pins its own versions and they may not match the SDK’s. Run mvn dependency:tree and look for two versions of the same artifact. Normal Java afternoon; not a sign you did anything wrong.

HTTP 429 on the very first call Reads like rate limiting, almost always means billing isn’t set up. Check your billing settings.

Every result scores about the same Either your query is much shorter or vaguer than your documents, or — the silent one — you embedded notes with one model and the query with another. Vectors from different models live in unrelated coordinate spaces; comparing them gives confident nonsense with no error message.

OutOfMemoryError once you scale up List<Float> costs roughly 16 bytes per number once you count object headers, against 8 for a double[] and 4 for a float[]. At 1,536 dimensions across a hundred thousand documents that’s the gap between a comfortable heap and a crash. Convert to primitive arrays early — which is what normalize does.

float[] is usually good enough

This guide uses double[] because it reads more naturally alongside Math.sqrt. In production, float[] halves your memory for essentially no loss in retrieval quality, since the models emit 32-bit floats anyway, so the extra precision was never real. On a large index that’s a straight 50% saving.

Where to go from here

How cosine similarity worksThe maths behind that loop, worked through with small numbers.

Chunking and vector databasesWhat changes when eight notes become eighty thousand documents.

Glossary and FAQEvery term here, defined plainly. RAG, ANN, dimensions, reranking.

The same app in C#Useful side by side: identical shape, different syntax.

If you’re using Spring Boot

Spring AI wraps all of this and handles configuration, retries and provider swapping:

<dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
</dependency>

@Autowired
private EmbeddingModel embeddingModel;

float[] vector = embeddingModel.embed("The spare set is in the kitchen drawer.");

Note it hands back a primitive float[] directly, which is the right call. Spring AI also fronts most vector stores behind a single VectorStore interface, so moving from in-memory to pgvector becomes a config change. Spring AI embeddings docs.

LangChain4j is the other well-established option and fits better if you’re not on Spring.

References

• openai-java on GitHub — official SDK source and current API
• com.openai:openai-java on Maven Central — current version
• OpenAI — Embeddings guide — models, dimensions, token limits
• Adoptium — free OpenJDK builds, if you need a JDK
• JEP 330: Launch Single-File Source-Code Programs — why java NotesSearch.java works with no compile step
• Spring AI — Embeddings API and Vector Databases
• LangChain4j documentation