Find the procedure while standing beside the machine
A technician arrives at a malfunctioning machine. Her tablet holds a manual, photographs from previous visits and maintenance recordings. She searches for “the fitting to check when pressure drops.” The relevant passage might use a different term, and a useful photo may have no caption. We will follow this hypothetical application throughout the article: what must it compute to present the right evidence?
Google released EmbeddingGemma 2 on October 6, 2026. This embedding model handles different content types, or modalities, to prepare them for retrieval. Its weights are published under Apache 2.0. This article examines its capabilities and the infrastructure around them; cited hardware measurements come from Google. Official announcement.
A vector helps retrieve the content
An embedding is a list of numbers, called a vector, computed from content. The model learns representations whose proximity helps retrieve relevant items. The application compares the query vector against vectors of previously indexed passages, then displays the highest-ranked results. This can connect ideas expressed with different wording. Semantic retrieval principles.
The technician initially receives selected passages or media. If the application also needs to write an explanation, it can pass those results to a generative model. Retrieving information before generation forms a retrieval-augmented generation pipeline, or RAG. EmbeddingGemma 2 contributes to evidence selection; it does not write the answer paragraph itself.
Ranking still estimates relevance. A similarity score of 0.8 is not an 80% probability that an answer is correct. A visually similar procedure might concern another equipment revision. Preserve the source and its version so the user can check what was retrieved.
A shared model with selectively loaded encoders
Advertised configurations range from approximately 270 million parameters for text and code to 740 million for text, vision and audio together. Each encoder transforms a modality into representations the shared network can process. Loading image and audio encoders for a text-manual index adds weights without helping that task. They can be disabled at load time. Documented configurations.
The network adapted from Gemma 4 uses bidirectional attention, drawing on information on both sides of a token, a text fragment or intermediate representation. Its outputs are mean-pooled, projected and normalized into a vector with 768 coordinates. Vision and audio feed into this common processing path, allowing their outputs to be compared with written questions. Transformers architecture documentation.
Training brings relevant pairs closer while distinguishing counterexamples. Google also distils from Gemini Embedding 2 before refining the space with contrastive objectives. That origin does not establish that the two models’ vectors are interchangeable. Encoder configurations of the same EmbeddingGemma 2 checkpoint do share the documented space. Technical report, developer guide.
Separate indexing from query execution
Before a site visit, the application extracts manual text and divides it into passages. For each passage, it retains the document identifier, section, version and content required for display. It then computes embeddings and stores them in an index. Preparation can be expensive, but it need not happen again for every question.
At search time, only the new question needs encoding. The index retrieves candidates. An optional second model can examine each passage together with the question to refine ranking, a stage called reranking. It has its own execution cost. Retrieve and rerank.
A corrected manual requires updating changed passages and removing obsolete ones. Changing the model or encoding recipe can require rebuilding the index. We recommend recording model ID, revision, task prefixes, vector dimensions and content hashes. Two lists of 768 numbers do not necessarily occupy the same space.
Add media without losing its references
A written query can be compared directly with maintenance-photo vectors. An application that previously captioned each photo and embedded that caption can test native visual encoding instead. It avoids an intermediate representation that may omit details, without guaranteeing improvement on every task. Google provides cross-modal retrieval demonstrations in AI Edge Gallery. Official examples.
A recording can contain speech and non-speech sounds. The audio encoder processes the signal directly, allowing retrieval over both with suitable clips and queries. Particular machine noises need evaluation on those sounds: an average result across general audio tasks does not establish performance for that specific use. Report audio evaluation.
The required output matters. Finding a scanned page and faithfully copying a part number are different objectives. Visual retrieval may select the page; text extraction or a dedicated reading stage remains useful when the application must quote exact characters. Likewise, audio embeddings help find a clip, while transcription supplies words to display.
The documented window is 8,192 tokens shared across modalities. The Transformers path samples video at one frame per second with a default maximum of 32 frames. Applications looking for short events must choose segmentation and sampling rather than assume every moment was observed. To locate a particular moment, index segments with their timestamps. Multimodal processing guide.
Where infrastructure can avoid work
Return to the tablet. Manuals alone need the text path; photographs require vision. If queries and the index stay local, content need not cross a network for retrieval. The cost moves onto the device: model storage, initial preparation and computation for new questions.
| Choice | Work potentially avoided | Cost or constraint to measure |
|---|---|---|
| Index before queries arrive | Re-encoding documents for every question | Initial preparation and updates |
| Encode media directly | Caption/transcribe then embed pipeline | Task quality and exact-output requirements |
| Load required modalities | Unused encoders’ weights and allocations | Available formats and backend support |
| Shorten vectors | Stored bytes and comparison arithmetic | Retrieval degradation and index overhead |
| Search on the device | Round trips and server-side retrieval load | Memory, energy and version distribution |
| Send fewer passages to generation | Input processing in the generative model | Omitting necessary evidence |
A single model family can simplify deployment, but file decoders, image preprocessing and updates still need maintenance. A shared encoder also does not establish perfect score comparability across media in your corpus. Check whether one content type dominates ranking at the expense of the others.
For tablets sharing identical manuals, rebuilding everything independently on each device can duplicate work. A hybrid design can prepare a common index on a server and distribute it while keeping private photographs indexed locally. Encoding recipes must remain compatible, and synchronization volume must be budgeted. This is an application design choice rather than an automatic model feature.
Better selection can change generation costs too
If the technician wants a written explanation, the application can send only selected passages with the question. Less input can shorten the generator’s prefill The opening phase of LLM inference: prompt tokens are processed in parallel to build the context state. That reuse raises arithmetic intensity and can make the phase compute-bound; the exact regime depends on the model, batch, context and backend. , the phase processing input tokens, and reduce its KV cache The stored key and value vectors an LLM has already computed for every token it has processed. It avoids recomputing attention over the whole history, at the cost of a memory footprint that grows with the context length. , which stores attention keys and values. Those savings depend on retaining the necessary evidence.
Total cost now includes question encoding, retrieval, optional reranking and generation. Adding a reranker can lengthen response time when its cost exceeds what it saves downstream. Conversely, accurate selection can avoid passing an entire long document to the generator. Measure the pipeline together, as discussed in our MLPerf RAG analysis.
Retrieved material must also be authorized for this user and intervention. Access rights, equipment revision and procedure validity belong in retrieval filters, rather than merely decorating the interface. High semantic similarity must not override those conditions.
Account separately for weights, runtime and index memory
Weight memory depends on storage format. As a calculation, 270 million parameters stored in four-byte FP32 occupy approximately 1.08 GB. That excludes temporary activations, libraries and decoded media. The full quantized LiteRT-LM package is advertised as a 485 MB download: file size is not running RAM usage. LiteRT-LM artifacts and formats.
The report gives peak memory figures of 191 MB for text only and 567 MB for full multimodality, using Q4_C on Pixel 11 Pro. Quantization uses shorter weight formats and a recipe adapted to that precision. Those measurements apply to that configuration; the LiteRT tables also use system metrics that can exclude accelerator memory. Establish the protocol before comparing them. Report quantization section, LiteRT measurement conditions.
On the tablet, successful model loading does not establish that the application can handle its largest passages while displaying media and running generation. The useful peak is the active system’s total. A persistent service avoids model initialization for every question, while retaining memory between requests.
Shorter vectors reduce a different memory budget
Matryoshka training makes prefixes of the full vector usable: 512, 256 or 128 coordinates instead of 768. Normalize again after truncation, and use equal dimensions for queries and documents. This changes stored representations; disabling encoders changes model weights. These are separate levers. Matryoshka Representation Learning.
| Dimensions | Bytes per FP32 vector | One million vectors, decimal GB |
|---|---|---|
| 768 | 3,072 | 3.072 |
| 512 | 2,048 | 2.048 |
| 256 | 1,024 | 1.024 |
| 128 | 512 | 0.512 |
The sixfold reduction from 768 to 128 applies to those raw bytes, not automatically to the entire database. Search structures add their own data. Quality also changes by task: EmbeddingGemma 2’s published results show greater degradation on some multimodal tasks at 128 dimensions. Start at 768 or 256 and compare relevant results rather than choosing maximum compression by default. Dimension-dependent evaluations.
Weight format, visual-token budget and output dimensions jointly define a configuration. Validate changes separately and then together: good quality with BF16 weights does not establish the quality of a quantized package that also reduces input detail and output dimensions.
The index becomes an infrastructure decision
For a few passages, a vector array is enough: compare the query against every row and sort the scores. With normalized vectors, their dot product equals cosine similarity, which compares their orientation. This avoids running the model on documents again, but comparison work increases with passage count and vector dimensions. Faiss distances and normalization.
As the corpus grows, approximate nearest-neighbour indexes can reduce the candidate search. HNSW, for example, arranges vectors in a neighbourhood graph. Links consume memory, while settings trade speed against recall, the proportion of relevant results retrieved. Faiss explicitly accounts for vector storage and structural overhead separately. Faiss index types.
Placement follows usage. A small local index works through network outages. A server shares computation and updates among users, but adds transfers and queues. Hybrid search can combine exact matches with semantic proximity: preserving a part number or GPU revision may matter more than meaning similarity.
Read benchmarks as evidence for a particular task
Code improves substantially in the reported evaluations. Multilingual text changes much less, and the English aggregate declines slightly against the first version. The following figures are Google’s reported comparison, not an evaluation of the maintenance corpus.
| Benchmark, mean per task | EmbeddingGemma 1 | EmbeddingGemma 2 | Qwen3-Embedding-0.6B |
|---|---|---|---|
| MTEB multilingual v2 | 61.15 | 61.36 | 64.34 |
| MTEB English v2 | 69.67 | 68.46 | 70.47 |
| MTEB Code | 68.76 | 78.68 | 75.42 |
The 9.92-point code improvement is approximately a 14.4% relative increase in that score. It is neither a speedup nor a general 14% RAG improvement. Qwen3-Embedding-0.6B remains a relevant text-only baseline; a generative Qwen assistant serves a different role. Report, Table 6, Qwen3-Embedding model card.
For our tablet, evaluate whether the correct procedure is retrieved, outdated revisions are excluded and ranking works in the required language. A multilingual average cannot answer each of those questions separately.
Mobile latency depends on inputs and backend
The LiteRT model card reports 8.3 ms for text on Pixel 11 Pro’s integrated TPU accelerator, using the 128-token text signature, the compiled path for that input size. Text plus vision takes 49 ms, with a 70-token vision budget. These are means over five iterations in that protocol, excluding complete application ingestion and search. An 8,192-token manual is not the same short input. LiteRT-LM protocol.
LiteRT and MediaPipe provide on-device execution paths. Universal Embedder produces vectors, while Semantic Retriever adds storage and retrieval functions. Support for an NPU Neural Processing Unit. An accelerator specialised in neural network operations. Depending on its design and software, it can support inference, training or both. Supported operators and numerical formats depend on the hardware. , an accelerator specialized for neural networks, depends on hardware and the compiled artifact. A mobile package’s available signatures can also differ from the budgets exposed through Transformers. Universal Embedder, Semantic Retriever.
A useful comparison therefore records exact input, format, backend and concurrency. Separate initial loading from queries with a resident model. Battery-life claims need energy measurements; a smaller parameter count alone does not provide joules per search.
Run a text search on a CPU
The companion example uses six short technical notes, a text-only FP32 model and normalized 256-dimensional vectors. It saves the index in a NumPy file and reloads it for queries. It calls no inference API and generates no answer. Initial package and weight downloads need Internet access; --offline subsequently uses cached model files.
Download the Python script, dependencies and English corpus, or the complete archive. Save them in a folder and use an isolated Python environment. The commands below use that environment’s Python.
python -m venv .venv
# Activate .venv, then install matching CPU packages:
python -m pip install torch==2.15.0 torchvision==0.29.2 --index-url https://download.pytorch.org/whl/cpu
python -m pip install -r requirements.txt
python local_search.py index --documents example_en.json --index index-en.npz
python local_search.py search --index index-en.npz --offline --query "How can I search my documents without sending them online?"In PowerShell, activate with .venv\Scripts\Activate.ps1; in a POSIX shell, use source .venv/bin/activate. The script pins a model revision, disables vision and audio, applies Document to passages and SearchQuery to queries. Its index manifest rejects incompatible query encoding. It also rejects passages exceeding context instead of silently truncating them.
Dependencies include Pillow and TorchVision: in the tested versions, the processor imports image and video preprocessing classes even with the corresponding encoders disabled. Those software dependencies remain, while loaded model weights are limited to the text path.
The program reports model loading, query encoding and ranking separately. These internal timings exclude Python imports and reading the index file. Each search command starts a process and reloads the model; a persistent service would retain it in memory. The script exposes the stages. For an end-to-end measurement, also time the complete command.
After caching model files, local verification built both indexes and ran the example questions offline. The local note about retrieval without a remote API ranks first in each language. Vectors are finite, normalized and 256-dimensional; an index declaring a different model revision is rejected. The verification report records these results and their scope.
Measure the application’s actual benefit
Before deployment, assemble questions with expected passages, including ambiguous cases and questions whose answers are absent from the corpus. Compare models with equal segmentation, filters and result counts. A relevant early candidate can still be demoted by reranking or misused during generation, so check each stage.
For infrastructure, report indexing time and volume, complete peak memory, update cost and retrieval latency under target load. The p50 is the median; p95 is the threshold below which 95% of measured latencies fall, with enough observations to make it useful. Keep cold starts separate from cached operation. Measure network traffic or energy too when those are the resources you intend to save.
We recommend starting with useful text retrieval and adding the modality the product actually lacks. Manuals, photographs and recordings do not necessarily need the same segmentation or precision. EmbeddingGemma 2 connects their representations; application architecture determines where computation runs, how often it repeats and whether the result helps its user.
Sources and method
Analysis current to November 12, 2026. The launch was October 6 and technical report v1 October 27. The examined model revision is f9972ac22e696a105c482202646979e33e4b5a1e. The 270M/740M sizes are public rounded figures; the report uses more precise counts. Architecture and API details are checked against Transformers documentation, the developer guide and multimodal guide.
Scores and hardware measurements are Google-reported results, without local reproduction of those benchmarks. Latencies and memory metrics retain the LiteRT-LM model card’s conditions. BF16, FP32 and mobile quantized recipes are not treated as equivalent. Retrieval mechanisms are documented by Sentence Transformers, its reranking guide, Faiss and vector normalization.
LeCompute functional verification used Windows, an Intel Core Ultra 9 275HX, Python 3.13.13, CPU PyTorch 2.15.0, CPU TorchVision 0.29.2, Sentence Transformers 6.3.0, Transformers 5.22.2, NumPy 2.5.4 and Pillow 13.0.0. The model runs in FP32 with four PyTorch threads over six notes per language. Verification covers indexes, offline queries and rejection of an incompatible index. It does not measure large-corpus quality, media encoding, peak memory, energy or mobile execution.
FP32 storage and the relative code-score increase are explicit calculations. The maintenance scenario and six demonstration notes are constructed educational examples, without a representative test set. Placement, indexing and evaluation choices are engineering recommendations to validate on the target corpus. The cover is an original conceptual illustration, with attribution and its generation prompt available beneath the image.
