# Lightwave Library RustSearch for Koha Version 0.6.0 · October 2026 development snapshot · Controlled pilot RustSearch is a native Rust/Tantivy search engine project for Koha. Its Perl adapters integrate search, query building and indexing with a loopback Rust service. Existing Koha and database services remain in the architecture. The tested integration baseline is Koha 26.05.04 on ARM64 Linux; a compatible plugin-engine integration patch may be required by the installed Koha version. ## Implemented and tested within defined profiles - Supported keyword/title/author/subject and ISBN search, native facets and pagination. Tantivy BM25 supplies term scoring for supported relevance-ranked searches. - Versioned Unicode normalization folds Latin diacritics consistently in indexed text and queries while preserving distinctions such as Japanese dakuten; it does not claim full multilingual relevance. - Defined MARC21 whole-field title/author profiles, ISBN normalization and Latin diacritic folding. - Separate bibliography, authority and bibliography-to-authority link indexes. - Preferred/alternate authority heading profiles, native authority-ID usage and Used-in links. - Candidate native authority duplicates with isolated installed-function acceptance. - Generation-aware queues and explicit coverage/readiness checks. - Rust-only routing when RustSearch is selected; supported requests retain the selected engine without automatic fallback to another engine. Koha remains the source of MARC metadata and embedded holdings. Visibility policy coverage is bounded, not universal. Custom MARC mappings need validation. ## Evidence snapshot The enlarged test catalog contains 70,422 bibliographic records, with 70,410 shared records used for the RustSearch–Zebra benchmark. There are 15 authority documents. The prior controlled pilot held 10,171 bibliographic records. The candidate passed 57 native ARM64 test executions across four targets (shared modules repeat across targets), 15 isolated installed-Koha duplicate assertions, and 21 local real-HTTP/client/coverage checks. The October 30 tuning engine passed 40 Rust tests; current offline verification passed 1,227 Perl assertions and 134 Python tests. Earlier test counts above describe separate historical authority targets. These counts are test evidence, not certification of complete Koha compatibility. The installed-function probe used process-local factory, CLI and coverage substitutions. The local HTTP harness used synthetic Koha DB/context/type/MARC objects and a shortened fault-test timeout. Neither is a complete browser/import or live factory/queue acceptance test. Duplicate matching mirrors the installed Koha function's lossy punctuation/zero-digit input filtering; it is not proof of exact equality of raw dates or arbitrary authority mappings. ## Architecture comparison - RustSearch: native Rust/Tantivy backend, no JVM required by that backend, library-focused MARC21 profiles, single-node focus and a controlled pilot scope. - Zebra: structured-record indexing/search engine with MARC/XML support and Z39.50 access; an established Koha backend with configuration-dependent behavior. - Elasticsearch: Java/Lucene search service with cluster, shard and replica capabilities; standard Lucene similarity uses BM25 by default. It is an established Koha backend with integration/mapping considerations. RustSearch uses Tantivy BM25 scoring; Elasticsearch uses Lucene relevance scoring (BM25 by default for standard similarity). Zebra has its own ranked-query behavior. The latest same-host three-engine comparison measures native adapters and shared Koha formatting, with hit-count and top-page overlap disclosure. These measurements do not establish a relevance ranking or a universal speedup or memory reduction. See the latest run’s separate methodology for the tested versions and settings. ## Local authority duplicate measurements AMD Ryzen 7 7735U, 16 logical CPUs, Linux x86-64 release build. One authority type holds the entire synthetic corpus. Nine timed sequential HTTP trials per case follow an initial sample. See benchmark.json and benchmark.csv for raw data. | Same-type authorities | Match median | Sampled p95 | | ---: | ---: | ---: | | 100 | 0.91 ms | 1.66 ms | | 1,000 | 5.76 ms | 6.69 ms | | 10,000 | 46.57 ms | 50.94 ms | | 50,000 | 250.49 ms | 433.06 ms | | 100,000 | 510.98 ms | 1003.24 ms | These measure authority duplicate detection, not general catalog queries. They exclude Perl/Koha/database overhead, concurrency and cold-disk behavior. The first sample is not a cold-disk test. Nine-trial p95 is not a production tail-latency estimate. Matching currently scans the stored headings selected by authority type. Whole-process peak RSS reached about 1.5 GiB across indexing and queries, not per request. ARM64 performance, concurrency and agreed latency budgets remain open. ## Current engineering work The bounded batch-aware worker is deployed on the AWS test host. Under the installed cron command (`--limit 500 --max-seconds 45`), a sample of 500 already-indexed records completed in one invocation: 500 completed successfully in about 41 seconds. A separate manual sample reindexed 1,000 records in ten batches of 100 in 90.525 seconds (~663 records/minute), including Koha shell/Perl startup, with a 300-second worker budget. Historical logs showed 34–40 records/minute before batching, but this was not a controlled same-window comparison. Full-catalog nightly reconciliation, sustained concurrency and cold-cache acceptance remain open. Broader advanced queries/limits, browse/scan, authority normalization/mappings, public visibility policies, multilingual relevance and complete production/browser/import acceptance also remain open. Distributed/federated search and Koha REST API investigation are deferred. Existing database-backed API contracts are preserved. A pilot should freeze matching artifacts, verify queue/index coverage, capture backups, prepare rollback, and validate actual library workflows. This brief is not a production installer or an instruction to change system preferences. ## References - https://www.indexdata.com/resources/software/zebra/ - https://www.elastic.co/docs/deploy-manage/deploy/self-managed/installing-elasticsearch - https://www.elastic.co/docs/deploy-manage/distributed-architecture/clusters-nodes-shards - https://koha-community.org/manual/latest/en/html/administrationpreferences.html#searchengine - https://docs.rs/tantivy/0.26.2/tantivy/ Contact: https://lightwavelibrary.com ## Configuration monitoring and cataloguing assistance The plugin’s Tools screen shows a green **Normal** indicator when its protected baseline matches the 10 monitored Koha integration files. It compares both file sizes and SHA-256 fingerprints, including changes that leave a file’s size unchanged. Checks run when the screen opens or refreshes. When monitored files change, an authorized administrator can use **Repair the Misconfiguration**. The protected server helper reapplies compatible integration changes, preserves unrelated Koha updates, validates the proposed files, and saves backups. Incompatible updates require administrator review; this feature does not establish compatibility with every future Koha release. **Recommend Fix** copies the selected record’s MARC text, indexing diagnostic, and a request for explained corrections into a prompt for the user’s preferred AI chat. A manual-copy option is available when the browser cannot access the clipboard. It sends nothing automatically and does not change MARC records; a cataloguer reviews suggested edits. ## Latest three-engine measurements The latest published run compares RustSearch, native Zebra, and Elasticsearch 8.19.23 through their native Koha adapters and the same Koha formatter. The host has two ARM64 vCPUs and approximately 7.6 GiB RAM. Elasticsearch uses ICU analysis, a fixed 1 GiB heap, one primary shard, and no replicas per Koha index. All three services are co-resident. The latest data and methodology are downloadable as `three-engine-comparison.json`, `three-engine-comparison.csv`, and `three-engine-methodology.md`. The panel checks for new published measurements every 60 seconds; it does not execute live benchmarks on a visitor’s behalf. A separate A/B/A/B tuning experiment reused Tantivy sidecar readers while checking atomic commit metadata on each health request. Twenty samples per workload in each of two phases per build gave 40 samples per build/workload. Identical result IDs were preserved. The measured health-check-plus-native-adapter-and-formatting path improved 17.3% for keyword, 19.7% for title, and 18.4% for author requests. Health checks improved about 99%; search-only adapter timings changed little. These are readiness-path probes, not general browser-speed claims. No search results, holdings, or readiness verdicts are memoized. Before server setup, missing integrations now show **Setup required** in amber. **Configuration overwritten** requires a recorded baseline showing a change. Uploading or enabling the KPZ alone does not install or activate the Rust engine. ## October 30 query-path tuning Immutable facet dictionary reuse reduced direct loopback query latency by about 90–96% in repeated A/B runs, with identical complete responses. Batched metadata loading retains native item embedding and OPAC policy. Full Koha formatted medians in the targeted follow-up improved about 7–14% on the sampled keyword/title/author/accent cases against the earlier snapshot; this is not a controlled end-to-end A/B. Four-client mixed throughput was 2.89 requests/s for RustSearch and 2.92 for Elasticsearch in a 40-request-per-engine bounded probe. Title and author remain faster with Elasticsearch; no overall winner is claimed. See search-tuning-report.md and three-engine-methodology.md. ## Guided in-place updates Configuration now provides Update RustSearch with compatible-update preflight and SSH apply commands. The administrator, rather than the web process, builds/tests the bundled engine and updates matching plugin/protected-worker files. The switch holds both indexing locks, backs up files, restarts the service, checks coverage and complete smoke-query fingerprints, and restores the original schedule. File rollback is automatic on validation failure; a protected rollback command is printed for the latest update. This release supports enabled 0.6.0/schema-6 updates without reindexing. Other migrations are refused. It does not establish full production or fresh-server acceptance.