-[Basic Start](#load-library) •
+[Basic Start](#load-library) •
[API Reference](#api-overview) •
[Encoder](doc/encoder.md) •
[Document Search](doc/document-search.md) •
@@ -98,124 +98,21 @@ Benchmarks:
The benchmark was measured in terms per seconds, higher values are better (except the test "Memory").
The memory value refers to the amount of memory which was additionally allocated during search.| Library | -Memory | -Query: Single | -Query: Multi | -Query: Large | -Query: Not Found | -
|---|---|---|---|---|---|
| flexsearch | -16 | -50955718 | -11912730 | -13981110 | -51706499 | -
| jsii | -2188 | -13847 | -949559 | -1635959 | -3730307 | -
| wade | -980 | -60473 | -443214 | -419152 | -1239372 | -
| js-search | -237 | -22982 | -383775 | -426609 | -994803 | -
| minisearch | -4777 | -30589 | -191657 | -5849 | -304233 | -
| orama | -5355 | -29445 | -170231 | -4454 | -225491 | -
| elasticlunr | -3073 | -14326 | -48558 | -101206 | -95840 | -
| lunr | -2443 | -11527 | -51476 | -88858 | -103386 | -
| ufuzzy | -13754 | -2799 | -7788 | -58544 | -9557 | -
| bm25 | -33963 | -3903 | -4777 | -12657 | -12471 | -
| fuzzysearch | -300147 | -148 | -229 | -455 | -276 | -
| fuse | -247107 | -422 | -321 | -337 | -329 | -
| Feature | -flexsearch.bundle.js | -flexsearch.compact.js | -flexsearch.light.js | -
| - Presets - | -✓ | -✓ | -✓ | -
| - Async Processing - | -✓ | -✓ | -- | -
| - Workers (Web + Node.js) - | -✓ | -- | -- | -
| - Context Search - | -✓ | -✓ | -✓ | -
| - Document Search - | -✓ | -✓ | -- | -
| - Document Datastore - | -✓ | -✓ | -- | -
| - Partial Matching - | -✓ | -✓ | -✓ | -
| - Auto-Balanced Cache by Popularity/Last Queries - | -✓ | -✓ | -- | -
| - Tag Search - | -✓ | -✓ | -- | -
| - Suggestions - | -✓ | -✓ | -✓ | -
| - Phonetic Search (Fuzzy Search) - | -✓ | -✓ | -- | -
| Encoder | -✓ | -✓ | -✓ | -
| Export / Import Indexes | -✓ | -✓ | -- | -
| Resolver | -✓ | -- | -- | -
| Result Highlighting | -✓ | -✓ | -- | -
| Persistent Index (IndexedDB) | -✓ | -- | -- | -
| File Size (gzip) | -16.3 kb | -11.4 kb | -4.5 kb | -
| Option | -Values | -Description | -Default | -
| preset | -
- "memory" - "performance" - "match" - "score" - "default" - |
-
- The configuration profile as a shortcut or as a base for your custom settings. - |
- "default" | -
| tokenize | -
- "strict" / "exact" - "tolerant" - "forward" - "reverse" / "bidirectional - "full" - |
- - Indicates how terms should be indexed by tokenization. - | -"strict" | -
| resolution | -- Number - | -Sets the scoring resolution | -9 | -
| encoder | -
- new Encoder(options) - Charset.Exact - Charset.Default - Charset.Normalize - Charset.LatinBalance - Charset.LatinAdvanced - Charset.LatinExtra - Charset.LatinSoundex - Charset.CJK - false - |
- Choose one of the built-in encoder Read more about Encoder |
- "default" | -
| encode | -- function(string) => string[] - | -Pass a custom encoding function Read more about Encoder |
- "default" | -
| context | -
- Boolean - Context Options - |
- Enable/Disable context index. When passing "true" as a value will use the defaults for the context. | -false | -
| cache | -
- Boolean - Number - |
- Enable/Disable and/or set capacity of cached entries. The cache automatically balance stored entries related to their popularity. |
- false | -
| fastupdate | -- Boolean - | -Additionally add a fastupdate index which boost any replace/update/remove task to a high performance level by also increasing index size by ~30%. | -false | -
| priority | -- Number - | -Sets the task execution priority (1 low priority - 9 high priority) when using the async methods | -4 | -
| score | -- function(string) => number - | -Use a custom score function | -- |
| keystore | -- Number - | -Increase available size for In-Memory-Index by additionally using uniform balanced registers (Keystore). You can apply values from 1 to 64. | -false | -
| - Persistent Options: - | -|||
| db | -- StorageInterface - | -Pass an instance of a persistent adapter | -- |
| commit | -- Boolean - | -When disabled any changes won't commit, instead it needs calling index.commit() manually to make modifications to the index (add, update, remove) persistent. |
- true | -
| Option | -Values | -Description | -Default | -
| limit | -number | -Sets the limit of results | -100 | -
| offset | -number | -Apply offset (skip items) | -0 | -
| resolution | -number | -Limit the resolution (score) of the results | -- |
| suggest | -Boolean | -Enables Suggestions in results | -false | -
| cache | -Boolean | -Use a Query Cache | -false | -
| resolve | -Boolean | -When set to false, an instance of a Resolver is returned to apply further operations |
- true | -
| Option | -Description | -Example | -Memory Factor (n = length of term) | -
"strict""exact""default" |
- index the full term | -foobar |
- 1 | -
"forward" |
- index term in forward direction (supports right-to-left by Index option rtl: true) |
- foobarfoobar |
- n | -
"reverse""bidirectional" |
- index term in both directions | -foobarfoobarfoob arfo obar |
- 2n - 1 | -
"tolerant" |
- index the full term by also being tolerant against typos like swapped letters and missing letters | -foobrafoboarfoobrfooba |
- 2(n - 2) + 2 | -
"full" |
- index every consecutive partial | -foobarf oobar |
- n * (n - 1) | -
| Option | -Description | -Charset Type | -Compression Ratio | -
Exact |
- Bypass encoding and take exact input | -Universal (multi-lang) | -0% | -
NormalizeDefault |
- Case in-sensitive encoding Charset normalization Letter deduplication |
- Universal (multi-lang) | -~ 7% | -
LatinBalance |
- Case in-sensitive encoding Charset normalization Letter deduplication Phonetic basic transformation |
- Latin | -~ 30% | -
LatinAdvanced |
- Case in-sensitive encoding Charset normalization Letter deduplication Phonetic advanced transformation |
- Latin | -~ 45% | -
LatinExtra |
- Case in-sensitive encoding Charset normalization Letter deduplication Soundex-like transformation |
- Latin | -~ 60% | -
LatinSoundex |
- Full Soundex transformation | -Latin | -~ 70% | -
function(str) => [str] |
- Pass a custom encoding function to the Encoder |
- - | - |
| Encoder: | -Exact |
- Normalize (Default) |
- LatinBalance |
- LatinAdvanced |
- LatinExtra |
- LatinSoundex |
-
|---|---|---|---|---|---|---|
| Index Size | -3.1 Mb | -1.9 Mb | -1.7 Mb | -1.6 Mb | -1.1 Mb | -0.7 Mb | -
| Struldbrugs | -✓ | -✓ | -✓ | -✓ | -✓ | -✓ | -
| strũlldbrųĝgs | -- | ✓ | -✓ | -✓ | -✓ | -✓ | -
| strultbrooks | -- | - | ✓ | -✓ | -✓ | -✓ | -
| shtruhldbrohkz | -- | - | - | ✓ | -✓ | -✓ | -
| zdroltbrykz | -- | - | - | - | ✓ | -✓ | -
| struhlbrogger | -- | - | - | - | - | ✓ | -
| Option | -Values | -Description | -Default | -
| resolution | -- Number - | -Sets the scoring resolution for the context. | -3 | -
| depth |
-
- false - Number - |
- Enable/Disable context index and also sets the maximum initial distance of related terms. | -1 | -
| bidirectional | -- Boolean - | -If enabled the context direction (aka "context chain") can move bidirectional. You should ony disable this options when you need a more exact match with fewer results. | -true | -
| Flag | -Values | -Info | -
Feature Flags |
- ||
| SUPPORT_WORKER | -true, false | -Worker Indexes | -
| SUPPORT_ENCODER | -true, false | -When not included you'll need to pass a custom encode method when creating an index |
-
| SUPPORT_CHARSET | -true, false | -Includes: LatinBalance, LatinAdvanced, LatinExtra, LatinSoundex |
-
| SUPPORT_CACHE | -true, false | -Support for index.searchCache() |
-
| SUPPORT_ASYNC | -true, false | -The async version of index standard methods | -
| SUPPORT_STORE | -true, false | -Document Datastore | -
| SUPPORT_SUGGESTION | -true, false | -Use the option suggestions when searching |
-
| SUPPORT_SERIALIZE | -true, false | -Export / Import / Serialize Index | -
| SUPPORT_DOCUMENT | -true, false | -Document Indexes | -
| SUPPORT_TAGS | -true, false | -Tag-Search | -
| SUPPORT_PERSISTENT | -true, false | -Use any of the persistent indexes | -
| SUPPORT_KEYSTORE | -true, false | -Extended size for InMemory indexes | -
| SUPPORT_RESOLVER | -true, false | -Apply complex queries by chaining boolean operations | -
| SUPPORT_HIGHLIGHTING | -true, false | -Result Highlighting for Document-Search (also requires SUPPORT_STORE) |
-
Compiler Flags |
- ||
| DEBUG | -true, false | -Apply common checks and throw errors more frequently, output debug information and helpful hints to the console | -
| RELEASE | -custom custom.module |
- Choose build schema: custom = Legacy Browser (window.FlexSearch), custom.module = ES6 Modules (ESM) |
-
| POLYFILL | -true, false | -Include Polyfills (based on LANGUAGE_OUT) |
-
| PROFILER | -true, false | -Just used for automatic performance tests | -
| LANGUAGE_OUT | -ECMASCRIPT3 ECMASCRIPT5 ECMASCRIPT_2015 ECMASCRIPT_2016 ECMASCRIPT_2017 ECMASCRIPT_2018 ECMASCRIPT_2019 ECMASCRIPT_2020 ECMASCRIPT_2021 ECMASCRIPT_2022 ECMASCRIPT_NEXT STABLE |
- Target language | -
| Option | -Values | -Description | -Default | -
document |
- Document Descriptor | -Includes any specific information about how the document data should be indexed | -(mandatory) | -
worker |
- Boolean String |
- Enable a worker distributed model. Read more about here: Worker Index | -false |
-
| Option | -Values | -Description | -Default | -
indexfield |
- String Array<String> Array<SearchOptions> |
- Sets the document fields which should be searched. When no field is set, all fields will be searched. Custom options per field are also supported. | -- |
tag |
- Object<field:tag> | -Sets the document fields which should be searched. When no field is set, all fields will be searched. Custom options per field are also supported. | -- |
enrich |
- Boolean | -Enrich IDs from the results with the corresponding documents. | -false |
-
highlight |
-
- Highlighting Options - String - |
- Highlight query matches in the result (for Document Indexes only) | -false |
-
merge |
- Boolean | -Merge multiple fields in resultset into one and group results per ID | -false |
-
pluck |
- String | -Pick and apply search to just one field and return a flat result representation | -false |
-
| Option | -Values | -Description | -Default | -
id |
- String | -- | "id" |
-
index |
- String Array<String> Array<FieldOptions> |
- - | - |
tag |
- String Array<String> Array<FieldOptions> |
- - | - |
store |
- Boolean String Array<String> Array<FieldOptions> |
- - | false |
-
| Option | -Values | -Description | -Default | -
field |
- String | -The field name (colon seperated syntax) | -(mandatory) | -
filter |
- Function | -- | - |
custom |
- Function | -- | - |
| Option | -Values | -Description | -Default | -
| You can just choose one of those 3 options: | -|||
include |
- - Encoder Split Options - | -Define which of the string contents should be included (inclusion properties defaults to false) | -{ letter: true, number: true } | -
exclude |
- - Encoder Split Options - | -Define which of the string contents should be excluded (exclusion properties defaults to true) | -false | -
split |
-
- false - RegExp - String - Encoder Split Options - |
- - The expression used to split the content into terms - | -→ include { letter: true, number: true } | -
| Other options: | -|||
dedupe |
- - Boolean - | -Deduplicate consecutive letters, e.g. "missing" to "mising" | -true | -
numeric |
- - Boolean - | -By default, the extended numeric support (Triplets) inherits from chosen Encoder Split Options. You probably might want to disable Triplets to get a more exact result (fewer entries) in some cases. | -true | -
minlength |
- - Number - | -Set the minimum term length which should be added to the index. This limit does not apply to the forward tokenizer. You still get results when just typing "f" on a term "flexsearch" when e.g. minlength: 4 was used. |
- 1 | -
maxlength |
- - Number - | -Set the maximum term length which should be added to the index. Larger content will drop. | -1 | -
rtl |
- - Boolean - | -Force Right-To-Left encoding (you should just apply this when the string content was not already encoded as RTL) | -false | -
normalize |
-
- true enable normalization (default)- false disable normalization- function(str) => str custom function
- |
- The normalization stage will apply basic charset normalization e.g. by replacing "é" to "e" | -true | -
prepare |
-
- function(str) => str custom function
- |
- The preparation stage is a custom function direct followed when normalization was done | -false | -
finalize |
-
- function([str]) => [str] custom function
- |
- The finalization stage is a custom function executed at the last task in the encoding pipeline (here it gets an array of tokens and need to return an array of tokens) | -false | -
filter |
-
- Set(["and", "to", "be"])- function(str) => bool custom function
- encoder.addFilter("and")
- |
- Stop-word filter is like a blacklist of words to be filtered out from indexing at all (e.g. "and", "to" or "be"). This is also very useful when using Context Search | -false | -
stemmer |
-
- Map([["ing", ""], ["ies", "y"]])
- encoder.addStemmer("ing", "")
- |
- Stemmer will normalize several linguistic mutations of the same word (e.g. "run" and "running", or "property" and "properties"). This is also very useful when using Context Search | -false | -
mapper |
-
- Map([["é", "e"], ["ß", "ss"]])
- encoder.addMapper("é", "e")
- |
- Mapper will replace a single char (e.g. "é" into "e") | -false | -
matcher |
-
- Map([["and", "&"], ["usd", "$"]])
- encoder.addMatcher("and", "&")
- |
- Matcher will do same as Mapper but instead of single chars it will replace char sequences | -false | -
replacer |
-
- [/[^a-z0-9]/g, "", /([^aeo])h(.)/g, "$1$2"])
- encoder.addReplacer(/[^a-z0-9]/g, "")
- |
- Replacer takes custom regular expressions and couldn't get optimized in the same way as Mapper or Matcher. You should take this as the last option when no other replacement can do the same. | -false | -
cache |
- - Boolean - | -In some very rare situations (large consecutive content with high cardinality) it might be useful to disable the internal event-loop-cache | -true | -
| Option | -Values | -Description | -Default | -
letter |
- - Boolean - | -Toggle inclusion of letters on/off | -true | -
number |
- - Boolean - | -Toggle inclusion of numerics on/off | -true | -
symbol |
- - Boolean - | -Toggle inclusion of symbols on/off | -false | -
punctuation |
- - Boolean - | -- Toggle inclusion of punctuation on/off - | -false | -
control |
- - Boolean - | -Toggle inclusion of control chars on/off | -false | -
char |
-
- String - Array[String] - |
- Toggle inclusion of specific chars on/off | -false | -
| Store | -Add | -Search 1 | -Search N | -Replace | -Remove | -Not Found | -Scaling | -
|---|---|---|---|---|---|---|---|
| - | terms per sec | -terms per sec | -terms per sec | -terms per sec | -terms per sec | -terms per sec | -- |
| IndexedDB | -123,298 | -83,823 | -62,370 | -57,410 | -171,053 | -425,744 | -No | -
| Redis | -1,566,091 | -201,534 | -859,463 | -117,013 | -129,595 | -875,526 | -Yes | -
| Sqlite | -269,812 | -29,627 | -129,735 | -174,445 | -1,406,553 | -122,566 | -No | -
| Postgres | -354,894 | -24,329 | -76,189 | -324,546 | -3,702,647 | -50,305 | -Yes | -
| MongoDB | -515,938 | -19,684 | -81,558 | -243,353 | -485,192 | -67,751 | -Yes | -
| Clickhouse | -1,436,992 | -11,507 | -22,196 | -931,026 | -3,276,847 | -16,644 | -Yes | -
| Method | -Description | -Return | -
- .and(options,...)- .or(options,...)- .not(options,...)- .xor(options,...)
- |
- Apply an operation | -Returns a Resolver when resolve was not set to false within the options, otherwise it returns the result (or promise in async context). |
-
- .limit(number)- .offset(number)- .boost(number)
- |
- Apply boost, limit and offset to the result | -Returns a Resolver |
-
- .resolve(options)
- |
- Resolve results | -Returns the final result or promise in async context (can't be executed twice) | -
| Option | -Values | -Description | -Default | -
| Resolver Task Options: | -|||
query |
- - String - | -The search query | -- |
index |
-
- Index- Document
- |
- Assign the index where the query should be applied to | -- |
suggest |
- - Boolean - | -Enables suggestions in results | -false | -
boost |
- - Number - | -Boost or reduce the score of this query | -0 | -
async |
- - Boolean - | -Use a parallel processing workflow | -false | -
queue |
- - Boolean - | -Use a queued processing workflow | -false | -
- and- or- not- xor- |
- - Array<ResolverOptions> - | -Apply nested queries | -- |
resolve |
- - Boolean - | -
- Resolve the result immediately or not. When set to true all final resolve options are also allowed and there can't exist any further resolver operations.
- |
- false | -
| Document Resolver Options: | -|||
fieldpluck |
- - String - | -Select the Document field on which the query should apply to. | -- |
| Final Resolve Options: | -|||
enrich |
- - Boolean - | -Enrich IDs from the results with the corresponding documents (for Document Indexes only) | -true | -
highlight |
-
- Highlighting Options - String - |
- Highlight query matches in the result (for Document Indexes only) | -- |
limit |
- - Number - | -Sets the limit of results | -100 | -
offset |
- - Boolean - | -Apply offset (skip items) | -0 | -
@@ -439,7 +298,7 @@ function CustomResolver(raw){
return output;
}
-const result = index.search("a short query", {
+const result = index.search("a short query", {
resolve: CustomResolver
});
```
diff --git a/doc/result-highlighting.md b/doc/result-highlighting.md
index c094b697..a89943bd 100644
--- a/doc/result-highlighting.md
+++ b/doc/result-highlighting.md
@@ -10,7 +10,7 @@ Alternatively you can simply upgrade id-content-pairs to a flat document when ca
// 1. create the document index
const index = new Document({
document: {
- // using store is required
+ // using store is required
store: true,
index: [{
field: "title",
@@ -33,7 +33,7 @@ index.add({
// 3. perform a query
const result = index.search({
query: "karmen or clown or not found",
- // also get results when query has no exact match
+ // also get results when query has no exact match
suggest: true,
// use highlighting options or pass a template, where $1 is
// a placeholder for the matched partial
@@ -60,116 +60,26 @@ There are several options to customize result highlighting.
### Highlighting Options
-| Option | -Values | -Description | -Default | -
template |
- - String - | -The template to be applied on matches (e.g. "<b>$1</b>"), where $1 is a placeholder for the matched partial |
- (mandatory) | -
boundary |
-
- Boundary Options - Number - |
- Limit the total length of highlighted content (add ellipsis by default). The template markup does not stack to the total length. | -false |
-
ellipsis |
-
- Ellipsis Options - Boolean - String - |
- - Define a custom ellipsis or disable - | -"..." |
-
merge |
- +| | | | | +|----------|--------|-------------------------------------------------------------------------------------------------------------------|-------------| +| Option | Values | Description | Default | +| template | String | The template to be applied on matches (e.g. "<b>$1</b>"), where $1 is a placeholder for the matched partial | (mandatory) | +| boundary | Boundary Options + Number | Limit the total length of highlighted content (add ellipsis by default). The template markup does not stack to the total length. | false | +| ellipsis | Ellipsis Options Boolean - | -Wrap consecutive matches by just a single template | -false |
-
clip |
- - Boolean - | -Allow to clip terms | -true |
-
| Boundary Options | -|||
boundary.total |
- - Number - | -Limit the total length of highlighted content | -false |
-
boundary.before |
- - Number - | -Limit the length of content before highlighted parts | -(auto) | -
boundary.after |
- - Number - | -Limit the length of content after highlighted parts | -(auto) | -
| Ellipsis Options | -|||
ellipsis.template |
- - String - | -The template to be applied on ellipsis (e.g. "<i>$1</i>"), where $1 is a placeholder for the ellipsis |
- (mandatory) | -
ellipsis.pattern |
-
- Boolean - String - |
- - Define a custom ellipsis or disable - | -"..." |
-
| Option | -Values | -Description | -
| config | -String | -Either the absolute URL to the config file when used in Browser context (should match the Same-Origin-Policy) or the filepath to the configuration file when used in Node.js context | -
| export | -function | -The export handler function. Read more about Export | -
| import | -function | -The export handler function. Read more about Import | -