Use Lucene for new and rebuilt indexes by ramonsmits · Pull Request #6035 · Particular/ServiceControl

ramonsmits · 2026-10-20T15:00:46Z

Since 6.21.1 IndexDeployment keeps a search engine that is set on an index itself. An index without a setting of its own was left alone, also when its definition changed. RavenDB then built the replacement with the database default. On every installation that ran 6.20.0 to 6.21.2 that default is Corax, so a definition change moved a Lucene index that had no setting of its own, for example after a reset under a Lucene default, back to Corax.

The rules are now:

A search engine set in our own definition wins.
A search engine set on the index on the server, or on its pending replacement, is kept. The definition then matches and RavenDB does not rebuild.
A new index, or an index that RavenDB rebuilds because its definition changed, uses Lucene, the application default. When the database setting is Lucene the index inherits it and gets no setting of its own. Otherwise Lucene is set on the index.
An unchanged index without a setting of its own is left alone. RavenDB treats a setting added to the index configuration as a change and rebuilds the index, also when the engine is the same.

Rule 3 means a Corax index without a setting moves to Lucene at the next definition change, without an operator action, and a Lucene index never falls back to Corax.

A definition change is detected with IndexDefinition.Compare on the maps, reduce, fields, additional sources and assemblies, the same differences for which RavenDB builds a side-by-side replacement.

Tests in IndexSetupTests cover a new index in a Lucene and in a Corax database, a changed Lucene index without a setting in a Corax database, a changed Corax index without a setting in a Lucene database, and an unchanged index without a setting that must not be rebuilt.

mauroservienti · 2026-10-21T07:35:00Z

+                var changes = existing == null ? IndexDefinitionCompareDifferences.None : RebuildChanges(existing, index, store);
+                var changed = changes != IndexDefinitionCompareDifferences.None;


The namings here are a bit hard to follow, I had to dive in to the code down below to understand what was happening

mauroservienti · 2026-10-21T07:38:33Z

+                // 3. A new index, or an index that RavenDB rebuilds because the definition changed, uses Lucene, the
+                // application default. It inherits Lucene when the database is set to Lucene. Otherwise Lucene is set on
+                // the index, so the index does not follow a database default of Corax.
+                if (existing == null || changed)
                {
-                    index.Configuration[StaticSearchEngineTypeKey] = nameof(SearchEngineType.Lucene);
-                    Logger.LogInformation("Index {IndexName} is created with the Lucene search engine", index.IndexName);
+                    var reason = existing == null ? "is created" : $"is rebuilt because its definition changed ({changes})";
+
+                    if (databaseDefaultIsLucene)
+                    {
+                        Logger.LogInformation("Index {IndexName} {Reason} and inherits the Lucene search engine from the database", index.IndexName, reason);
+                    }
+                    else
+                    {
+                        index.Configuration[StaticSearchEngineTypeKey] = nameof(SearchEngineType.Lucene);
+                        Logger.LogInformation("Index {IndexName} {Reason} with the Lucene search engine set on the index, the database setting is {DatabaseDefault}", index.IndexName, reason, databaseDefault ?? "not set");
+                    }
+
+                    continue;
                }


Isn't this in conflict with step 1 above?

If the definition contains an explicit search engine type, but the index was also changed, this will override the explicit search engine type in the index defition always to Lucene

Add an ADR for the decisions in #6034, #6035, #6036 and #6013: ServiceControl no longer writes search engine settings to existing databases, picks the engine of each new or rebuilt index, and reports and approves index definition changes. Link it from the engineering context index.

ramonsmits · 2026-10-21T14:40:14Z

+                // 3. A new index, or an index that RavenDB rebuilds because the definition changed, uses Lucene, the
+                // application default. It inherits Lucene when the database is set to Lucene. Otherwise Lucene is set on
+                // the index, so the index does not follow a database default of Corax.
+                if (existing == null || changed)


Use field to describe the expression usage

ramonsmits · 2026-10-21T14:41:28Z

+            | IndexDefinitionCompareDifferences.AdditionalSources
+            | IndexDefinitionCompareDifferences.AdditionalAssemblies;
+
+        static IndexDefinitionCompareDifferences RebuildChanges(IndexDefinition existingDefinition, AbstractIndexCreationTask index, IDocumentStore store)


Suggested change

static IndexDefinitionCompareDifferences RebuildChanges(IndexDefinition existingDefinition, AbstractIndexCreationTask index, IDocumentStore store)

static IndexDefinitionCompareDifferences GetDetectedChanges(IndexDefinition existingDefinition, AbstractIndexCreationTask index, IDocumentStore store)

ramonsmits · 2026-10-21T15:07:27Z

                // 1. A search engine set in our index definition always wins. It can be set through SearchEngineType or Configuration.
-                if (index.SearchEngineType.HasValue
-                    || (index.Configuration.TryGetValue(StaticSearchEngineTypeKey, out var configuredSearchEngineType) && !string.IsNullOrEmpty(configuredSearchEngineType)))
+                if (index.SearchEngineType.HasValue)


@mauroservienti as discussed in our sync I'm no longer convinced we want this.

If users want to override the search engine type, they apply it and the lock the index.

Then at SC start we detect the index is locked, and log a warn, if we detect the definition is incompatible we throw and we want the user to either UNLOCK so we can apply the new definition or the user manually applies the definition changes and locks it again so next start we detect the definition is identical (except for maybe the search engine).

ramonsmits added this pull request to stack #6037 October 20, 2026 15:00

internalautomation Bot assigned ramonsmits Oct 20, 2026

Use Lucene for new and rebuilt indexes

712053d

ramonsmits force-pushed the search-engine-rebuilt-indexes branch from c44bed6 to 712053d Compare October 20, 2026 15:02

ramonsmits requested a review from mauroservienti October 20, 2026 15:12

ramonsmits added the Bug label Oct 20, 2026

mauroservienti reviewed Oct 21, 2026

View reviewed changes

mauroservienti mentioned this pull request Oct 21, 2026

Record the RavenDB index search engine and rebuild decision #6043

Open

ramonsmits commented Oct 21, 2026

View reviewed changes

Navigation Menu

Repository navigation

Use Lucene for new and rebuilt indexes - #6035

Use Lucene for new and rebuilt indexes#6035
ramonsmits wants to merge 1 commit into
search-engine-database-settingfrom
search-engine-rebuilt-indexes

ramonsmits commented Oct 20, 2026 •

edited

Loading

Uh oh!

mauroservienti Oct 21, 2026

Uh oh!

mauroservienti Oct 21, 2026

Uh oh!

ramonsmits Oct 21, 2026

Uh oh!

ramonsmits Oct 21, 2026

Uh oh!

ramonsmits Oct 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		var changes = existing == null ? IndexDefinitionCompareDifferences.None : RebuildChanges(existing, index, store);
		var changed = changes != IndexDefinitionCompareDifferences.None;

	static IndexDefinitionCompareDifferences RebuildChanges(IndexDefinition existingDefinition, AbstractIndexCreationTask index, IDocumentStore store)
	static IndexDefinitionCompareDifferences GetDetectedChanges(IndexDefinition existingDefinition, AbstractIndexCreationTask index, IDocumentStore store)

Conversation

ramonsmits commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mauroservienti Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

mauroservienti Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

ramonsmits Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

ramonsmits Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

ramonsmits Oct 21, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

ramonsmits commented Oct 20, 2026 •

edited

Loading