TINKERPOP-3282 TinkerStorageGraph

CHANGELOG.asciidoc

@@ -27,7 +27,9 @@ image::https://raw.githubusercontent.com/apache/tinkerpop/master/docs/static/ima
 
 * Fixed `gremlin-go` to report a malformed or truncated GraphBinary response as a deserialization error rather than a bare decoder message.
 * Made `TinkerGraph` an interface and renamed the in-memory implementation to `TinkerMemoryGraph`; `TinkerGraph.open()` and `gremlin.graph=...TinkerGraph` behave as before. *(breaking)*
-* Renamed `TinkerTransactionGraph` to `TinkerStorageGraph`. *(breaking)*
+* Renamed `TinkerTransactionGraph` to `TinkerStorageGraph`.
+* Added a pluggable storage layer to `TinkerStorageGraph` that durably persists each committed transaction to disk.
+* Removed automatic persistence from `TinkerMemoryGraph`, which is now purely in-memory and ignores `gremlin.tinkergraph.graphLocation`/`graphFormat`. Use `TinkerStorageGraph` for durability or `g.io()` for interchange.
 
 [[release-4-0-0-beta-3]]
 === TinkerPop 4.0.0-beta.3 (July 20, 2026)

docs/src/upgrade/release-4.x.x.asciidoc

@@ -71,6 +71,67 @@ which affects providers that extended them.
 
 See: link:https://lists.apache.org/thread/2zt62kvfssh6xz5vnf2lk1g7cstq9vod[DISCUSS thread]
 
+==== TinkerGraph Disk Storage
+
+TinkerGraph has always been a pure in-memory graph with some limited capability to write to disk via the `io()` step
+or through that same function on close. This approach kept TinkerGraph simple, but limited its functionality for certain
+use cases. For TinkerPop 4, TinkerGraph offers a basic file-based persistence option tied to its transaction capability.
+This new feature not only gives TinkerGraph another operational dimension for production use cases, but also makes its
+transactional capabilities have more purpose.
+
+The feature is offered via `TinkerStorageGraph` which gained the optional disk storage hinted by its rename. A storage
+engine is selected with the new `gremlin.tinkergraph.storage` configuration key, and
+`gremlin.tinkergraph.storage.directory` names the directory that holds the durable data. When a storage engine is
+configured, each committed transaction is durably written to disk and the graph is rebuilt from that data when it is
+opened again, so a graph survives a restart of the JVM.
+
+The reference engine, `graphbinary`, records committed transactions as an append-only log serialized with GraphBinary
+and folds that log into a compact snapshot on close. The storage layer is pluggable: the value of the storage key may
+also be the fully-qualified class name of a custom engine, following the same convention as the `IdManager` selection.
+
+[source,groovy]
+----
+conf = new BaseConfiguration()
+conf.setProperty('gremlin.graph', 'org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerStorageGraph')
+conf.setProperty('gremlin.tinkergraph.storage', 'graphbinary')
+conf.setProperty('gremlin.tinkergraph.storage.directory', '/data/mygraph')
+
+graph = TinkerStorageGraph.open(conf)
+g = traversal().with(graph)
+g.addV('person').property('name','marko').iterate()
+g.tx().commit()
+graph.close()
+
+// reopening the same location restores the committed data
+graph = TinkerStorageGraph.open(conf)
+g = traversal().with(graph)
+c = g.V().count().next()
+----
+
+The in-memory `TinkerMemoryGraph` no longer persists to disk. Earlier versions of TinkerGraph would automatically read
+from `gremlin.tinkergraph.graphLocation` on open and write back to it on close, using the `gremlin.tinkergraph.graphFormat`
+interchange format. That automatic behavior is removed, and `TinkerMemoryGraph` now ignores both keys and reports
+`FEATURE_PERSISTENCE` as `false`. Note that `gremlin.tinkergraph.graphLocation` named a *file* to be read and written
+in an interchange format, while the new `gremlin.tinkergraph.storage.directory` names a *directory* managed by a
+storage engine. They are deliberately different keys because they mean different things. Durable persistence is the
+responsibility of `TinkerStorageGraph` and its storage engine, while moving data in and out of any graph in an
+interchange format remains the job of the `io()` step:
+
+[source,groovy]
+----
+// interchange, on demand, for any graph
+g.io('/tmp/graph.kryo').write().iterate()
+g.io('/tmp/graph.kryo').read().iterate()
+----
+
+Configurations that previously relied on the in-memory graph loading itself from `graphLocation` on open must either
+call `io().read()` explicitly or switch to `TinkerStorageGraph` with a storage engine. Both
+`gremlin.tinkergraph.graphLocation` and `gremlin.tinkergraph.graphFormat` are retired as TinkerGraph settings; no graph
+implementation reads either one. They survive only in Gremlin Server's `SimpleAuthenticator`, which uses them to read
+its credential store at startup.
+
+See: <<tinkergraph-gremlin,TinkerGraph>>
+
 === Upgrading for Providers
 
 ==== Graph System Providers

gremlin-console/conf/tinkergraph-gryo.properties → gremlin-console/conf/tinkergraph-storage.properties

@@ -15,7 +15,12 @@
 # specific language governing permissions and limitations
 # under the License.
 
-gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
+# Sample configuration for a durable, transactional TinkerStorageGraph. Opening a graph with this configuration
+# (for example via GraphFactory) produces a TinkerStorageGraph that persists committed transactions to the
+# configured storage directory.
+gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerStorageGraph
 
-gremlin.tinkergraph.graphFormat=gryo
-gremlin.tinkergraph.graphLocation=/tmp/tinkergraph.kryo
+# built-in storage engine; without this key the graph is transactional but in-memory only
+gremlin.tinkergraph.storage=graphbinary
+# directory holding the durable data (created if absent; a location may be opened by only one graph at a time)
+gremlin.tinkergraph.storage.directory=/tmp/tinkergraph

gremlin-server/conf/tinkergraph-credentials.properties

@@ -16,5 +16,9 @@
 # under the License.
 gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph
 gremlin.tinkergraph.vertexIdManager=LONG
+# This credential store is an in-memory TinkerGraph. TinkerGraph no longer auto-loads from disk on open, so
+# SimpleAuthenticator reads the store explicitly from graphLocation using graphFormat at startup. Both keys are
+# retired as TinkerGraph settings and are honoured only for this load; they name an interchange FILE, not the
+# storage DIRECTORY that a durable TinkerStorageGraph configures with gremlin.tinkergraph.storage.directory.
 gremlin.tinkergraph.graphLocation=data/credentials.kryo
 gremlin.tinkergraph.graphFormat=gryo

gremlin-server/conf/tinkerstoragegraph-persistent.properties

@@ -0,0 +1,40 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#   http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# Sample configuration for a durable, transactional TinkerStorageGraph on Gremlin Server. Reference it from the
+# server's "graphs" block, for example in gremlin-server-transaction.yaml:
+#   graphs: { graph: conf/tinkerstoragegraph-persistent.properties }
+gremlin.graph=org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerStorageGraph
+
+# Enables durable persistence. "graphbinary" is the built-in storage engine; the fully qualified class name of a
+# custom TinkerStorage implementation may be used instead. Without this key the graph is transactional but in-memory.
+gremlin.tinkergraph.storage=graphbinary
+# Directory holding the durable data. Required when a storage engine is set. It is created if absent and may be
+# opened by only one graph at a time (single writer, whether in this JVM or another process).
+gremlin.tinkergraph.storage.directory=/tmp/tinkerstoragegraph
+
+gremlin.tinkergraph.vertexIdManager=LONG
+gremlin.tinkergraph.edgeIdManager=LONG
+gremlin.tinkergraph.vertexPropertyIdManager=LONG
+
+# Optional storage tuning (defaults shown, uncomment to change):
+# "commit" forces each transaction to disk (survives OS crash/power loss); "os" is faster but survives only a JVM crash
+#gremlin.tinkergraph.storage.sync=commit
+# auto-compact the append log once it grows past this many bytes (default 67108864 = 64MB; 0 disables)
+#gremlin.tinkergraph.storage.compactThreshold=67108864
+# persist auto-generated vertex-property ids across a reopen (default false)
+#gremlin.tinkergraph.storage.preserveVertexPropertyIds=false

gremlin-server/src/main/java/org/apache/tinkerpop/gremlin/server/auth/SimpleAuthenticator.java

@@ -21,14 +21,17 @@
 import org.apache.tinkerpop.gremlin.groovy.jsr223.dsl.credential.CredentialTraversal;
 import org.apache.tinkerpop.gremlin.groovy.jsr223.dsl.credential.CredentialTraversalDsl;
 import org.apache.tinkerpop.gremlin.groovy.jsr223.dsl.credential.CredentialTraversalSource;
+import org.apache.commons.configuration2.Configuration;
 import org.apache.tinkerpop.gremlin.structure.Graph;
 import org.apache.tinkerpop.gremlin.structure.Vertex;
+import org.apache.tinkerpop.gremlin.structure.io.IoCore;
 import org.apache.tinkerpop.gremlin.structure.util.GraphFactory;
 import org.apache.tinkerpop.gremlin.tinkergraph.structure.TinkerGraph;
 import org.mindrot.jbcrypt.BCrypt;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
+import java.io.File;
 import java.net.InetAddress;
 import java.nio.charset.StandardCharsets;
 import java.util.Arrays;
@@ -80,12 +83,63 @@ public void setup(final Map<String,Object> config) {
             // have to create the indices because they are not stored in gryo
             final TinkerGraph tinkerGraph = (TinkerGraph) graph;
             tinkerGraph.createIndex(PROPERTY_USERNAME, Vertex.class);
+
+            // TinkerGraph no longer auto-loads from graphLocation on open, so read the credential store here from
+            // the configured location/format. TinkerStorageGraph (which persists via its own storage engine) manages
+            // its own data and is left untouched.
+            loadCredentialStore(tinkerGraph);
         }
 
         credentialStore = graph.traversal(CredentialTraversalSource.class);
         logger.info("CredentialGraph initialized at {}", credentialStore);
     }
 
+    /**
+     * Reads the credential store into the supplied in-memory {@link TinkerGraph} from the
+     * {@code gremlin.tinkergraph.graphLocation} and {@code gremlin.tinkergraph.graphFormat} entries of its
+     * configuration, if present. Earlier versions of TinkerGraph read those keys themselves on open, so the
+     * credential store loaded as a side effect of {@code GraphFactory.open}; that automatic behaviour was removed
+     * and this method preserves it for the credential store alone.
+     * <p/>
+     * No TinkerGraph reads either key any more, which is why they are named here as literals rather than through
+     * constants. Despite the {@code gremlin.tinkergraph} prefix they are in effect settings of this authenticator,
+     * and belong in its own {@code config} block beside {@code credentialsDb} rather than in the graph's properties
+     * file. They are left in place here only to keep existing credential configurations working. A graph with a
+     * storage engine manages its own data and is skipped.
+     */
+    private static void loadCredentialStore(final TinkerGraph graph) {
+        final Configuration conf = graph.configuration();
+        final String location = conf.getString("gremlin.tinkergraph.graphLocation", null);
+        // a storage engine manages its own persistence and is not an interchange-format load
+        final String storage = conf.getString(TinkerGraph.GREMLIN_TINKERGRAPH_STORAGE, null);
+        if (null == location || storage != null)
+            return;
+
+        final File f = new File(location);
+        if (!f.exists() || !f.isFile())
+            return;
+
+        final String format = conf.getString("gremlin.tinkergraph.graphFormat", "gryo");
+        try {
+            switch (format) {
+                case "graphml":
+                    graph.io(IoCore.graphml()).readGraph(location);
+                    break;
+                case "graphson":
+                    graph.io(IoCore.graphson()).readGraph(location);
+                    break;
+                case "gryo":
+                    graph.io(IoCore.gryo()).readGraph(location);
+                    break;
+                default:
+                    graph.io(IoCore.createIoBuilder(format)).readGraph(location);
+                    break;
+            }
+        } catch (Exception ex) {
+            throw new IllegalStateException(String.format("Could not load credential store at %s with format %s", location, format), ex);
+        }
+    }
+
     @Override
     public SaslNegotiator newSaslNegotiator(final InetAddress remoteAddress) {
         return new PlainTextSaslAuthenticator();