diff --git a/content/blog/2026-11-20-datafusion-56.0.0.md b/content/blog/2026-11-20-datafusion-56.0.0.md new file mode 100644 index 00000000..7ca99207 --- /dev/null +++ b/content/blog/2026-11-20-datafusion-56.0.0.md @@ -0,0 +1,748 @@ +--- +layout: post +title: Apache DataFusion 56.0.0 Released +date: 2026-11-20 +author: pmc +categories: [release] +--- + + + +[TOC] + + + +We are proud to announce the release of [DataFusion 56.0.0], which includes many +new features such as `ASOF` joins, SQL registration of external catalogs, Avro +and ALP write support, better support for extension types, and many performance +and stability improvements. The complete list of changes is available in the +[changelog]. This release represents roughly 12 weeks of development and 1200 +commits. Thanks to the [247 contributors] who made it possible. + +[DataFusion 56.0.0]: https://crates.io/crates/datafusion/56.0.0 +[changelog]: https://github.com/apache/datafusion/blob/branch-56/dev/changelog/56.0.0.md +[247 contributors]: https://github.com/apache/datafusion/blob/branch-56/dev/changelog/56.0.0.md#credits + +## New Features โœจ + +### `ASOF` Joins + +DataFusion 56 adds `ASOF JOIN`, a long-standing feature request ([#318]) that +matches each row on the left with the *closest* row on the right rather than an +exact match. Such joins are common in time-series workloads such as finding the +most recent quote for each trade, or aligning two sensors that sample at +different rates. + +For example, given tables of `temperature` and `humidity` readings from two +sensors that report at different times: + +| id | location | ts | temp | +|----|----------|---------------------|------| +| T1 | roof | 2026-11-20T09:30:05 | 11.5 | +| T2 | roof | 2026-11-20T09:30:12 | 11.7 | +| T3 | lobby | 2026-11-20T09:30:07 | 20.1 | + +| id | location | ts | humidity | +|----|----------|---------------------|----------| +| H1 | roof | 2026-11-20T09:30:00 | 71.0 | +| H2 | roof | 2026-11-20T09:30:10 | 70.5 | +| H3 | lobby | 2026-11-20T09:30:10 | 45.2 | + +Since no temperature reading has the exact same timestamp as any humidity +reading, a regular equality join on `location` and `ts` would match nothing. +Instead, an `ASOF` join efficiently pairs each temperature reading with the most +recent humidity reading at or before it: + +```sql +SELECT t.id AS temp_id, h.id AS humidity_id, t.location, t.ts, t.temp, h.humidity +FROM temperature AS t ASOF JOIN humidity AS h +MATCH_CONDITION (t.ts >= h.ts) +ON t.location = h.location +ORDER BY t.id; +``` + +```text ++---------+-------------+----------+---------------------+------+----------+ +| temp_id | humidity_id | location | ts | temp | humidity | ++---------+-------------+----------+---------------------+------+----------+ +| T1 | H1 | roof | 2026-11-20T09:30:05 | 11.5 | 71.0 | +| T2 | H2 | roof | 2026-11-20T09:30:12 | 11.7 | 70.5 | +| T3 | NULL | lobby | 2026-11-20T09:30:07 | 20.1 | NULL | ++---------+-------------+----------+---------------------+------+----------+ +``` + +
+Full Example + +```sql +CREATE TABLE temperature (id VARCHAR, location VARCHAR, ts TIMESTAMP, temp DOUBLE) AS VALUES + ('T1', 'roof', '2026-11-20T09:30:05', 11.5), + ('T2', 'roof', '2026-11-20T09:30:12', 11.7), + ('T3', 'lobby', '2026-11-20T09:30:07', 20.1); + +CREATE TABLE humidity (id VARCHAR, location VARCHAR, ts TIMESTAMP, humidity DOUBLE) AS VALUES + ('H1', 'roof', '2026-11-20T09:30:00', 71.0), + ('H2', 'roof', '2026-11-20T09:30:10', 70.5), + ('H3', 'lobby', '2026-11-20T09:30:10', 45.2); + +SELECT t.id AS temp_id, h.id AS humidity_id, t.location, t.ts, t.temp, h.humidity +FROM temperature AS t ASOF JOIN humidity AS h +MATCH_CONDITION (t.ts >= h.ts) +ON t.location = h.location +ORDER BY t.id; +``` + +
+ +DataFusion supports the [Snowflake syntax][Snowflake ASOF JOIN syntax]; please +see the [documentation][ASOF JOIN documentation] for more details. Thanks to +[@Xuanwo] for contributing this feature and to [@jayzhan211], +[@Kristofferjoha], [@2010YOUY01], [@xudong963], [@alamb], and [@jonathanc-n] +for reviews and improvements. + +[Snowflake ASOF JOIN syntax]: https://docs.snowflake.com/en/sql-reference/constructs/asof-join +[ASOF JOIN documentation]: https://datafusion.apache.org/user-guide/sql/select.html#asof-join + +### External Catalogs in SQL + +Connecting DataFusion to an external catalog, such as an Iceberg REST catalog, +previously required writing Rust code. DataFusion 56 adds support for +`CREATE EXTERNAL CATALOG` and `DROP CATALOG` statements ([#25112]) to attach +catalogs with SQL in the same way as external tables: + +```sql +CREATE EXTERNAL CATALOG my_catalog +STORED AS ICEBERG +LOCATION 's3://bucket/warehouse' +OPTIONS ('catalog.uri' 'http://rest-catalog:8181', 'warehouse' 'my_catalog'); + +SELECT * FROM my_catalog.sales.orders LIMIT 10; +``` + +Note that DataFusion does not include any catalog implementations itself. +Instead, users register a [`CatalogProviderFactory`][CatalogProviderFactory] +for each `STORED AS` type, and then attach catalogs with SQL; see the +[`CREATE EXTERNAL CATALOG` documentation][CREATE EXTERNAL CATALOG documentation] +for more details. + +Thanks to [@pepijnve] and [@TTyChud] for this work. + +[CatalogProviderFactory]: https://docs.rs/datafusion/56.0.0/datafusion/catalog/trait.CatalogProviderFactory.html +[CREATE EXTERNAL CATALOG documentation]: https://datafusion.apache.org/user-guide/sql/ddl.html#create-external-catalog + +### File Format Improvements + +**Writing Avro**: DataFusion has been able to read [Apache Avro] files for a +long time, and DataFusion 56 adds support for writing them: + +```sql +COPY orders TO 'orders.avro' STORED AS AVRO; +``` + +
+Full Example + +```sql +CREATE TABLE orders (id INT, customer VARCHAR, total DOUBLE) AS VALUES + (1, 'alice', 42.50), + (2, 'bob', 17.25), + (3, 'carol', 99.99); + +-- Write the table to an Avro file +COPY orders TO 'orders.avro' STORED AS AVRO; + +-- Read it back +SELECT * FROM 'orders.avro'; ++----+----------+-------+ +| id | customer | total | ++----+----------+-------+ +| 1 | alice | 42.5 | +| 2 | bob | 17.25 | +| 3 | carol | 99.99 | ++----+----------+-------+ +``` + +
+ +**Parquet ALP Encoding**: [ALP] (Adaptive Lossless floating-Point) is a new +Parquet encoding that often stores `FLOAT` and `DOUBLE` columns much more +compactly. DataFusion 56 can read and write ALP-encoded files: + +```sql +COPY measurements TO 'measurements.parquet' STORED AS PARQUET +OPTIONS ( + 'format.encoding::price' 'alp', + 'format.dictionary_enabled::price' 'false' +); +``` + +
+Full Example + +```sql +-- One million prices with two decimal places +CREATE TABLE measurements AS +SELECT i AS id, round(100000 * random(), 2) AS price +FROM generate_series(1, 1000000) AS t(i); + +-- Write with the default settings, and with ALP for the price column +COPY measurements TO 'default.parquet' STORED AS PARQUET; +COPY measurements TO 'alp.parquet' STORED AS PARQUET +OPTIONS ( + 'format.encoding::price' 'alp', + 'format.dictionary_enabled::price' 'false' +); + +-- Compare the size of the price column in the two files. +-- Note that the ALP-encoded column is about a third smaller than with +-- the default settings. +SELECT 'default' AS file, encodings, total_compressed_size +FROM parquet_metadata('default.parquet') WHERE path_in_schema = '"price"' +UNION ALL +SELECT 'alp', encodings, total_compressed_size +FROM parquet_metadata('alp.parquet') WHERE path_in_schema = '"price"'; ++---------+------------------------------+-----------------------+ +| file | encodings | total_compressed_size | ++---------+------------------------------+-----------------------+ +| default | [PLAIN, RLE, RLE_DICTIONARY] | 4481181 | +| alp | [RLE, ALP] | 3018657 | ++---------+------------------------------+-----------------------+ +``` + +
+ +Thanks to [@pepijnve] and [@Surajsuthar] for this work. + +[Apache Avro]: https://avro.apache.org/ +[ALP]: https://github.com/apache/parquet-format/blob/master/Encodings.md + +### More SQL Support + +**`MATERIALIZED` CTEs**: By default, DataFusion inlines Common Table Expressions +(CTEs), so a CTE that is referenced multiple times is evaluated multiple times. +DataFusion 56 supports the Postgres `WITH ... AS [NOT] MATERIALIZED` syntax to +evaluate a CTE once and reuse the result ([#25580]). The syntax is currently +only recognized by the `postgres` SQL dialect. + +
+Example + +```sql +SET datafusion.sql_parser.dialect = 'postgres'; + +-- The CTE runs once, so both references see the same random value +WITH r AS MATERIALIZED (SELECT random() AS v) +SELECT a.v = b.v AS same FROM r AS a, r AS b; ++------+ +| same | ++------+ +| true | ++------+ + +-- Without MATERIALIZED, each reference evaluates random() separately +WITH r AS (SELECT random() AS v) +SELECT a.v = b.v AS same FROM r AS a, r AS b; ++-------+ +| same | ++-------+ +| false | ++-------+ +``` + +
+ +Thanks to [@adriangb] for this work. + +### `datafusion-cli` Improvements + +`datafusion-cli` adds [`parquet_file_metadata`][parquet_file_metadata] and +[`parquet_page_index`][parquet_page_index] table functions for inspecting +Parquet files. Thanks to [@jay-dee7] for this work, with reviews from +[@kumarUjjawal]. + +[parquet_file_metadata]: https://datafusion.apache.org/user-guide/cli/functions.html#parquet-file-metadata +[parquet_page_index]: https://datafusion.apache.org/user-guide/cli/functions.html#parquet-page-index + +### Versioned Documentation + +The [DataFusion documentation] site previously always reflected the latest +`main` branch, making it hard to tell which features were available in what +version. The site now has a version picker and publishes a snapshot of the +documentation for each release. Thanks to [@yiaany] for this work and +[@Omega359] for the original request. + + + +**Figure 1**: The new version picker on the [DataFusion documentation] site. + +[DataFusion documentation]: https://datafusion.apache.org/ + +### Improved Support for Extension Types + +As described in [the User Defined Types blog], DataFusion's type system can be +extended using [Arrow extension types] to attach a logical type to an Arrow +physical type. This technique allows the DataFusion ecosystem to add entirely +new data types *without* changes to the DataFusion core (see +[Ecosystem Highlights] below). DataFusion 56 improves this support in several +ways: + +- The SQL types `UUID` and `JSON` now map to the Arrow + [canonical extension types] `arrow.uuid` and `arrow.json`. +- `DESCRIBE`, `SHOW COLUMNS`, and `information_schema.columns` show the + extension type name rather than the storage type. +- Extension types are preserved when reading Parquet files with the default + settings. +- Extension types are preserved through more operators such as casts, + `min` / `max`, scalar subqueries, and `INSERT`. + +For example, a column declared with the SQL `JSON` type is stored as a standard +Arrow string and tagged with the `arrow.json` extension type: + +```sql +CREATE TABLE events (doc JSON); +INSERT INTO events VALUES ('{"a":1}'); +SELECT doc, + arrow_typeof(doc) AS doc_type, + arrow_metadata(doc)['ARROW:extension:name'] AS doc_extension +FROM events; ++---------+----------+---------------+ +| doc | doc_type | doc_extension | ++---------+----------+---------------+ +| {"a":1} | Utf8 | arrow.json | ++---------+----------+---------------+ +``` + +Thanks to [@cblp], [@cyberbeam524], [@paleolimbot], [@M-Tesla], and +[@osipovartem] for this work. + +[Arrow extension types]: https://arrow.apache.org/docs/format/Columnar.html#extension-types +[canonical extension types]: https://arrow.apache.org/docs/format/CanonicalExtensions.html +[Ecosystem Highlights]: #ecosystem-highlights +[the User Defined Types blog]: https://datafusion.apache.org/blog/2025/09/21/custom-types-using-metadata + +## Ecosystem Highlights ๐ŸŒฑ + +DataFusion is designed to be extended, and this section highlights two projects +in the DataFusion ecosystem that are made possible by the extension points in +DataFusion 56. + +### Semi-Structured Data (e.g. JSON) + +Parquet [Variant] is a type for storing semi-structured data, similar to JSON +but with an efficient binary encoding, that was recently added to Apache Parquet +and Apache Iceberg (see the [blog on the Parquet site] for more details). The +[datafusion-variant] crate extends DataFusion to add the `VARIANT` SQL type and +Spark-compatible functions. Register it with a single call to `register_all` on +a `SessionContext`: + +```rust +let mut ctx = SessionContext::new(); +datafusion_variant::register_all(&mut ctx)?; +``` + +Then you can load, query, and store semi-structured data using SQL. For +example, given an `events` table with a `VARIANT` column: + +```sql +SELECT * FROM events; ++------------------------------------------------------------------+ +| event | ++------------------------------------------------------------------+ +| {"latency_ms":12,"service":"api","status":200,"tags":["cached"]} | +| {"error":null,"latency_ms":85,"service":"api","status":500} | +| {"error":"Somethings not working","service":"api","status":504} | ++------------------------------------------------------------------+ +``` + +you can extract and filter on individual fields: + +```sql +-- Extract fields with the Spark-style `:` operator +SELECT event:service::VARCHAR, event:latency_ms::BIGINT +FROM events +WHERE event:status::BIGINT >= 400; ++----------------------+-------------------------+ +| events.event:service | events.event:latency_ms | ++----------------------+-------------------------+ +| api | 85 | +| api | NULL | ++----------------------+-------------------------+ +``` + +
+Full Example + +```sql +-- Load JSON data into a VARIANT column +CREATE TABLE events AS +SELECT parse_json(json_data) AS event +FROM (VALUES + ('{"service":"api","status":200,"latency_ms":12,"tags":["cached"]}'), + ('{"service":"api","status":500,"latency_ms":85,"error":null}'), + ('{"service":"api","status":504,"error":"Somethings not working"}') +) AS input(json_data); + +-- Show the contents of the table +SELECT * FROM events; + +-- Extract fields with the Spark-style `:` operator +SELECT event:service::VARCHAR, event:latency_ms::BIGINT +FROM events +WHERE event:status::BIGINT >= 400; +``` + +
+ +Variant columns can be written to Parquet files with `COPY` statements, +optionally [*shredding* commonly used fields][Variant shredding] into typed +Parquet columns for faster queries: + +```sql +COPY events TO 'shredded_events.parquet' STORED AS PARQUET +OPTIONS ('variant.shredding' '{"event": "STRUCT"}'); +``` + +The `datafusion-variant` crate is under active development; refer to the +[project documentation][datafusion-variant] for more details. Thanks to +[@sdf-jkl], [@friendlymatthew], [@alamb], [@raghuvanshraj], [@getChan], and the +other contributors for making it possible. + +[Variant]: https://parquet.apache.org/docs/file-format/types/variantencoding/ +[blog on the Parquet site]: https://parquet.apache.org/blog/2026/02/27/variant-type-in-apache-parquet-for-semi-structured-data/ +[datafusion-variant]: https://datafusion-contrib.github.io/datafusion-variant/ +[Variant shredding]: https://parquet.apache.org/docs/file-format/types/variantshredding/ + +### Geospatial Data + + + +[geodatafusion] provides spatial extensions for DataFusion, built on the +[GeoArrow] extension types and modeled after the [PostGIS] SQL API. It includes +around 90 spatial functions such as [`ST_Distance`][ST_Distance] and +[`ST_Intersects`][ST_Intersects], support for reading [GeoParquet], reading and +writing [FlatGeobuf], and writing GeoJSON files, and Python bindings. + +Install it with a single call: + +```rust +let ctx = SessionContext::new(); +geodatafusion::register(&ctx); +``` + +And then you can query geometries using familiar PostGIS-style SQL: + +```sql +SELECT ST_Distance(ST_Point(0, 0), ST_Point(3, 4)) AS dist; ++------+ +| dist | ++------+ +| 5.0 | ++------+ + +SELECT ST_Area(ST_GeomFromText('POLYGON((0 0, 4 0, 4 4, 0 4, 0 0))')) AS area; ++------+ +| area | ++------+ +| 16.0 | ++------+ +``` + +
+Full Example + +```sql +-- A table of named places +CREATE TABLE places (name VARCHAR, x DOUBLE, y DOUBLE) AS VALUES + ('cafe', 1.0, 1.0), + ('library', 3.0, 4.0), + ('airport', 9.0, 9.0); + +-- The distance of each place from the origin, and whether it is inside +-- the 5 x 5 "downtown" square +SELECT name, + ST_AsText(ST_Point(x, y)) AS location, + round(ST_Distance(ST_Point(x, y), ST_Point(0, 0)), 2) AS distance, + ST_Intersects( + ST_Point(x, y), + ST_GeomFromText('POLYGON((0 0, 5 0, 5 5, 0 5, 0 0))') + ) AS downtown +FROM places +ORDER BY distance; ++---------+------------+----------+----------+ +| name | location | distance | downtown | ++---------+------------+----------+----------+ +| cafe | POINT(1 1) | 1.41 | true | +| library | POINT(3 4) | 5.0 | true | +| airport | POINT(9 9) | 12.73 | false | ++---------+------------+----------+----------+ +``` + +
+ +Thanks to [@kylebarron] for creating `geodatafusion`, and to [@Xuanwo], +[@gadomski], [@ianthetechie], [@sharkinsspatial], and the other contributors +for their work on it. + +[geodatafusion]: https://github.com/datafusion-contrib/geodatafusion +[GeoArrow]: https://geoarrow.org/ +[PostGIS]: https://postgis.net/docs/reference.html +[GeoParquet]: https://geoparquet.org/ +[FlatGeobuf]: https://flatgeobuf.org/ +[ST_Distance]: https://docs.rs/geodatafusion/latest/geodatafusion/udf/geo/measurement/struct.Distance.html +[ST_Intersects]: https://docs.rs/geodatafusion/latest/geodatafusion/udf/geo/relationships/struct.Intersects.html + +## Performance Improvements ๐Ÿš€ + +DataFusion 56 continues to get faster across the board. Here is a +representative sample of the improvements in this release. + + + +| Improvement | Representative Result | Area | +|-----------------------------------------------------|--------------------------------------------------------------------|-------------| +| `IN` / `NOT IN` subqueries in the `SELECT` list | [533 ms โ†’ 2 ms][#25338] | Subqueries | +| Ordered `ARRAY_AGG` and `STRING_AGG` | [10.5x faster, 59% less memory][#24392] | Aggregation | +| `GROUP BY` on fully sorted keys | [2.8x - 5x faster][#25150] *(micro)* | Aggregation | +| `array_agg(DISTINCT ...)` with many groups | [up to 36x faster][#26357] *(micro)* | Aggregation | +| Dynamic filters across join keys | [up to 2.3x faster on Join Order Benchmark queries][#25255] | Joins | +| Sort-merge joins with many matches per key | [~35% faster][#26277] *(micro)* | Joins | +| Nested loop join memory | [2x lower peak memory][#25371] | Joins | +| `LAST_VALUE ... IGNORE NULLS` | [13.15 ms โ†’ 111 ยตs][#26533] *(micro)* | Window | +| Planning `SELECT`s with many aggregates | [5.9x faster for 1000 aggregates][#25739] | Planning | +| Debug build of `datafusion` from scratch | [~3x faster end to end, some crates up to 10x][#24329] | Build | + +### Faster Builds + +A debug build of the `datafusion` crate and its dependencies from scratch is +now about 3x faster end to end, and some individual crates, including the core +`datafusion` crate, compile up to 10x faster (see [#24329], [#24325], [#24330], +[#24326]). Thanks to [@Dandandan]. + +### Faster TopK and `LIMIT` Queries + +Queries that return only a few rows keep getting faster. DataFusion 56 can +evaluate `ORDER BY ... LIMIT` *before* an outer join when the sort columns come +from the preserved side, so far fewer rows are joined ([#21621]). For example, +this query now joins only three orders rather than every order: + +```sql +SELECT o.id, o.total, c.name +FROM orders AS o LEFT JOIN customers AS c ON o.customer_id = c.id +ORDER BY o.total DESC +LIMIT 3; +``` + +Parquet scans now stop after reading only the *pages* that are guaranteed to +satisfy a `LIMIT` query's filter ([#25000]), and custom table providers can +implement `OFFSET` pushdown, which is useful for paging through results +([#25404]). Thanks to [@SubhamSinghal], [@peterxcli], and [@martin-g]. + +### Faster Joins + +Dynamic filters created by a join are now applied to *other* tables that are +joined on the same key, and pushed through nested loop joins, skipping more +data at the scan ([#25255], [#25030]). Other join improvements include faster +sort-merge joins ([#26277]), a nested loop join that needs half the memory +([#25371]), faster range joins ([#24579]), and rewriting joins against a single +constant row into filters ([#26404]). Thanks to [@jayzhan211], [@yashrb24], +[@andygrove], [@2010YOUY01], and [@xudong963]. + +### Faster Aggregations and Subqueries + +Aggregations over already sorted inputs now avoid hashing entirely +([#25150], [#26370]), and ordered `ARRAY_AGG` and `STRING_AGG` are over 10x +faster while using less than half the memory ([#24392]). `IN` and `NOT IN` +subqueries used as expressions (for example in the `SELECT` list) are now +planned as a single join, which makes some queries hundreds of times faster +([#25338]). Thanks to [@QinXi-ai], [@geoffreyclaude], [@lyne7-sc], +[@zhuqi-lucas], and [@adriangb]. + +### Faster Functions and Expressions + +As in previous releases, many functions are faster, including `upper`, `lower`, +`translate`, `replace`, `left`, `right`, and `levenshtein`, along with `IN` +list evaluation. Thanks to [@neilconway], [@geoffreyclaude], [@andygrove], +[@Rich-T-kid], and [@comphead] for making this happen. + +## Stability Improvements ๐Ÿ›ก๏ธ + +**Correctness**: As a foundational piece of technology, it is important that +DataFusion's results can be relied on. During this release cycle the community +systematically focused on finding and fixing bugs that produced incorrect +results. The tracking epic ([#26672]) lists more than 60 such bugs that are +fixed in DataFusion 56, along with the ones we are still working on. Thanks to +[@haohuaijin], [@adriangb], [@AdamGS], [@matthewmturner], [@viirya], +[@mbutrovich], and the many others who reported, fixed, and reviewed these +issues. + +**More Robust Larger-Than-Memory Queries**: Queries that spill to disk when +memory runs out are more robust. Aggregations with very large groups (such as +`array_agg` over a few keys) no longer fail after spilling ([#25853]), spill +files for string and dictionary data are smaller ([#23565], [#27015]), and +memory accounting is more accurate ([#24744], [#25188], [#26074]). Thanks to +[@andygrove], [@cetra3], [@brancz], and [@kosiew]. + +## Upgrade Guide and Changelog ๐Ÿ“– + +DataFusion 56 upgrades to [Arrow and Parquet 60] and +[`object_store` 0.14][object_store 0.14], and the minimum supported Rust +version is now 1.95.0. Upgrading should be straightforward for most users. See +the [Upgrade Guide] for details and migration snippets, and the [changelog] for +the full list of changes. + +## About DataFusion + +[Apache DataFusion] is an extensible query engine, written in [Rust], that uses +[Apache Arrow] as its in-memory format. DataFusion is used by developers to +create new, fast, data-centric systems such as databases, dataframe libraries, +and machine learning and streaming applications. While [DataFusion's primary +design goal] is to accelerate the creation of other data-centric systems, it +provides a reasonable experience directly out of the box as a [dataframe +library], [Python library], and [command-line SQL tool]. + +DataFusion's core thesis is that, as a community, together we can build much +more advanced technology than any of us as individuals or companies could build +alone. Without DataFusion, highly performant vectorized query engines would +remain the domain of a few large companies and world-class research +institutions. With DataFusion, we can all build on top of a shared foundation +and focus on what makes our projects unique. + +## How to Get Involved + +DataFusion is not a project built or driven by a single person, company, or +foundation. Rather, our community of users and contributors works together to +build a shared technology that none of us could have built alone. + +If you are interested in joining us, we would love to have you. You can try out +DataFusion on some of your own projects and let us know how it goes, contribute +suggestions, documentation, bug reports, or a PR with documentation, tests, or +code. [Please don't dump unreviewed AI output on us] ๐Ÿ™. A list of open issues +suitable for beginners is [here], and you can find out how to reach us on the +[communication doc]. + +If you want to know what we are planning to work on next, or to suggest what you +think is important, please join the [2026 Q4 Roadmap discussion]. + +[2026 Q4 Roadmap discussion]: https://github.com/apache/datafusion/issues/26684 +[Apache DataFusion]: https://datafusion.apache.org/ +[Rust]: https://www.rust-lang.org/ +[Apache Arrow]: https://arrow.apache.org +[DataFusion's primary design goal]: https://datafusion.apache.org/user-guide/introduction.html#project-goals +[dataframe library]: https://datafusion.apache.org/user-guide/dataframe.html +[Python library]: https://datafusion.apache.org/python/ +[command-line SQL tool]: https://datafusion.apache.org/user-guide/cli/ +[Arrow and Parquet 60]: https://arrow.apache.org/blog/2026/09/29/arrow-rs-60.0.0/ +[object_store 0.14]: https://github.com/apache/arrow-rs-object-store/releases/tag/v0.14.0 +[Upgrade Guide]: https://datafusion.apache.org/library-user-guide/upgrading/56.0.0.html +[here]: https://github.com/apache/datafusion/issues?q=is%3Aissue+is%3Aopen+label%3A%22good+first+issue%22 +[communication doc]: https://datafusion.apache.org/contributor-guide/communication.html +[Please don't dump unreviewed AI output on us]: https://datafusion.apache.org/contributor-guide/ai-policy.html#what-is-an-ai-dump-and-why-it-is-not-helpful + +[@2010YOUY01]: https://github.com/2010YOUY01 +[@adriangb]: https://github.com/adriangb +[@andygrove]: https://github.com/andygrove +[@brancz]: https://github.com/brancz +[@cblp]: https://github.com/cblp +[@cetra3]: https://github.com/cetra3 +[@comphead]: https://github.com/comphead +[@cyberbeam524]: https://github.com/cyberbeam524 +[@Dandandan]: https://github.com/Dandandan +[@geoffreyclaude]: https://github.com/geoffreyclaude +[@jayzhan211]: https://github.com/jayzhan211 +[@kosiew]: https://github.com/kosiew +[@Kristofferjoha]: https://github.com/Kristofferjoha +[@lyne7-sc]: https://github.com/lyne7-sc +[@M-Tesla]: https://github.com/M-Tesla +[@martin-g]: https://github.com/martin-g +[@neilconway]: https://github.com/neilconway +[@Omega359]: https://github.com/Omega359 +[@osipovartem]: https://github.com/osipovartem +[@paleolimbot]: https://github.com/paleolimbot +[@pepijnve]: https://github.com/pepijnve +[@peterxcli]: https://github.com/peterxcli +[@QinXi-ai]: https://github.com/QinXi-ai +[@Rich-T-kid]: https://github.com/Rich-T-kid +[@SubhamSinghal]: https://github.com/SubhamSinghal +[@Surajsuthar]: https://github.com/Surajsuthar +[@TTyChud]: https://github.com/TTyChud +[@Xuanwo]: https://github.com/Xuanwo +[@xudong963]: https://github.com/xudong963 +[@yashrb24]: https://github.com/yashrb24 +[@yiaany]: https://github.com/yiaany +[@zhuqi-lucas]: https://github.com/zhuqi-lucas + +[#318]: https://github.com/apache/datafusion/issues/318 +[#21621]: https://github.com/apache/datafusion/pull/21621 +[#23565]: https://github.com/apache/datafusion/pull/23565 +[#24325]: https://github.com/apache/datafusion/pull/24325 +[#24326]: https://github.com/apache/datafusion/pull/24326 +[#24329]: https://github.com/apache/datafusion/pull/24329 +[#24330]: https://github.com/apache/datafusion/pull/24330 +[#24392]: https://github.com/apache/datafusion/pull/24392 +[#24579]: https://github.com/apache/datafusion/pull/24579 +[#24744]: https://github.com/apache/datafusion/pull/24744 +[#25000]: https://github.com/apache/datafusion/pull/25000 +[#25030]: https://github.com/apache/datafusion/pull/25030 +[#25112]: https://github.com/apache/datafusion/pull/25112 +[#25150]: https://github.com/apache/datafusion/pull/25150 +[#25188]: https://github.com/apache/datafusion/pull/25188 +[#25255]: https://github.com/apache/datafusion/pull/25255 +[#25338]: https://github.com/apache/datafusion/pull/25338 +[#25371]: https://github.com/apache/datafusion/pull/25371 +[#25404]: https://github.com/apache/datafusion/pull/25404 +[#25580]: https://github.com/apache/datafusion/pull/25580 +[#25739]: https://github.com/apache/datafusion/pull/25739 +[#25853]: https://github.com/apache/datafusion/pull/25853 +[#26074]: https://github.com/apache/datafusion/pull/26074 +[#26277]: https://github.com/apache/datafusion/pull/26277 +[#26357]: https://github.com/apache/datafusion/pull/26357 +[#26370]: https://github.com/apache/datafusion/pull/26370 +[#26404]: https://github.com/apache/datafusion/pull/26404 +[#26533]: https://github.com/apache/datafusion/pull/26533 +[#26672]: https://github.com/apache/datafusion/issues/26672 +[#27015]: https://github.com/apache/datafusion/pull/27015 +[@alamb]: https://github.com/alamb +[@jonathanc-n]: https://github.com/jonathanc-n +[@haohuaijin]: https://github.com/haohuaijin +[@AdamGS]: https://github.com/AdamGS +[@matthewmturner]: https://github.com/matthewmturner +[@viirya]: https://github.com/viirya +[@mbutrovich]: https://github.com/mbutrovich +[@sdf-jkl]: https://github.com/sdf-jkl +[@friendlymatthew]: https://github.com/friendlymatthew +[@raghuvanshraj]: https://github.com/raghuvanshraj +[@getChan]: https://github.com/getChan +[@kylebarron]: https://github.com/kylebarron +[@gadomski]: https://github.com/gadomski +[@ianthetechie]: https://github.com/ianthetechie +[@sharkinsspatial]: https://github.com/sharkinsspatial +[@jay-dee7]: https://github.com/jay-dee7 +[@kumarUjjawal]: https://github.com/kumarUjjawal diff --git a/content/images/datafusion-56.0.0/versioned_docs.png b/content/images/datafusion-56.0.0/versioned_docs.png new file mode 100644 index 00000000..41b8d2d9 Binary files /dev/null and b/content/images/datafusion-56.0.0/versioned_docs.png differ