You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adds support for the float16 base type of the SQL Server vector data type, on both .NET and .NET Framework.
Behaviour
float16 is exchanged in its binary form only when a connection opts in, through the new Vector Type Support keyword (off | v1 | v2, default v1). This mirrors vectorTypeSupport in the JDBC driver, including its default. At v1 a float16 column is returned as a varchar(max) containing a JSON array, exactly as today, so upgrading the driver changes nothing until an application asks for v2.
At v2:
.NET
.NET Framework
GetValue / GetFieldType
SqlVector<Half>
string (JSON array)
Strongly typed read
GetSqlVector<Half>
GetSqlVector<float> — widening, which is exact
Write
SqlVector<Half>, SqlVector<float>, or a JSON string
SqlVector<float> or a JSON string
System.Half does not exist on .NET Framework, so a float16 column is surfaced there as its JSON rendering rather than by substituting a different base type. A caller that wants a strongly typed value asks for SqlVector<float> explicitly.
Column metadata
A vector column now reports its base type and dimension count through the existing DbColumn indexer, with no new API:
This is the only way to distinguish the two base types when a float16 column is surfaced as a JSON string, because GetFieldType reports string for such a column just as it does for a varchar one.
Bulk copy
The INSERT BULK statement states the destination column's base type, and the server then requires a binary payload of exactly that width and performs no conversion within the data stream. Measured at every negotiated version: a text declaration for a vector column is refused with Invalid column type from bcp client, and text sent under a vector declaration is refused with a length mismatch. Only a client which never negotiates the feature extension may send text.
So a textual source is parsed into the destination's base type by the driver, and a payload read from another vector column keeps its own base type — copying between columns of different base types is reported by the server rather than silently narrowed. Both match the JDBC driver.
Public API
One addition: the SqlVectorTypeSupport enum and SqlConnectionStringBuilder.VectorTypeSupport. SqlVector<T> is unchanged.
Tests
float16 added to the existing generic native vector suite, run over a v2 connection
the same suite run through SqlVector<float> on every framework, which is the .NET Framework representation and where the hand written binary16 codec is the production path
version negotiation per keyword value, including the client's own version ceiling
the binary16 codec validated against System.Half across all 65,536 binary16 and all 4,294,967,296 binary32 patterns
Checklist
Tests added or updated
Public API changes documented
Ensure no breaking changes introduced — the default keeps the current representation
Notes for reviewers: the second commit reworks bulk copy and negotiation following a design review, so it is worth reading over the first. One review thread is left open pending that discussion.
A vector column's base type and number of dimensions were already available
from the column schema, but only as a numeric scale and a column size which
the caller had to decode. They are now surfaced under their own names, so
that applications inspecting result set metadata do not have to know that
encoding.
Also registers the vector type in the DataTypes schema collection, where it
was missing.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
SQL Server transports vector(N, float16) elements as raw binary16 values.
System.Half is only available on .NET, so the conversion is implemented
manually for .NET Framework.
The manual implementation is compiled for every target framework rather than
only for .NET Framework, so that it can be validated exhaustively against
System.Half on .NET while remaining the code path .NET Framework actually
uses. It is verified against every binary16 bit pattern, a strided sweep of
the single precision range, and the rounding, subnormal, overflow and
underflow boundaries.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
Advertises version 2 of the VECTORSUPPORT feature extension, so that a
vector(N, float16) column is exchanged in its native binary form rather than
as a varchar(max) JSON string.
On .NET such a column is surfaced as SqlVector<Half>. .NET Framework has no
System.Half, so it is reported as a string there, matching how it is already
presented when the server does not negotiate float16 support. Callers on
either framework can explicitly request a strongly typed value via
GetSqlVector<float>, which widens the elements without loss.
SqlVector<T> continues to derive the base type written to the wire from T
alone. Conversion between base types is left to the server, which performs it
for parameters. Bulk copy is the exception: it declares the destination's base
type in the INSERT BULK statement, so a payload using a different base type is
rejected as a column length error rather than converted, and is rewritten by
the driver first. That conversion runs after coercion, because the payload
coercion produces uses the source value's own base type: a JSON string always
yields float32, which is how a float16 column reads back where System.Half is
unavailable.
SqlVector<T>.ToString() now returns the vector's values as a JSON array rather
than the type name.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
Describes the base types a vector column can have, how they map to
SqlVector<T>, and how a float16 column is read and written on .NET Framework,
where System.Half does not exist. Also documents the vector feature extension
versions and the column metadata properties.
Adds a sample covering both frameworks, reading a float16 column as an exact,
widened or JSON value, inspecting a column's base type and dimensions, and
converting between base types.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Adds end-to-end support for SQL Server vector(N, float16) by negotiating VECTORSUPPORT feature extension version 2, introducing an IEEE-754 binary16 codec, and wiring float16 handling through SqlVector<T> read/write paths (including bulk copy), with accompanying docs and tests.
Changes:
Negotiate vector feature extension v2 and track negotiated vector capability version (float32/float16) on the connection.
Add float16 vector support across SqlVector<T>, SqlDataReader, SqlBuffer, SqlParameter, SqlCommand, and SqlBulkCopy, including payload conversion for bulk copy.
Add unit/manual tests plus docs/snippets/sample updates; expose vector base type + dimensions via DbColumn indexer and register vector in the DataTypes schema collection.
Reviewed changes
Copilot reviewed 23 out of 23 changed files in this pull request and generated 2 comments.
ConvertPayloadElementType doesn’t validate the vector header magic/version bytes before using the length and element type fields. This can cause non-vector payloads to be converted (or to fail later with less appropriate exceptions). Validate VecHeaderMagicNo/VecVersionNo up front, consistent with GetCountsOrThrow.
if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
{
throw ADP.InvalidVectorHeader();
}
Bulk copy read a vector column through the representation the reader
surfaces, which is a JSON string on frameworks without System.Half. That
round trip is both larger than the payload it encodes and unable to carry a
negative zero, because System.Text.Json on .NET Framework serialises one as
zero and parses a negative zero literal back as positive zero.
Reading the payload directly avoids both. It is chosen once per column, when
the source and destination are both vector columns, alongside the existing
decimal and streaming decisions. Any difference in base type between the two
is still resolved when the value is converted.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
SqlTypes.SqlVector<float> doesn’t resolve to any namespace/type in this file (there’s no using SqlTypes = ... and no SqlTypes namespace). This should be fully qualified to Microsoft.Data.SqlTypes.SqlVector<float> (or add an alias) to avoid a compile error.
// The payload is converted directly rather than through a strongly typed vector,
// so that .NET Framework, which has no System.Half, can also write to float16
// destinations.
return SqlTypes.SqlVector<float>.ConvertPayloadElementType(payload, destinationElementType);
FromTdsPayload reads header fields (element type/length) without validating the vector magic/version bytes. This makes the widening path accept malformed payloads that GetCountsOrThrow would reject.
if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
{
throw ADP.InvalidVectorHeader();
}
ConvertPayloadElementType should validate the vector header magic/version before interpreting element type and length; otherwise malformed byte[] values can be converted and sent on the wire rather than failing fast with InvalidVectorHeader.
if (tdsBytes.Length < TdsEnums.VECTOR_HEADER_SIZE)
{
throw ADP.InvalidVectorHeader();
}
doc/samples/SqlVectorFloat16Example.cs:146
These interpolated strings won’t compile because the expression uses double quotes (e.g., column["VectorBaseType"]) inside a double-quoted string literal. Escape the quotes (or assign to a local variable) before interpolating.
Console.WriteLine($"\nColumn base type: {column["VectorBaseType"]}");
Console.WriteLine($"Column dimensions: {column["VectorDimensions"]}");
The existing suite covers nulls where the source and destination share a base
type, but not where they differ, which is the path that converts the payload.
Verified that nulls survive in every combination, interleaved with non-null
rows so that a row's nullness cannot be satisfied by position alone.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The reason will be displayed to describe this comment to others. Learn more.
Pull request overview
Copilot reviewed 23 out of 23 changed files in this pull request and generated no new comments.
Suppressed comments (2)
doc/samples/SqlVectorFloat16Example.cs:146
These interpolated strings won’t compile because the expression contains a string literal with double quotes (e.g., column["VectorBaseType"]) which terminates the outer interpolated string. Assign the indexer results to variables (or constants) first, then interpolate those variables.
Console.WriteLine($"\nColumn base type: {column["VectorBaseType"]}");
Console.WriteLine($"Column dimensions: {column["VectorDimensions"]}");
This SqlVector special-case is redundant/unreachable because SqlVector implements ISqlVector (so it will already be handled by the earlier value is ISqlVector branch). Keeping the extra branch increases maintenance burden and risks diverging behavior.
else if (currentType == typeof(SqlVector<Half>))
{
value = ((ISqlVector)value).VectorPayload;
}
#endif
The reason will be displayed to describe this comment to others. Learn more.
Summary
This adds vector(N, float16) by advertising VECTORSUPPORT v2, introducing a hand-written binary16 codec, teaching SqlVector<T> about System.Half, and switching vector→vector bulk copy to a raw-payload transfer. The engineering is careful, and I want to call out specifically that the endianness and element-size arithmetic is correct throughout — I went looking for a missed (ColumnSize - 8) / 4 and there isn't one. The commit split is clean and the rationale in the description is unusually good.
I have one blocking correctness issue, plus a set of suggestions. Inline comments carry the detail and suggested diffs; this is the map.
Blocking
SqlBuffer.GetSqlVector<T>() succeeds or throws depending on the row's nullness. The IsNull branch builds a vector for anyT without consulting the column's base type, while the non-null branch validates. GetSqlVector<Half>() over a float32 column therefore returns Null for NULL rows and throws NotSupportedException for non-NULL rows in the same result set. Data-dependent rather than schema-dependent, so it is hard to find in testing and impossible to guard against in caller code.
Suggestions
The codec is not bit-exact against System.Half for NaN, contrary to the description, and the tests are written to step around exactly that case (continue / IsNaN-only). float.NaN carries the sign bit, so widening flips the sign of every NaN relative to System.Half; narrowing canonicalises payloads that (Half)float preserves. Either match System.Half or pin the canonical form with explicit assertions and drop the "bit-exact" claim.
Bulk copy's vector case uses metadata.scale rather than the scale local that the surrounding code establishes for encrypted columns.
The widening path skips the magic-number and version validation that the matching path gets via GetCountsOrThrow.
Capabilities.Float16VectorType is never read anywhere in src/, so a SqlVector<Half> on a v1-negotiated connection fails server-side rather than client-side. (Float32VectorType was already dead in main; this adds a second.)
The negotiation theory's 0x3 case doesn't test what its comment claims — the simulated server caps the ack itself, so the client's own ceiling check at SqlConnectionInternal.cs:1660 stays untested.
Docs say narrowing "fails for values outside its range"; the code saturates to ±Infinity and leaves it to the server.
MetaData is dereferenced without a null check in CreateSourceColumnMetadata.
Things I checked and found correct
Worth recording, since they're the parts most likely to be wrong in a change like this:
Subnormal widening, signed zero, overflow to infinity, the flush-to-zero boundary (2⁻²⁵ ties to even → 0, 2⁻²⁶ flushes), binary32 subnormal inputs, and the rounding carry into the exponent are all correct. Compiling Manual* on every TFM so the netfx path is under test on .NET too is a good arrangement.
Bulk copy null handling is correct as-is — GetValueFromSourceRow returns DBNull.Value with isNull = true and ConvertValue returns before ConvertVectorToBaseType. Note a5de733c3 is test-only; it documents behaviour that already worked rather than fixing anything.
The raw-payload path can't be taken by DataTable, DataRow[], or non-SqlClient DbDataReader sources (they keep ValueMethod.GetValue), and reordering column mappings are safe because sourceOrdinal is the mapped ordinal.
No unacknowledged breaking changes beyond the three listed: GetDataTypeName is unchanged, GetFieldType/GetProviderSpecificFieldType both route through the single new GetVectorFieldType so they stay consistent, SqlMetaDataFactory.DataTypes only adds a row (gated on MinimumVersionKey), the float32 declaration is deliberately unchanged, and SqlDbColumn's new indexer falls through to base[property].
No shared mutable state: Float16Converter is stateless and SqlVector<T> is a readonly struct with no static caches.
On testing
Taking as given that CI has no float16-capable server and no Azure SQL DB connectivity, so the manual suite is the only gate that will ever run — I looked at whether it is complete enough for a lab run rather than whether CI covers it. It is substantial: 11 behaviour tests plus the inherited NativeVectorTestsBase matrix, and the sample data is well chosen (Half.MaxValue, Half.Epsilon, -0.0f, exactly-representable eighths). Gaps I'd close:
.NET Framework gets none of the NativeVectorTestsBase matrix — NativeVectorFloat16Tests.cs is entirely #if NET. That is where the hand-rolled Manual* codec is the production path.
No async coverage for the new representation on any framework, and none at all for float16 on netfx.
DataTestUtility.CheckVectorFloat16Supported fails open through the code under test. It reads the probe vector with GetString + JsonSerializer.Deserialize and catches JsonException → false. A driver regression that produces malformed JSON silently skips the whole float16 suite green. Since the manual run is the only gate, that is the wrong failure mode. (Outside this diff, so no inline comment — but worth fixing alongside. It also leaves PREVIEW_FEATURES = ON on the shared test database as a side effect.)
Two range tests assert only that someSqlException was thrown; they'd pass on an unrelated failure, and they can't distinguish "client rejects" from "client saturates and server rejects" — which is exactly the ambiguity in the doc wording above.
Nothing covers the blocking issue: reading a float32 column as SqlVector<Half>, for a NULL and a non-NULL row.
Bulk copy with a dimension-count mismatch between source and destination is uncovered. Mitigating: float32→float32 now routes through the new raw-payload path too and is covered by the existing NativeVectorFloat32Tests, which runs against any vector-capable server — so regression risk to shipped functionality is covered.
Minor
SqlVector<T>.ToString() changing for existing SqlVector<float> callers is justified and correctly surfaced in the ref assembly and docs, but it is unrelated to float16 — it wants its own release-note entry as a behavioural break, not just an API-list line. Related: GetString() uses JsonSerializer.Serialize, which throws on NaN/Infinity by default, and on .NET Framework that is now the default GetValue() path.
ConvertPayloadElementType is internal static on SqlVector<T> but never uses T, so callers write SqlVector<float>.ConvertPayloadElementType(...), which reads as though it returns a float32 result.
On .NET Framework the reader surfaces a float16 column as string while an output parameter surfaces it as SqlVector<float>. Self-consistent, but worth documenting.
Preprocessor directives in the new code are indented to the surrounding block; the dominant style in these files is column 0.
ConnectionCapabilities.cs:179 says vectors were "introduced in SQL Server 2022" — pre-existing, but the new Float16VectorType doc sits right beside it.
Worth confirming the doc/samples build resolves the locally-packed driver: SqlVectorFloat16Example.cs references SqlVector<Half> and GetSqlVector<Half>, which exist in no released package.
Review assisted by GitHub Copilot; findings verified against the code at a5de733c3.
@apoorvdeshmukh and @cheenamalhotra, I think we will need to call this API a known limitation for preventing backward migration from .Net Runtime to NetFx.
BTW, I saw changes to ref assembly. Should I expect two copies of changes, one for netcore and another for NetFx? I am curious about how the APIs will show up in contract assemblies targeting 2 different frameworks.
Correctness
- Reject a narrowing read consistently for null and populated rows. The
element type is now checked before the null check in GetSqlVector<T>, so
reading a float32 column as SqlVector<Half> fails for every row rather
than succeeding for the null ones.
- Validate the vector header's magic number and version on the widening and
payload conversion paths, which previously checked only the length.
- Quieten a signalling NaN when widening to single precision, and preserve a
NaN's sign and payload in both directions, so the hand written codec and
System.Half agree on all 65,536 bit patterns.
- Read the redirected scale when converting a bulk copy value, so an
encrypted column uses its base type rather than the wrapping metadata's.
- Guard against a null MetaData when deciding whether a bulk copy source can
supply a raw vector payload.
Behaviour
- Report a value which cannot be narrowed to float16 during a bulk copy as an
OverflowException, rather than saturating it to an infinity and letting the
server reject the result as a malformed vector.
- Remove the unused Float32VectorType and Float16VectorType capability
properties. The negotiated version is still recorded in VectorVersion. A
client side guard was considered in their place, but the server already
reports an unrecognised base type clearly, so the guard would only have
replaced a good error with a worse one, and would have made float16 fail
differently from float32 for the same cause.
- Return the JSON rendering of a float16 vector as a SqlString from the
provider specific accessors, so that every provider specific value remains
a type from System.Data.SqlTypes. GetValue continues to return a string.
Tests
- Run the whole native vector matrix against a float16 column through the
single precision representation, which covers .NET Framework, where the
hand written codec is the production path rather than a test double.
- Assert NaN bitwise rather than skipping it, which is what allowed the
codec divergence above to go unnoticed.
- Cover reading a float32 column as a narrower vector, for null and populated
rows, synchronously and asynchronously.
- Exercise the client's own feature extension version ceiling, by letting the
simulated server acknowledge a version regardless of what the client
requested. The existing case only proved the harness capped the version.
- Assert the server's error number for an out of range value rather than
accepting any SqlException.
- Show the column metadata driving a read for a caller which does not know
the schema in advance.
- Remove the SqlVector<T>.ToString() override added earlier in this branch.
It changed the rendering of the already shipped SqlVector<float> as well, and
the reader already exposes the JSON form through GetString and
GetFieldValue<string>. The internal GetString is unchanged.
- Move Float16Converter into the Microsoft.Data.Common namespace, matching the
folder it lives in and its neighbours there.
Docs
- State that a bulk copy reports an out of range narrowing itself, and
correct the SQL Server version for the float32 base type.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The reason will be displayed to describe this comment to others. Learn more.
Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.
Note
This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.
The comment on IsTextSourcedVectorColumn described an earlier design in which
the column was declared as a varchar(max) and the server parsed the JSON
array. The server rejects that declaration for a vector column, so the client
parses the text and rewrites the payload to the destination's base type
instead. The call site already says so; only this comment was left behind.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
❌ Patch coverage is 86.65480% with 75 lines in your changes missing coverage. Please review.
✅ Project coverage is 69.42%. Comparing base (8387aeb) to head (0b70737). ⚠️ Report is 4 commits behind head on main.
Three conflicts, all in code that main changed underneath this branch.
The json type is now reported from the negotiated JSONSUPPORT capability
rather than the server version (#4682), which is the same change this branch
made for the vector type. The two gates are independent, so the factory now
carries both and LoadDataTypesDataTables takes each.
In the simulated server tests, main rewrote the comment above
TestConnWithUserAgentFeatureExtension as an XML doc comment while this branch
added the vector feature extension tests above it. Both are kept.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The PR description claims validation across all 4,294,967,296 binary32 bit patterns, but this loop samples only every 1039th pattern. Either make the test exhaustive or describe this as sampled coverage so the stated verification scope matches what CI actually runs. src/Microsoft.Data.SqlClient/tests/ManualTests/SQL/VectorTest/VectorFloat16BehaviourTests.cs:60
The new test methods and private test helpers in this class have no XML <summary> documentation. Add behavior-focused summaries as required by the repository testing instructions, especially for the sync/async narrowing and bulk-copy scenarios.
Carry the vector type support setting through the SqlConnectionOptions copy
constructor. That constructor is used for User Instance connections, and the
enum's default is Off, so the setting was silently dropped there: the
VECTORSUPPORT feature extension was suppressed and vector columns regressed to
JSON strings even for the default V1. Covered by a test which fails without
the fix.
Make the float16 behaviour fixture's setup failure-safe, matching
NativeVectorTestsBase. xUnit does not call Dispose when a constructor throws,
so a table created before the failure would have been left behind, and cleanup
now tolerates partially initialized fields.
Document the float16 to float32 widening on GetSqlVector<T>, whose remarks
still said no conversions are performed, and note in IsTextSourcedVectorColumn
that a float16 column on .NET Framework describes itself as a string and so
takes the textual path rather than being rejected.
Add the XML summaries the testing instructions require to the test types and
methods added by this branch.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The reason will be displayed to describe this comment to others. Learn more.
Copilot review overview
🟡 Changes recommended
Valid larger float16 vectors and mixed-negotiation bulk-copy scenarios currently fail, with additional test-documentation and coverage issues identified.
Get a fresh assessment by requesting another Copilot review.
When the destination connection does not negotiate float16 support (for example, Off or V1), a float16 target is exposed as varchar(max), so this condition is false even when the source reader negotiated V2. On .NET, GetValue then supplies SqlVector<Half> to the varchar coercion, which falls through Convert.ChangeType and fails instead of sending the JSON representation for server conversion. Handle a native vector source when the destination is textual.
Binary32 validation samples patterns instead of exhaustive coverage
This loop advances by 1039, so it checks only a small sample of the 2^32 binary32 bit patterns, not all 4,294,967,296 patterns claimed in the PR description. Many exponent/mantissa boundaries are therefore untested. Either make the exhaustive validation genuinely exhaustive (for example, partition it) or correct the stated coverage.
New test methods lack required XML summary documentation
The test methods in this new class, starting here, omit behavior-focused XML <summary> documentation, as do the remaining [ConditionalFact]/[ConditionalTheory] methods below. The repository testing guidance requires a summary on every test method; please document each new test (and the helper methods where applicable).
The element count limit is the number of elements of T whose payload fits in a
TDS packet, so it is a property of the type a value is sent as: 1998 for
float32 and 3996 for float16. It was being applied to values coming from the
server as well, where the destination type is not what governs the size.
A float16 column declaring more than 1998 dimensions was therefore unreadable
as SqlVector<float>, which on .NET Framework means unreadable by any means,
since every read path there widens to float32. Null rows failed the same way
through CreateNull. A JSON string bulk copied into such a column failed too,
because it is parsed into float32 before the payload is rewritten for the
destination's base type, and that intermediate was held to the float32 limit
even though the value finally sent is float16.
Construction by a caller still enforces the limit, so a vector which cannot be
sent as T is never built from user input.
Also correct a typeparamref written as paramref in the GetSqlVector<T> docs,
which broke the build.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
The loop advances by 1039, so this test checks only about 4.1 million binary32 patterns rather than all 4,294,967,296 patterns claimed by the PR description. Since ManualFromSingle is the .NET Framework production path, the stated exhaustive validation is not actually provided; either make the coverage exhaustive or correct the claim and add targeted boundary cases.
The acknowledgement was bounded by the highest version the driver implements
rather than the version this connection requested, so a server which did not
cap its answer could raise a v1 connection to v2. That would return float16
columns in their binary form to an application which never opted in, which is
the change the keyword exists to prevent. Bound it by the requested version
instead.
Also decide whether a bulk copy source is textual from the value as well as
the column's declared type. A column declared as object reports nothing
useful while still yielding a JSON string row by row, so its value was coerced
to a float32 payload and then sent unconverted, and a float16 destination
rejected it as an invalid column length. Only a string counts, so a raw vector
payload is still never mistaken for text.
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e17ed782-c576-4cb7-9b4b-7ad286d7a7d0
When the destination connection is v1/off, the server exposes a vector column as varchar(max), so metadata.type is not Vector and this raw-payload branch is skipped. On .NET, a v2 float16 source's GetValue is SqlVector<Half>; the fallback then sends that struct through the varchar conversion path, where SqlParameter.CoerceValue cannot convert it to text. A bulk copy from a v2 float16 reader into a v1/off destination therefore fails instead of using the JSON representation. Handle vector sources when the destination is textual by reading/coercing their string representation.
Float32 test does not cover the claimed full binary32 space
This test does not validate the full binary32 space claimed by the PR and by its own summary: b += 1039 samples only about 4.1 million of 4,294,967,296 bit patterns. That leaves many rounding-boundary and special-value encodings untested, so either the exhaustive claim must be made true or the documentation should describe the sampling and add targeted boundary coverage.
This branch has not been deployed
No deployments
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Area\VectorUse this for issues that are targeted for the Vector feature in the driver.
7 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
apoorvdeshmukh
added
the
Area\Vector
saurabh500
left a comment
paulmedynski
added
Hotfix 7.1.0
Hotfix 7.0.3
cheenamalhotra
removed
the
Hotfix 7.0.3
Adds support for the
float16base type of the SQL Servervectordata type, on both .NET and .NET Framework.Behaviour
float16is exchanged in its binary form only when a connection opts in, through the newVector Type Supportkeyword (off|v1|v2, defaultv1). This mirrorsvectorTypeSupportin the JDBC driver, including its default. Atv1afloat16column is returned as avarchar(max)containing a JSON array, exactly as today, so upgrading the driver changes nothing until an application asks forv2.At
v2:GetValue/GetFieldTypeSqlVector<Half>string(JSON array)GetSqlVector<Half>GetSqlVector<float>— widening, which is exactSqlVector<Half>,SqlVector<float>, or a JSON stringSqlVector<float>or a JSON stringSystem.Halfdoes not exist on .NET Framework, so afloat16column is surfaced there as its JSON rendering rather than by substituting a different base type. A caller that wants a strongly typed value asks forSqlVector<float>explicitly.Column metadata
A vector column now reports its base type and dimension count through the existing
DbColumnindexer, with no new API:This is the only way to distinguish the two base types when a
float16column is surfaced as a JSON string, becauseGetFieldTypereportsstringfor such a column just as it does for avarcharone.Bulk copy
The
INSERT BULKstatement states the destination column's base type, and the server then requires a binary payload of exactly that width and performs no conversion within the data stream. Measured at every negotiated version: a text declaration for a vector column is refused withInvalid column type from bcp client, and text sent under a vector declaration is refused with a length mismatch. Only a client which never negotiates the feature extension may send text.So a textual source is parsed into the destination's base type by the driver, and a payload read from another vector column keeps its own base type — copying between columns of different base types is reported by the server rather than silently narrowed. Both match the JDBC driver.
Public API
One addition: the
SqlVectorTypeSupportenum andSqlConnectionStringBuilder.VectorTypeSupport.SqlVector<T>is unchanged.Tests
float16added to the existing generic native vector suite, run over av2connectionSqlVector<float>on every framework, which is the .NET Framework representation and where the hand written binary16 codec is the production pathSystem.Halfacross all 65,536 binary16 and all 4,294,967,296 binary32 patternsChecklist
Notes for reviewers: the second commit reworks bulk copy and negotiation following a design review, so it is worth reading over the first. One review thread is left open pending that discussion.