Skip to content

About

No description, website, or topics provided.

Resources

Code of conduct

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Repository files navigation

Aerospike Python SDK

A high-performance, developer-friendly Python interface for Aerospike. A dual first-class sync/async Pythonic API with a chainable session model, fluent query builder, and AEL string filters layered over the Aerospike Python Native Client (PNC) with first-class free-threaded Python support.

Server requirement: The Aerospike Python SDK supports only Aerospike Server 8.2.0 and later.

AI coding agent entry point

PyPI package aerospike-sdk. Authoritative version: the root VERSION file, currently 1.0.0. Python 3.11+. Aerospike Server 8.2.0 or later only; earlier servers are not supported. Async-first: the top-level aerospike_sdk package is the async surface; aerospike_sdk.sync is an independent synchronous implementation of the same surface. Both connect through ClusterDefinition.

Three Aerospike Python packages exist and they are not interchangeable:

Package What it is
aerospike-sdk this SDK - fluent, async-first, AEL string filters
aerospike-native the Python Native Client (PNC) this SDK is built on
aerospike the legacy client - a different API entirely

Write aerospike_sdk idioms. Importing aerospike_native types is correct only for the surfaces this SDK does not yet wrap - see Known traps.

What to read, by task

Task Read first Authoritative for
Connecting, sessions, behaviors docs/guide/connecting.md client and session setup
Reads, streams, fast-path docs/guide/reads.md read shapes
Writes, verbs, TTL docs/guide/writes.md write verb semantics
List and map operations docs/guide/cdt-operations.md CDT builder chain
AEL filters, expressions docs/guide/expression-ael.md filter grammar and limits
String operations docs/guide/string-ops.md server-side string ops
Secondary indexes docs/guide/indexes.md index create and query
Transactions docs/guide/transactions.md multi-record semantics
Errors, result handling docs/guide/error-handling.md error strategy, RecordResult
Background ops, UDFs docs/guide/background-udf.md background execution
Client logging docs/guide/logging.md logger names and levels
Client metrics docs/guide/metrics.md snapshots, exporters, counters
Runtime configuration docs/guide/dynamic-sdk-config.md YAML/env-driven settings
Builder vs fast-path, AsyncPool docs/guide/performance.md the trade-offs
Measured throughput, methodology docs/guide/benchmarking.md the numbers
Exact signatures docstrings in aerospike_sdk/ the API contract

Repository map

aerospike-client-python-sdk/
|__ README.md             entry point (this file)
|__ AGENTS.md             pointer to the section above
|__ VERSION               authoritative version
|__ aerospike_sdk/        the SDK - py.typed, inline type hints
|   |__ aio/              async client, session, operations
|   |__ sync/             independent sync implementation
|   |__ policy/           Behavior and policy configuration
|   |__ dataset.py, exp.py, record_stream.py, record_result.py,
|       error_strategy.py, exceptions.py, operation_result.py, ...
|__ docs/
|   |__ guide/            15 hand-written guides - start here
|   |__ api/              Sphinx autodoc stubs, one per class (+ sync/ mirror)
|   |__ conf.py           Sphinx config; Read the Docs builds from this
|__ examples/             25 runnable programs
|__ tests/
|   |__ README.md         which suite is authoritative for what
|   |__ unit/             77 files - no server required
|   |__ integration/      104 files - needs a running server
|__ benchmarks/
|__ Makefile              test, test-unit, test-int, coverage, docs targets

API reference: https://aerospike-python-sdk.readthedocs.io/ Guides: https://aerospike.com/docs/develop/client/sdk/

Precedence when sources disagree

The code can move faster than the prose:

  1. Docstrings and inline type hints in aerospike_sdk/ for signatures. The files under docs/api/ are thin autodoc wrappers, so the docstrings are the reference.
  2. tests/ for actual behavior, including edge cases - tests/README.md says which suite answers what
  3. examples/ for idiomatic usage
  4. docs/guide/ for concepts and the task map
  5. aerospike.com/docs for server-side semantics and version gates
  6. aerospike-native documentation for surfaces this SDK does not wrap

A guide that contradicts a docstring is stale, not authoritative. Report it.

Known traps

  • Do not mix legacy aerospike idioms into SDK code. No aerospike.client(...), no policy dicts, no aerospike_helpers operation functions. This SDK uses verbs on a session, Behavior for configuration, and AEL strings for filters.
  • Path expressions are fully SDK-surfaced. Write them as AEL strings ("$.l:LIST.*[?(@:INT > 200)]") in where() / select_from() / upsert_from(), or build them with CdtOperation.select_by_path / modify_by_path / remove, the flag enums, CTX.all_children(), and the loop-variable family from aerospike_sdk directly - see docs/guide/expression-ael.md. Removal of matching elements is supported (CdtOperation.remove); do not conclude it does not exist.
  • Importing aerospike_native types is correct in exactly one documented case: the low-level exception types described in docs/guide/error-handling.md. Everywhere else, the aerospike_sdk re-exports cover it - aerospike_sdk.Exp IS the low-level FilterExpression. (The string "aerospike_native" as a logger name in logging config is fine - that is a name, not an import.)
  • Fast-path is single-key only. session.get(key) / session.put(key, bins) take no filters, no error-handler callbacks, and no batch semantics. Use the chained builder when you need any of those.
  • Do not loop single-key calls where a batch form exists. DataSet.ids(...) builds a key list; pass it rather than iterating DataSet.id(...).
  • Do not hand-roll batch fan-out. A node sub-batch of size 1 is already sent as a single-record command, automatically and per-node.
  • AsyncPool is a free-threading feature. On regular CPython it is slower than a single client.

Verifying generated code

make test-unit          # no server required
make coverage           # unit tests plus a line-coverage report
make test-int           # needs a running Aerospike server
python3 examples/basic_example.py

Aerospike agent skills

aerospike/agent-skills carries core-database and data-modeling guidance - key design, record sizing, collection choice, indexing. Complementary to this repo, which is authoritative for the SDK API.

Resources

Installation

pip install aerospike-sdk

Pin to a specific release if you need reproducible builds:

pip install aerospike-sdk==1.0.0

This installs the SDK plus its dependency on the Aerospike Python Native Client (aerospike-native). No Rust toolchain or git checkout required for ordinary use — pre-built wheels are available for Linux, macOS, and Windows on Python 3.11–3.15.

Quick start

import asyncio
from aerospike_sdk import Behavior, ClusterDefinition, DataSet


async def main():
    async with ClusterDefinition("localhost", 3000).connect() as cluster:
        session = cluster.create_session(Behavior.DEFAULT)
        users = DataSet.of("test", "users")

        # High-level key-value writes
        await session.upsert(users.id(1)).put({"name": "Alice", "age": 28, "country": "UK"}).execute()
        await session.upsert(users.id(2)).put({"name": "Bob", "age": 35, "country": "US"}).execute()

        # Filtered query with AEL — streams results memory-efficiently
        results = await (
            session.query(users)
            .where("$.age > 25 and $.country == 'US'")
            .execute()
        )
        async for row in results:
            if row.is_ok and row.record is not None:
                print(row.record.bins)

        # Or drain the entire stream into a list
        all_users = await session.query(users).execute()
        rows = await all_users.collect()


asyncio.run(main())

The same surface is available without asyncio — no async/await, no event loop. See Sync usage in the connecting guide for the sync variant of this program.

See the Quick Start guide for a deeper walkthrough; the API reference covers every public class and method in detail.

Examples

Runnable, self-contained scripts live in examples/ — one file per topic (query_examples.py, string_operations_example.py, batch_example.py, common_example.py, transaction_example.py, the SDK-config set, and more). Each is a standalone program; run one directly or run them all:

python examples/query_examples.py     # a single example
make examples                         # every example, in sequence

Every example opens its connection through the async context-manager convention shown in Quick start — async with _env.connect().connect() as cluster: (sync examples use with _env.sync_connect().connect() as cluster:) — so the cluster is always closed cleanly on exit.

examples/_env.py is a small examples-only helper (not part of the published package — the mirror of benchmarks/_env.py). It resolves connection settings from the environment so the scripts run with no edits and no manual exports: it loads aerospike.env if you made one, otherwise the committed aerospike.env.example, without clobbering variables you already exported. _env.connect() / _env.sync_connect() return a ClusterDefinition built from AEROSPIKE_HOST (default localhost:3000). A production application constructs ClusterDefinition directly — as the Quick start does — rather than importing _env.

A few examples need more than a default AP cluster and degrade to a clean skip message when it is absent:

  • Strong-consistency examples (transaction_example.py, roster_example.py) connect via _env.connect_sc(), which reads AEROSPIKE_HOST_SC (+ AEROSPIKE_AUTH_* credentials) and the SC namespace from AEROSPIKE_SC_NAMESPACE (default test_sc).
  • Server-version-gated examples (e.g. string_operations_example.py, server 8.2.0+) check _env.server_at_least(session, (8, 2, 0)) and skip if the cluster is older.

Connection variables are the same ones the test suite uses — see Configuration for aerospike.env.

Performance

Two API shapes: the chained builder for filters, batch operations, and error handlers; the fast-path session.get/session.put for single-key work. For the full decision guide — including free-threaded Python and AsyncPool — see docs/guide/performance.md; for the measured numbers and methodology, see docs/guide/benchmarking.md.

Documentation

The two complement each other: the guide site introduces concepts and works through realistic examples, while the API reference is the exhaustive source for Client, Session, query/update builders, AEL, behavior policies, and every public symbol.

Versioning

PSDK follows SemVer. Pre-releases use the MAJOR.MINOR.PATCH-{alpha,beta,rc}.N form (e.g. 1.1.0-rc.1). PyPI normalizes these on upload to the equivalent PEP 440 spelling (1.1.0rc1).

The top-level VERSION file is the single source of truth; pyproject.toml reads it dynamically, so the wheel and the working tree are guaranteed to match. See the Development section below for the bump procedure.

Pre-release builds (Aerospike internal)

Every merge to stage publishes a wheel and an sdist to Aerospike's internal package index, versioned as a dev release leading toward the next release — 1.0.1.dev123, where 123 is the publishing workflow's run number. This is for Aerospike test teams and internal consumers who need a specific dev build; external users should use the public PyPI releases.

These builds are not on public PyPI, so installing one requires credentials for the internal index. Generate an identity token in the JFrog UI (avatar → Edit Profile → Identity Tokens); your username is your Aerospike email address, and the @ in it must be URL-encoded as %40:

export PIP_EXTRA_INDEX_URL="https://<you>%40aerospike.com:<identity-token>@artifact.aerospike.io/artifactory/api/pypi/database-pypi-dev-local/simple/"

Persist it in ~/.config/pip/pip.conf under [global] extra-index-url, or put the credentials in ~/.netrc for artifact.aerospike.io, if you'd rather not set it per shell.

With that in place, installing needs no repository checkout:

pip index versions aerospike-sdk --pre        # what's available
pip install "aerospike-sdk==1.0.1.dev123"     # a specific build

Pin the exact dev version rather than reaching for --pre --upgrade. If the index is unconfigured or the token has expired, --pre quietly resolves the newest public release instead and looks like it worked; an exact dev version fails loudly with "no matching distribution".

The same index also serves aerospike-native (PNC) dev builds, so one credential setup resolves both. Adding --only-binary aerospike-native is worth it on unusual platforms: it turns a missing PNC wheel into a clear resolution error instead of a slow source build that needs a Rust toolchain. Report bugs against the exact aerospike_sdk.__version__ you installed.

Benchmarking a dev build (Aerospike internal)

The benchmark tools live in benchmarks/ in this repository and are not part of the published package — they are development tooling, with their own shell scripts and a Rust helper project. To benchmark a published dev build, install the package from the index (per Pre-release builds above) and check out only the tools:

git clone --depth 1 --branch stage --filter=blob:none --sparse \
  https://github.com/aerospike/aerospike-client-python-sdk.git psdk-bench
cd psdk-bench
git sparse-checkout set benchmarks           # directories only; root files come free

pip install "aerospike-sdk==1.0.1.dev123"

export AEROSPIKE_HOST=10.0.0.5:3000
export AEROSPIKE_USE_SERVICES_ALTERNATE=false
python -m benchmarks.benchmark -w RU,50 -k 100000 -z 32 -d 10

Nothing here needs Rust: the tools are plain scripts against the installed package.

Connection settings come from the environment. benchmarks/_env.py loads aerospike.env if you made one and otherwise the committed aerospike.env.example, skipping any key already exported — so exported values win and there is no file to edit. Set AEROSPIKE_USE_SERVICES_ALTERNATE explicitly rather than inheriting it: the example file ships true for container setups, and using alternate access addresses against a cluster that doesn't publish them strands the client on a single node, where most reads then fail to route.

Take the tools from the same branch the build came from. --depth 1 otherwise clones the default branch (main), and dev builds are published from stage, so bench flags introduced alongside a new SDK feature — --mode async-many, for instance — may not exist in main's copy of the tools yet, and the run dies on an unknown flag. For a build published by a manual dispatch from a feature branch, or when you want exact parity, use the git revision recorded in that build's JFrog build-info.

Use a sparse checkout rather than a full clone. benchmarks/benchmark.py prepends its parent directory to sys.path, so in a full checkout the repository's own aerospike_sdk/ shadows the installed package — at worst it silently measures your working tree instead of the build you pinned. With only benchmarks/ checked out there is nothing to shadow.

python -m benchmarks.compare is a maintainer tool rather than a tester one: it drives several client repositories side by side and expects a pyenv environment per repository. See benchmarks/README.md for the full flag reference and docs/guide/benchmarking.md for methodology.

License

Apache License 2.0. See LICENSE for details.


Development / Contributing

The sections below are for SDK contributors. Downstream users do not need any of this — pip install aerospike-sdk is sufficient to use the package.

Prerequisites

  • Python 3.11 - 3.15, or 3.14t / 3.15t (free-threaded) for high-throughput / AsyncPool work. The SDK supports every CPython version under upstream security support; the floor rises in minor releases as versions reach end-of-life. Recommended installer: uv (uv python install 3.14.5+freethreaded) or pyenv with a dedicated environment. Free-threaded PNC wheels (cp314t, cp315t) ship for Linux (x86_64, aarch64) and macOS arm64. (PyO3 0.29 dropped 3.13t support; PSDK's free-threaded build starts at 3.14t.)
  • Aerospike server — required for integration tests
  • Rust toolchain (rustc + cargo) — required only when building the Aerospike Python Native Client from source (e.g. for an unreleased PNC feature)

Setting up a dev environment

pip install -e ".[dev]"    # install with dev extras

On the stage branch, the pinned aerospike-native (PNC) version may be a pre-release build published to Aerospike's internal package index rather than public PyPI. When it is, the plain install above needs one extra step that depends on who you are:

External contributors: the internal index requires Aerospike credentials, but PNC's source is public — build it locally per Local PNC checkout below (requires a Rust toolchain), then install this SDK with --no-deps. Released versions of aerospike-sdk on public PyPI depend only on public PyPI packages and need none of this.

Aerospike engineers: configure the internal index once, per Pre-release builds above, then:

pip install -e ".[dev]"

CI does the equivalent with short-lived OIDC credentials; ReadTheDocs builds need PIP_EXTRA_INDEX_URL set as an environment variable in the RTD project dashboard.

Local PNC checkout

To build against a local Aerospike Python Native Client working tree — whether because you're changing PNC itself or because you don't have access to the internal index — install it editable first and pass --no-deps to this SDK so pip doesn't try to resolve the exact PNC pin from an index:

pip install -e /path/to/aerospike-client-python-native
pip install -e ".[dev]" --no-deps

Or use requirements-local.txt (edit the file: path for your machine).

Configuration

Copy aerospike.env.example to aerospike.env in the repo root and adjust hosts or ports. aerospike.env is not committed.

cp aerospike.env.example aerospike.env
source aerospike.env

Pytest loads aerospike.env when present; otherwise conftest.py loads aerospike.env.example for unset variables only (so CI env vars still win).

Running tests

make test          # all tests
make test-unit     # unit tests only
make test-int      # integration tests only (requires running Aerospike server)

macOS file descriptor limit. On macOS, you may encounter OSError: [Errno 24] Too many open files when running the full test suite. The default limit (256) is not enough for the concurrent async connections created during testing.

ulimit -n 4096

To make this permanent, add it to your shell profile (~/.zshrc or ~/.bash_profile).

Building docs locally

API docs are built with Sphinx (Furo theme, MyST-Parser for Markdown). The same Sphinx config is what Read the Docs builds from.

pip install -e ".[docs]"   # one-time: install Sphinx toolchain
make docs                  # build static HTML to docs/_build/html/
make docs-serve            # live-reloading local preview

Docstrings use Google style with Sphinx cross-references (:meth:, :class:, etc.).

Lint

ruff check .

Bumping the version

Bumps are manual and happen in PRs against stage. Promotion workflows (stage → main) do not mutate the version.

# 1. Edit VERSION:
#    e.g. 1.0.0  →  1.0.1
echo '1.0.1' > VERSION

# 2. Confirm:
bin/get-version    # prints 1.0.1

# 3. Open a PR against stage with just this change.

Bumping the PNC pin

PSDK pins an exact Aerospike Python Native Client (aerospike-native) version in pyproject.toml under [project] dependencies. Releases must pin a public PyPI version:

[project]
dependencies = [
    "aerospike-native==1.0.0",
    # ...other deps
]

Between releases, stage may instead pin a dev-channel build from the internal index (e.g. aerospike-native==1.0.1.dev119) to pick up unreleased PNC work.

To bump: change the version, reinstall, and confirm the environment matches:

pip install "aerospike-native==<new version>"
make check-pin

Open the PR against stage. PSDK's own VERSION does not need to change for a PNC pin bump unless the underlying API contract has shifted enough to warrant it.

Reading the version programmatically

Anywhere a build script, CI step, or release tool needs the version:

bin/get-version    # → 1.0.0

The script reads VERSION and trims trailing whitespace. No Python or setuptools runtime dependency — usable from any shell, container, or CI environment.

About

No description, website, or topics provided.

Resources

Code of conduct

Security policy

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Used by

Contributors

Languages