Skip to content
temperlangPublic

About

Natural human ordering library for multiple programming languages

Resources

Stars

0 stars

Watchers

0 watching

Forks

Latest commit

 

History

18 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Ordable

Sorts strings in smart order, the way people expect: file2 before file10, since runs of digits compare as numbers. It's often called natural or human sorting; Ordable says "smart", since in Java and Kotlin "natural order" (naturalOrder()) means a type's own order, which for strings is the plain code-unit order Ordable replaces.

Examples

In Temper, and the same API in each supported language:

let names = ["img12.png", "img10.png", "img2.png", "img1.png"];
sortSmart(names); // img1.png img2.png img10.png img12.png

compareSmart("file2", "file10"); // Negative: file2 comes first
compareSmart("a", "B"); // Negative: case only breaks ties

// Any items, by a string from each, such as files by name.
sortSmartByKey(files) { file => file.name };

// A string that sorts in smart order by plain comparison, as in a database.
smartKey("file10"); // Sorts after smartKey("file2")

// Options: an Order has the same functions as methods.
let caseFirst: Order = { casing: Casing.first, upperFirst: true };
caseFirst.sort(names); // Every IMG before every img, as plain sorting does
caseFirst.sortByKey(files) { file => file.name };

Also: a tour of the whole API in each language, with identical output, and a site to try smart sorting in the browser, beside plain sorting and Intl.Collator.

Rules

  • Runs of decimal digits compare by value, at any length, without overflow. Digits in any script count (Unicode 15.0), such as Arabic-Indic ٢ or fullwidth 2, and a run is all from one script.
  • Letters compare without their accents or case first, so é and É sort with e, and letters fold fully, so Σ matches ς and ß matches ss. An accent written as a combining mark after its letter (e and U+0301) counts as the precomposed letter would (é), just after it. Everything else compares by code point.
  • Two strings compare equal only if they're the same, so sorting gives one answer whatever the input order. Leading zeros (1 before 01), the digits' script (1 before ١), accents (e before é), case (a before A), and odd forms (K before the Kelvin sign) only break ties, in that order.
  • Options, in an Order: when case is considered, its casing, from Casing.ignored (File equals file), through Casing.last (the default: case only breaks ties), to Casing.first (case before the letter, so every img comes before every IMG); upperFirst, for uppercase first wherever case counts; ignoreAccents (é equals e); and ignoreNumberForms (01, 1, and ١ are equal, as with Intl.Collator's numeric option). With these, different strings can compare equal, with the same keys.
  • No locale rules: that's for ICU or the platform.

Ordable's Unicode tables (decimal digits, case folding as 202 compact runs plus 104 full foldings, 977 accented letters' base letters, 30 odd forms, combining marks as 346 runs, and 905 compositions of a letter and a mark) are generated by uv run scripts/unicode_tables.py, which checks them against Python's Unicode data for every code point, pinned to Unicode 15.0.0. They're internal, so Ordable can switch to Unicode support built into Temper later without changing its API. Smart keys depend on them, so a release that changes keys (a new Unicode version, say) will say so.

Storing keys

A smart key (smartKey, or key on an Order) sorts by plain comparison exactly as its string sorts in smart order, so it can be stored and sorted anywhere, such as a database column with ORDER BY:

  • Compare keys as plain code points. In a database, give the column a binary collation: COLLATE "C" in PostgreSQL, a binary collation such as utf8mb4_bin in MySQL, or SQLite's default. A language-aware or case-insensitive collation would reorder keys.
  • Keys contain control characters, such as U+0001 between levels, so they're not for XML 1.0, which forbids them, nor for showing to people. They stay below U+D800 and never contain U+0000, so UTF-8, UTF-16, and code points all order them alike.
  • Keys say what made them. Each starts with 0, for this key format (later formats count up from it), and then a code point for the options that made it, so keys from other options or another format are easy to spot. Compare only keys made the same way.
  • Keys don't change by accident: every backend's tests check a frozen set of exact keys (key vectors). A change to keys would come with a new format code point and a new major version.

Roadmap

  • Done: smart order with numbers of any length, a total order, sorting by key, decimal digits in any script, and Unicode case folding.
  • Compared with the usual tools: see docs/comparisons.md.
  • Done: smart sort keys (smartKey), which sortSmart uses for lists past a few items, and which Python can sort by and databases can ORDER BY.
  • Done: accents sort with their base letter (e, é, f), and ß matches ss.
  • Done: defaults as people expect, with case only breaking ties (lowercase first), and an Order class for options (casing and upperFirst).
  • Done: options that let different strings compare equal (ignoreAccents, Casing.ignored, and ignoreNumberForms, which makes 01, 1, and ١ equal, as Intl.Collator's numeric option does), with matching keys.
  • Done: Order.sortByKey, sorting any items by a string from each, with an order's options.
  • Done: decomposed text: e plus a combining accent sorts as é, just after it.
  • Next: smarter punctuation, checked against real-world samples; and more options: signed and decimal numbers.
  • A spec: the source as a readable specification of the default order and every option, with its tests beside it, and comparison tables so people can choose well.
  • Started: a showcase site, where you pick or type lines and see smart order beside plain sorting and Intl.Collator, with the code in every language.
  • Done: compact smart keys, usually about as long as their string: levels after the first only record what's unusual (accents, uppercase, leading zeros), so file10's key is 8 code points.
  • Quick tests, long runs on demand: the property tests compare every pair of generated strings, so generatedCount in src/properties.temper.md keeps them quick; raise it for a long run or a benchmark. Later, property testing and benchmarks across backends could be built into Temper itself.
  • Done: a key format to keep: a header naming the format and the options, frozen key vectors checked on every backend, and notes on storing keys.
  • Done: named it "smart" (sortSmart, compareSmart, smartKey): what's usually called natural sorting, since in Java and Kotlin "natural order" means a type's own compareTo, the plain order Ordable replaces.

Languages

Made in Temper, Ordable supports these languages, and its tests run on each:

  • C++
  • C#
  • Java
  • JavaScript
  • Lua
  • Python
  • Rust

Development

Written with Claude Code and Opus 5.5.

Run tests with Temper, on one backend while working:

temper test -b py

Or on all of them:

temper test -b cpp -b csharp -b java -b js -b lua -b py -b rust

Run the tour in every language against temper.out/, checking each prints the same text (see demos):

uv run demos/run.py

Build and preview the site, which uses the JS backend's output:

temper build -b js
npm ci --prefix site
uv run site/build.py
python3 -m http.server --bind 127.0.0.1 -d site/build

CI runs the tests and the tour on every backend for pull requests and pushes to main, using Temper's nightly dev build, and publishes the site from main. Source files in src/ are literate Markdown, with tests next to the code they cover.

About

Natural human ordering library for multiple programming languages

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages