From fcca9f998703a86643e2d1a34863ddc64ff25640 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Tue, 10 Nov 2026 23:27:50 -0800 Subject: [PATCH 01/24] ADFA-7083: Add sqlite-js documentation pages (draft) Four Help+Manual style pages for documentation.db, under i/ and linked from the Terminal page: an overview with install, load and safety, the sqlite-js function reference, how SQL and JavaScript values convert and what the engine supports, and a JavaScript introduction. pages/*.md is the source. build.py turns it into the JSON facts that helpmanual-topic.peb renders; apply.py adds the rows to a copy of documentation.db in one transaction, compressed against its dictionary, and appends a link to i/terminal-top.html. Run on a copy of the current database: 4 rows added, 1 changed, integrity_check ok, rerun is idempotent. test_examples.py runs every SQL example and checks links against the sqlite-js 1.3.7-1 host build: 32 of 32 pass. Not verified yet: the technical claims (see CLAIMS.md for each and how it was checked), the pages in the app, and the phone install steps. The table of contents and search index are Help+Manual output and do not list the new pages yet. Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 66 ++++++ docs/docdb/ADFA-7083/README.md | 24 +++ docs/docdb/ADFA-7083/apply.py | 109 ++++++++++ docs/docdb/ADFA-7083/build.py | 193 +++++++++++++++++ .../docdb/ADFA-7083/pages/javascript-intro.md | 119 +++++++++++ .../ADFA-7083/pages/sqlite-js-functions.md | 201 ++++++++++++++++++ .../ADFA-7083/pages/sqlite-js-javascript.md | 123 +++++++++++ docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 79 +++++++ docs/docdb/ADFA-7083/test_examples.py | 59 +++++ 9 files changed, 973 insertions(+) create mode 100644 docs/docdb/ADFA-7083/CLAIMS.md create mode 100644 docs/docdb/ADFA-7083/README.md create mode 100644 docs/docdb/ADFA-7083/apply.py create mode 100644 docs/docdb/ADFA-7083/build.py create mode 100644 docs/docdb/ADFA-7083/pages/javascript-intro.md create mode 100644 docs/docdb/ADFA-7083/pages/sqlite-js-functions.md create mode 100644 docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md create mode 100644 docs/docdb/ADFA-7083/pages/sqlite-js-top.md create mode 100644 docs/docdb/ADFA-7083/test_examples.py diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md new file mode 100644 index 0000000000..ae51c47e3c --- /dev/null +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -0,0 +1,66 @@ +# Claims in the ADFA-7083 pages and how each was checked + +`run` = executed by `test_examples.py` against the sqlite-js 1.3.7-1 host build. `code` = read in the patched source. `TODO` = still to verify, for the accuracy review. + +## sqlite-js-top + +| Claim | Check | +|-------|-------| +| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | TODO: package names and apt behavior on a phone (ADFA-7081 phone test) | +| `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | TODO phone; same claim as sqlite-brotli's README, verified there | +| `js_create_scalar` prints 0; slugify and titles examples | run | +| A function lasts until `sqlite3` closes | run (functions page) | +| `js_init_table(1)` runs stored code; `db.exec` can run any SQL including `readfile()`/`writefile()` | code; TODO: `readfile`/`writefile` exist in the Termux `sqlite3` shell | +| Ctrl-C stops code, query fails with `interrupted` | run (package run-tests AC8) | +| 64 MB memory limit; `out of memory` | run (AC9) | +| Do not put `.load` in `~/.sqliterc` | advice, follows from the above | + +## sqlite-js-functions + +| Claim | Check | +|-------|-------| +| Each `js_` function but `js_version` works only in top-level SQL | run (AC10, flags check) | +| Examples for scalar, aggregate (per-group state), window, collation, `js_eval`, `js_init_table`, `js_version` | run | +| Redefining in the same session fails with the quoted message | run (package AC4) | +| `init` can be NULL | code (`init_code` optional); TODO: add a run example | +| `value`/`inverse` semantics | code, run | +| Collation: only the sign counts; a throwing collation compares as equal | code (patch history, 1.3.6 changelog); TODO run | +| `js_init_table()` columns and "whenever that table exists, each function you create is stored" | code (`js_add_to_table`: returns early if the table is missing); TODO: run a case where the table exists from an earlier session and `js_init_table()` is not called | +| `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | code; TODO run | +| `js_set_memory_limit`: default 64 MB, 0 no limit, integer only, error text, returns NULL | run (package AC9), code | +| `js_set_max_stack_size`: default 1 MB; 0 crashes deep recursion | code (`JS_DEFAULT_STACK_SIZE`); run by hand: size 0 + recursion segfaults, default gives "Maximum call stack size exceeded" | +| `js_version(1)` is quickjs-ng | run (package AC2) | +| The six fixes listed | code, package tests | +| Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | + +## sqlite-js-javascript + +| Claim | Check | +|-------|-------| +| Engine supports const/let, arrow functions, classes, template strings, destructuring, Map/Set, BigInt, regexes, JSON, Math, Date, typed arrays | run: class/destructuring/template/map example; TODO: probe Map, Set, BigInt, Date, typed arrays individually | +| `window`, `fetch`, `setTimeout`, `require`, `process` undefined | run (also `document`, `alert`, `Buffer` named in prose: TODO) | +| Promises never finish | run (package AC11) | +| "JavaScript has no file, network or process functions of its own" | run (AC11: `std`, `os` not reachable) | +| `console.log` and `print` write to `sqlite3` standard output | run by hand (`print` TODO) | +| SQL-to-JS and JS-to-SQL conversion tables | run (package AC6), code | +| 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` | run | +| Parameters convert like results | run (package AC6) | +| Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own | run (AC3) | +| Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | +| Stack overflow message `Maximum call stack size exceeded` | run by hand | +| `js_create_` reports syntax errors when the function is created | run (package AC5) | + +## javascript-intro + +| Claim | Check | +|-------|-------| +| Every snippet and its output | run | +| `const` is never reassigned, `let` can be; arrow function syntax | JavaScript language facts | + +## Format + +| Claim | Check | +|-------|-------| +| Pages render through `helpmanual-topic.peb` | rendered with Jinja2 as a Pebble stand-in: title, body, breadcrumbs present. TODO: render in the app | +| Phone layout of tables and code blocks | body-only render in headless Chrome (code blocks wrap, tables fit); TODO: on a device at font scale 1.0 and 2.0 | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md new file mode 100644 index 0000000000..69f8881570 --- /dev/null +++ b/docs/docdb/ADFA-7083/README.md @@ -0,0 +1,24 @@ +# ADFA-7083: sqlite-js documentation pages + +Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached from the Terminal page (`i/terminal-top.html`): + +| Page | Path | +|------|------| +| JavaScript in SQLite (sqlite-js) | `i/sqlite-js-top.html` | +| sqlite-js function reference | `i/sqlite-js-functions.html` | +| JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | +| JavaScript basics | `i/javascript-intro.html` | + +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-1. Pages for apps follow ADFA-7082. + +- `pages/*.md`: the source, a small Markdown subset (see `build.py`). +- `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. +- `apply.py DB`: adds the pages to a `documentation.db` in one transaction, compressed against its `CompressionDictionary`, and appends a link to `i/terminal-top.html`. Run it on a copy first; the master is edited by a `docdb-studio` maintainer. Needs the `brotli` command. +- `test_examples.py`: runs every ` ```sql ` example against a real `sqlite3` with sqlite-js loaded, and checks that links resolve: `SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py`. +- `CLAIMS.md`: every factual claim in the pages, and how each was checked. + +## Not done here + +- The table of contents (`i/hmcontent.html`), the keyword index and the search index (`i/zoom_*.js`) are generated by Help+Manual and do not list these pages. The pages are reachable from `terminal-top.html` and from each other. Regenerate those files with the next Help+Manual export. +- Attribution: `sqlite-js-functions.html` adapts the upstream README (Apache-2.0) and carries the notice. The other pages are original text. `javascript-intro.html` reuses no MDN text. +- External addresses appear as text, not links: documentation stays on the device. diff --git a/docs/docdb/ADFA-7083/apply.py b/docs/docdb/ADFA-7083/apply.py new file mode 100644 index 0000000000..f6b5dece9b --- /dev/null +++ b/docs/docdb/ADFA-7083/apply.py @@ -0,0 +1,109 @@ +#!/usr/bin/env python3 +"""Adds the ADFA-7083 pages to a documentation.db. + +Usage: python3 apply.py DOCUMENTATION_DB + +Builds the pages (build.py), compresses each with Brotli against the +database's CompressionDictionary (quality 11, lgwin 24, as the documentation +pipeline does), and in ONE transaction: + - upserts a Content row per page, path i/, templateId of + helpmanual-topic.peb, text/html, EN-us; + - appends a link to the new pages to i/terminal-top.html, unless it has one; + - records the change in LastChange (content-i and wholedb). +Any failure rolls everything back. Safe to run again: it replaces its rows. + +Needs the `brotli` command-line tool. Run it on a copy first: this edits the +file in place, and the master lives in OfflineDocumentationTools' workflow. +""" +import json +import os +import shutil +import sqlite3 +import subprocess +import sys +import tempfile + +HERE = os.path.dirname(os.path.abspath(__file__)) +LINK_FILE = "terminal-top.html" +LINK_TARGET = "sqlite-js-top.html" +LINK_HTML = ('

To write SQL functions in JavaScript, see ' + 'JavaScript in SQLite (sqlite-js).

\r\n') +WHO = "ADFA-7083" + + +def brotli(args, data, workdir): + src, dst = os.path.join(workdir, "in"), os.path.join(workdir, "out") + for f in (dst,): + if os.path.exists(f): + os.remove(f) + with open(src, "wb") as f: + f.write(data) + subprocess.run(["brotli", "-f", "-o", dst] + args + [src], check=True) + with open(dst, "rb") as f: + out = f.read() + if not out: + raise RuntimeError("brotli produced no output") + return out + + +def main(): + db_path = sys.argv[1] + if not shutil.which("brotli"): + sys.exit("the brotli command-line tool is required") + work = tempfile.mkdtemp(prefix="adfa7083-") # mode 700 + try: + out_dir = os.path.join(work, "pages") + subprocess.run([sys.executable, os.path.join(HERE, "build.py"), out_dir], check=True) + + db = sqlite3.connect(db_path, isolation_level=None) + db.execute("PRAGMA foreign_keys = ON") + template = db.execute("SELECT id FROM Templates WHERE name = 'helpmanual-topic.peb'").fetchone() + html_type = db.execute("SELECT id FROM ContentTypes WHERE value = 'text/html'").fetchone() + lang = db.execute("SELECT id FROM Languages WHERE lower(value) = 'en-us'").fetchone() + dictionary = db.execute("SELECT data FROM CompressionDictionary WHERE id = 1").fetchone() + if not (template and html_type and lang and dictionary): + sys.exit("this does not look like a documentation.db: a template, content type, language or " + "dictionary is missing") + dict_path = os.path.join(work, "dict") + with open(dict_path, "wb") as f: + f.write(dictionary[0]) + + db.execute("BEGIN IMMEDIATE") + try: + for name in sorted(os.listdir(out_dir)): + with open(os.path.join(out_dir, name), "rb") as f: + data = f.read() + blob = brotli(["-q", "11", "-w", "24", "-D", dict_path], data, work) + db.execute( + "INSERT INTO Content (path, languageID, content, contentTypeID, templateId) " + "VALUES (?, ?, ?, ?, ?) ON CONFLICT(path) DO UPDATE SET " + "languageID = excluded.languageID, content = excluded.content, " + "contentTypeID = excluded.contentTypeID, templateId = excluded.templateId", + ("i/" + name[:-len(".json")], lang[0], blob, html_type[0], template[0])) + print("upserted i/%s (%d bytes)" % (name[:-len(".json")], len(blob))) + + row = db.execute("SELECT content FROM Content WHERE path = ?", ("i/" + LINK_FILE,)).fetchone() + if row is None: + raise RuntimeError("i/%s is not in the database" % LINK_FILE) + page = json.loads(brotli(["-d", "-D", dict_path], row[0], work)) + if LINK_TARGET in page["hmBody"]: + print("i/%s already links to %s" % (LINK_FILE, LINK_TARGET)) + else: + page["hmBody"] += LINK_HTML + blob = brotli(["-q", "11", "-w", "24", "-D", dict_path], + json.dumps(page, ensure_ascii=False).encode("utf-8"), work) + db.execute("UPDATE Content SET content = ? WHERE path = ?", (blob, "i/" + LINK_FILE)) + print("added a link to i/%s" % LINK_FILE) + + for doc_set in ("content-i", "wholedb"): + db.execute("INSERT INTO LastChange (documentationSet, who) VALUES (?, ?)", (doc_set, WHO)) + db.execute("COMMIT") + except BaseException: + db.execute("ROLLBACK") + raise + db.close() + finally: + shutil.rmtree(work, ignore_errors=True) + + +main() diff --git a/docs/docdb/ADFA-7083/build.py b/docs/docdb/ADFA-7083/build.py new file mode 100644 index 0000000000..6e9e244aba --- /dev/null +++ b/docs/docdb/ADFA-7083/build.py @@ -0,0 +1,193 @@ +#!/usr/bin/env python3 +"""Builds the ADFA-7083 documentation pages. + +Reads pages/*.md and writes one JSON file per page into the output directory, +in the shape documentation.db stores for the Help+Manual pages (templateId +3384, `helpmanual-topic.peb`): file, hmTitle, hmBody and the other facts. + +Usage: python3 build.py OUT_DIR + +A page is Markdown, a small subset of it, after a front matter block: + + --- + file: sqlite-js-top.html + title: Page title + description: One sentence. + parent: terminal-top.html + prev: terminal-top.html + next: sqlite-js-functions.html + --- + +Supported: `##` and `###` headings, paragraphs, `-` and `1.` lists, `>` notes, +pipe tables, fenced code blocks, `code`, **bold** and [links](page.html). A link to +an http(s) address is shown as text, because the documentation stays on the device. +""" +import html +import json +import os +import re +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) + +# Pages of the existing help that these pages hang under, outermost first. +ROOT_TRAIL = [("cogo-top.html", "Code on the Go"), ("terminal-top.html", "Terminal")] + +P_STYLE = { + "p_Normal": "", + "li": "text-indent: 0; padding-left: 1.2500rem; margin-left: 1.3125rem;", + "code": "margin: 0.1875rem 0.0625rem 0.1875rem 1.2500rem; white-space: pre-wrap;", + "note": "margin-left: 1.2500rem; padding-left: 0.5000rem; border-left: 0.1875rem solid #999999;", + "h2": "page-break-after: avoid;", + "h3": "page-break-after: avoid;", +} +CELL = "border: 1px solid #999999; padding: 0.2500rem 0.5000rem; text-align: left; vertical-align: top;" + + +def esc(s): + return html.escape(s, quote=False) + + +INLINE = re.compile(r"`([^`]+)`|\*\*((?:[^*`]|`[^`]*`)+)\*\*|\[([^\]]+)\]\(([^)]+)\)") + + +def inline(s): + out, pos = [], 0 + for m in INLINE.finditer(s): + out.append(esc(s[pos:m.start()])) + if m.group(1) is not None: + out.append('%s' % esc(m.group(1))) + elif m.group(2) is not None: + out.append('%s' % inline(m.group(2))) + else: + text, url = m.group(3), m.group(4) + if re.match(r"https?://", url): + # CoGo keeps documentation on the device: show the address, do not link it + out.append("%s (%s)" % (esc(text), esc(url))) + else: + out.append('%s' % (html.escape(url), esc(text))) + pos = m.end() + out.append(esc(s[pos:])) + return "".join(out) + + +def paragraph(text): + return '

%s

' % inline(text) + + +def list_item(marker, text): + return ('

%s%s

' + % (P_STYLE["li"], marker, inline(text))) + + +def code_block(lines): + text = "\n".join(lines) + return ('

%s

' + % (P_STYLE["code"], esc(text))) + + +def table(rows): + def cells(row): + return [c.strip() for c in row.strip().strip("|").split("|")] + head, body = cells(rows[0]), [cells(r) for r in rows[2:]] + out = [''] + out += ['' % (CELL, inline(c)) for c in head] + out.append("") + for row in body: + out.append("" + "".join('' % (CELL, inline(c)) for c in row) + "") + out.append("
%s
%s
") + return "".join(out) + + +def body_html(md): + blocks, lines, i = [], md.split("\n"), 0 + while i < len(lines): + line = lines[i] + if not line.strip(): + i += 1 + elif line.startswith("```"): + j = i + 1 + while not lines[j].startswith("```"): + j += 1 + blocks.append(code_block(lines[i + 1:j])) + i = j + 1 + elif line.startswith("## ") or line.startswith("### "): + level = 2 if line.startswith("## ") else 3 + text = line[level + 1:] + blocks.append('

%s

' + % (level, P_STYLE["h%d" % level], level, inline(text))) + i += 1 + elif line.startswith("- ") or re.match(r"\d+\. ", line): + while i < len(lines) and (lines[i].startswith("- ") or re.match(r"\d+\. ", lines[i])): + m = re.match(r"(\d+\.) (.*)", lines[i]) + blocks.append(list_item(m.group(1), m.group(2)) if m else list_item("•", lines[i][2:])) + i += 1 + elif line.startswith("|"): + j = i + while j < len(lines) and lines[j].startswith("|"): + j += 1 + blocks.append(table(lines[i:j])) + i = j + elif line.startswith("> "): + j = i + while j < len(lines) and lines[j].startswith("> "): + j += 1 + text = " ".join(l[2:] for l in lines[i:j]) + blocks.append('

%s

' % (P_STYLE["note"], inline(text))) + i = j + else: + j = i + while j < len(lines) and lines[j].strip() and not re.match(r"```|## |### |- |\d+\. |\||> ", lines[j]): + j += 1 + blocks.append(paragraph(" ".join(l.strip() for l in lines[i:j]))) + i = j + return "\r\n".join(blocks) + "\r\n" + + +def read_page(path): + text = open(path, encoding="utf-8").read() + m = re.match(r"---\n(.*?)\n---\n(.*)", text, re.S) + meta = dict(l.split(": ", 1) for l in m.group(1).split("\n")) + return meta, m.group(2) + + +def main(): + out_dir = sys.argv[1] + os.makedirs(out_dir, exist_ok=True) + pages = {} + for name in sorted(os.listdir(os.path.join(HERE, "pages"))): + if name.endswith(".md"): + meta, md = read_page(os.path.join(HERE, "pages", name)) + pages[meta["file"]] = (meta, md) + titles = dict(ROOT_TRAIL) + titles.update({f: m["title"] for f, (m, _) in pages.items()}) + parents = {f: m["parent"] for f, (m, _) in pages.items()} + + for file, (meta, md) in pages.items(): + trail, p = [], meta["parent"] + while p in titles and p != "cogo-top.html": + trail.insert(0, p) + p = parents.get(p, "terminal-top.html" if p != "terminal-top.html" else "cogo-top.html") + trail.insert(0, "cogo-top.html") + crumbs = " > ".join('%s' % (f, esc(titles[f])) for f in trail) + facts = { + "file": file, + "hmKeywords": meta.get("keywords", ""), + "hmTitle": meta["title"], + "hmDescription": meta["description"], + "hmPrevLink": meta.get("prev", ""), + "hmNextLink": meta.get("next", ""), + "hmParentLink": meta["parent"], + "hmBreadCrumbs": crumbs, + "hmTitlePath": " > ".join([titles[f] for f in trail] + [meta["title"]]), + "hmHeader": '

%s

\n\r' + % esc(meta["title"]), + "hmBody": body_html(md), + } + with open(os.path.join(out_dir, file + ".json"), "w", encoding="utf-8") as f: + json.dump(facts, f, ensure_ascii=False) + print("built", file, len(facts["hmBody"]), "bytes of body") + + +main() diff --git a/docs/docdb/ADFA-7083/pages/javascript-intro.md b/docs/docdb/ADFA-7083/pages/javascript-intro.md new file mode 100644 index 0000000000..2ba7d4194e --- /dev/null +++ b/docs/docdb/ADFA-7083/pages/javascript-intro.md @@ -0,0 +1,119 @@ +--- +file: javascript-intro.html +title: JavaScript basics +description: A short introduction to JavaScript for writing sqlite-js functions. +parent: sqlite-js-top.html +prev: sqlite-js-javascript.html +--- + +This page teaches enough JavaScript to write [sqlite-js functions](sqlite-js-top.html). Each example runs in `sqlite3` after you `.load sqlite-js`. The `js_eval` function runs JavaScript and returns the value of the last expression. + +## Values and variables + +`const` makes a variable that you never reassign. `let` makes one that you can. Text goes in quotes, and a template string, in backticks, can include values with `${...}`. + +```sql +SELECT js_eval(' + const name = "Code on the Go"; + let count = 3; + count = count + 1; + `${name} has ${count} tools`; +'); +-- Code on the Go has 4 tools +``` + +## Numbers + +JavaScript has one kind of number. `Math` has the usual functions. + +```sql +SELECT js_eval('Math.max(3, 9, 4)'), js_eval('Math.round(2.5)'), js_eval('10 % 3'), js_eval('2 ** 10'); +-- 9|3|1|1024 +``` + +## Text + +```sql +SELECT js_eval('" Hello, World ".trim().toUpperCase()'), js_eval('"a,b,c".split(",").length'), js_eval('"abcdef".slice(1, 4)'), js_eval('"hello".includes("ell")'); +-- HELLO, WORLD|3|bcd|1 +``` + +## Arrays + +An array is an ordered list. Its `map`, `filter` and `reduce` methods take a small function, written as an arrow function `x => ...`. + +```sql +SELECT js_eval('[1, 2, 3, 4].map(x => x * x).join(",")'), + js_eval('[1, 2, 3, 4].filter(x => x % 2 === 0).join(",")'), + js_eval('[1, 2, 3, 4].reduce((sum, x) => sum + x, 0)'); +-- 1,4,9,16|2,4|10 +``` + +## Objects and JSON + +An object holds named values. `JSON.stringify` turns it into text, and `JSON.parse` turns text back into an object. This is how a function can work with JSON stored in a column. + +```sql +SELECT js_eval('JSON.stringify({ name: "ada", langs: ["js", "sql"] })'), + js_eval('JSON.parse(`{"price": 12, "qty": 3}`).price * 3'); +-- {"name":"ada","langs":["js","sql"]}|36 +``` + +## Decisions and loops + +```sql +SELECT js_eval(' + let total = 0; + for (let i = 1; i <= 5; i++) { + if (i % 2 === 1) total += i; + } + total; +'); +-- 9 +``` + +## Your own functions + +A function takes values and returns one. An arrow function is a short way to write one. + +```sql +SELECT js_eval(' + function area(w, h) { return w * h; } + const half = n => n / 2; + half(area(3, 4)); +'); +-- 6 +``` + +## Dates + +```sql +SELECT js_eval('new Date(Date.UTC(2026, 10, 11)).toISOString().slice(0, 10)'); +-- 2026-11-11 +``` + +## Handling errors + +`try` and `catch` let your code recover when something throws an exception. + +```sql +SELECT js_eval(' + let result; + try { result = JSON.parse("not json"); } catch (e) { result = "bad input"; } + result; +'); +-- bad input +``` + +## Matching patterns + +A regular expression finds text that matches a pattern. + +```sql +SELECT js_eval('"order 66 shipped on 2026-11-11".match(/\d{4}-\d{2}-\d{2}/)[0]'); +-- 2026-11-11 +``` + +## Keep learning + +The [MDN JavaScript Guide](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide) teaches the whole language. In sqlite-js, ignore the parts about web pages, such as the DOM, `fetch` and timers: see [JavaScript in sqlite-js](sqlite-js-javascript.html) for what is available. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md new file mode 100644 index 0000000000..e32865e107 --- /dev/null +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -0,0 +1,201 @@ +--- +file: sqlite-js-functions.html +title: sqlite-js function reference +description: Every sqlite-js function, with its arguments, results and errors. +parent: sqlite-js-top.html +prev: sqlite-js-top.html +next: sqlite-js-javascript.html +--- + +This page lists every SQL function that sqlite-js adds. First [load sqlite-js](sqlite-js-top.html). Each `js_` function except `js_version` works only in your own SQL statements, not inside a view, trigger, CHECK constraint, generated column or index. + +An error in your JavaScript, such as a thrown exception or a syntax error, becomes an SQL error that carries the JavaScript message. + +## Create a function + +In the code you pass, a **function expression** is a JavaScript function written inside parentheses, like `(function (args) { ... })`. Every `js_create_` function returns `0` when it succeeds. + +A function cannot be redefined in the session that created it: you get `Function updates must be performed using a separate database connection.` Close `sqlite3`, open it again and create the new version. + +### js_create_scalar + +```text +js_create_scalar(name, code) +``` + +Creates a scalar function. `code` is a function expression. It receives one argument, an array with the SQL arguments, and returns the result. Both arguments must be TEXT. + +```sql +SELECT js_create_scalar('add_all', '(function (args) { + return args.reduce((sum, x) => sum + x, 0); +})'); +SELECT add_all(1, 2, 3), add_all(); +-- 6|0 +``` + +### js_create_aggregate + +```text +js_create_aggregate(name, init, step, final) +``` + +Creates an aggregate function. + +- `init` is JavaScript code that runs before the first row of each group, to set up variables. It can be NULL. +- `step` is a function expression that runs once for each row, with the array of arguments. +- `final` is a function expression that runs after the last row and returns the result. + +The variables you set up in `init` belong to one group. Each group, and each use of the function in a query, has its own. + +```sql +SELECT js_create_aggregate('longest', + 'best = "";', + '(function (args) { if (args[0].length > best.length) best = args[0]; })', + '(function () { return best; })'); +WITH w(g, s) AS (VALUES ('a', 'x'), ('a', 'xyz'), ('b', 'hello'), ('a', 'xy')) +SELECT g, longest(s) FROM w GROUP BY g ORDER BY g; +-- a|xyz +-- b|hello +``` + +### js_create_window + +```text +js_create_window(name, init, step, final, value, inverse) +``` + +Creates a window function, an aggregate that can be used with `OVER`. It takes the same `init`, `step` and `final` as an aggregate, and two more: + +- `value` is a function expression that returns the current result, while the window is still moving. +- `inverse` is a function expression that removes a row that leaves the window. It receives the array of arguments for that row. + +```sql +SELECT js_create_window('moving_sum', 'total = 0;', + '(function (args) { total += args[0]; })', + '(function () { return total; })', + '(function () { return total; })', + '(function (args) { total -= args[0]; })'); +WITH v(x) AS (VALUES (1), (2), (3), (4)) +SELECT group_concat(s, ' ') FROM ( + SELECT moving_sum(x) OVER (ORDER BY x ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS s FROM v); +-- 1 3 5 7 +``` + +### js_create_collation + +```text +js_create_collation(name, code) +``` + +Creates a collation, a way to compare two strings. `code` is a function expression that receives two strings and returns a negative number if the first sorts first, zero if they are equal, and a positive number if the second sorts first. Only the sign counts. Use the collation with `COLLATE name`. + +```sql +SELECT js_create_collation('by_length', '(function (a, b) { + return a.length - b.length || a.localeCompare(b); +})'); +WITH w(s) AS (VALUES ('ccc'), ('a'), ('bb')) +SELECT group_concat(s, ' ') FROM (SELECT s FROM w ORDER BY s COLLATE by_length); +-- a bb ccc +``` + +If a collation throws an exception or runs out of memory, the two strings compare as equal. `ORDER BY` then gives a wrong order and no error, so keep collation code simple. + +## Run code + +### js_eval + +```text +js_eval(code) +``` + +Runs JavaScript and returns the value of its last expression. Variables you create stay available to later `js_eval` calls, to your scalar functions and to your collations in the same session. + +```sql +SELECT js_eval('globalThis.rate = 0.2; 0'); +SELECT js_eval('Math.round(100 * (1 + rate))'); +-- 120 +``` + +## Keep functions in the database + +By default a function lasts only for the session. `js_init_table` makes the functions you create stay in the database. + +### js_init_table + +```text +js_init_table() +js_init_table(1) +``` + +`js_init_table()` creates the table `js_functions`, with the columns `name`, `kind`, `init_code`, `step_code`, `final_code`, `value_code` and `inverse_code`. Whenever that table exists, each function you create is stored in it, under the same name. + +`js_init_table(1)` also creates every function stored in the table, which is how a new session gets them back. + +```sql +SELECT js_init_table(); +SELECT js_create_scalar('twice', '(function (args) { return args[0] * 2; })'); +SELECT name, kind FROM js_functions; +-- twice|scalar +``` + +> Call `js_init_table(1)` only on databases you trust. It runs the JavaScript that the database stores. + +## Read files + +### js_load_text and js_load_blob + +```text +js_load_text(path) +js_load_blob(path) +``` + +Return the contents of a file, as TEXT or as a BLOB. An empty file gives an empty value. If the file cannot be opened, you get the error `Unable to open the file`. + +## Limits + +### js_set_memory_limit + +```text +js_set_memory_limit(bytes) +``` + +Sets the most memory, in bytes, that JavaScript may use in this session. The default is 64 MB. `0` means no limit. The argument must be an integer, or you get the error `The memory limit must be an integer number of bytes, or 0 for no limit`. The function returns NULL. + +### js_set_max_stack_size + +```text +js_set_max_stack_size(bytes) +``` + +Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Do not use `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. + +## Version + +### js_version + +```text +js_version() +js_version(1) +``` + +`js_version()` returns the version of sqlite-js. `js_version(1)` returns the version of the JavaScript engine, quickjs-ng. + +```sql +SELECT js_version(); +-- 1.3.7 +``` + +## What is different from the upstream project + +The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with six fixes, which were also sent to the upstream project. With them: + +- Ctrl-C stops JavaScript that never ends. +- JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. +- The functions that run code, read files or change limits cannot be used in a view, trigger or index. +- A function can return a BLOB, by returning an `ArrayBuffer` or a typed array such as `Uint8Array`. +- A whole number comes back as an INTEGER, even beyond the 32-bit range. +- An exception in an aggregate or window function fails the query. + +## Credits + +This page adapts the README of [sqlite-js](https://github.com/sqliteai/sqlite-js), which is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). It has been rewritten and changed for Code on the Go. The software is provided "as is", without warranties or conditions of any kind. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md new file mode 100644 index 0000000000..e4186e006b --- /dev/null +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -0,0 +1,123 @@ +--- +file: sqlite-js-javascript.html +title: JavaScript in sqlite-js +description: How SQL and JavaScript values convert, what the JavaScript engine supports, and its limits. +parent: sqlite-js-top.html +prev: sqlite-js-functions.html +next: javascript-intro.html +--- + +sqlite-js runs your code with quickjs-ng, a small JavaScript engine. This page explains what that means for your code. To see the engine version, enter `SELECT js_version(1);`. + +## What the engine supports + +The engine supports modern JavaScript: `const` and `let`, arrow functions, classes, template strings, destructuring, `Map` and `Set`, `BigInt`, regular expressions, `JSON`, `Math`, `Date`, typed arrays and the standard array and string methods. + +```sql +SELECT js_eval(' + class Point { constructor(x, y) { this.x = x; this.y = y; } get sum() { return this.x + this.y; } } + const { sum } = new Point(3, 4); + `${[1, 2, 3].map(n => n * n).join("+")} and ${sum}`; +'); +-- 1+4+9 and 7 +``` + +## What the engine does not have + +sqlite-js runs JavaScript inside SQLite, not in a web page or in Node.js. These do not exist: + +- Web page features: `window`, `document`, `fetch`, `alert` and `setTimeout`. +- Node.js features: `require`, `process` and `Buffer`. + +```sql +SELECT js_eval('[typeof window, typeof fetch, typeof setTimeout, typeof require, typeof process].join(" ")'); +-- undefined undefined undefined undefined undefined +``` + +Promises never finish, because nothing runs the queue that finishes them, so `async` functions and `.then` callbacks never run their next step. Write ordinary, synchronous code. + +JavaScript has no file, network or process functions of its own. + +## Print messages + +`console.log` and `print` write to the standard output of `sqlite3`, so the text appears among the query results. + +## Values from SQL to JavaScript + +| SQL | JavaScript | +|-----|------------| +| NULL | `null` | +| INTEGER, REAL | a number | +| TEXT | a string | +| BLOB | an `ArrayBuffer` | + +Your function receives its SQL arguments in an array, so the first argument is `args[0]`. + +## Values from JavaScript to SQL + +| JavaScript | SQL | +|------------|-----| +| `null`, `undefined` | NULL | +| `true`, `false` | INTEGER 1 or 0 | +| a whole number from -(2^53 - 1) to 2^53 - 1 | INTEGER | +| any other number | REAL | +| a BigInt | INTEGER | +| a string | TEXT | +| an `ArrayBuffer`, or a typed array such as `Uint8Array` | BLOB | +| any other object | NULL | + +To keep an object, return `JSON.stringify(value)`, which is a string. + +```sql +SELECT quote(js_eval('true')), quote(js_eval('new Uint8Array([1, 2, 3])')), quote(js_eval('({a: 1})')), js_eval('JSON.stringify({a: 1})'); +-- 1|X'010203'|NULL|{"a":1} +``` + +JavaScript has one number type, so it cannot tell 3.0 from 3. A REAL that holds a whole number, such as 3.0, comes back as the INTEGER 3. An INTEGER larger than 2^53 - 1 loses digits in JavaScript. To keep every digit, pass it as TEXT and convert it with `BigInt`: + +```sql +SELECT js_create_scalar('big_next', '(function (args) { return BigInt(args[0]) + 1n; })'); +SELECT big_next(CAST(9223372036854775806 AS TEXT)); +-- 9223372036854775807 +``` + +## Run SQL from JavaScript + +The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. + +| Result set | What it does | +|------------|--------------| +| `next()` | Moves to the next row. It returns `true`, or `false` when there are no more rows. | +| `get(i)` | Returns column `i` of the current row, counting from 0. | +| `name(i)` | Returns the name of column `i`. | +| `columnCount` | The number of columns. | +| `toArray()` | Returns all the remaining rows, each as an array. | + +```sql +SELECT js_eval('(function () { + const rows = db.exec("SELECT ? * 2 AS doubled, ? AS label", 21, "answer"); + rows.next(); + return rows.name(0) + "=" + rows.get(0) + " " + rows.get(1); +})()'); +-- doubled=42 answer + +SELECT js_eval('JSON.stringify(db.exec("SELECT 1 AS a, 2 AS b UNION ALL SELECT 3, 4").toArray())'); +-- [[1,2],[3,4]] +``` + +Parameters convert like function results do, so a `Uint8Array` is bound as a BLOB. + +## Variables and state + +- Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. +- Each running aggregate or window function has variables of its own, which its `init` code sets up. That is why two groups never mix their state. + +## Limits + +- **Memory.** JavaScript may use 64 MB in a session. Memory that variables still hold counts against the limit until you clear them. See `js_set_memory_limit`. +- **Endless code.** Ctrl-C stops running JavaScript, even code that catches every error. The query fails with `interrupted` and `sqlite3` keeps running if you were typing commands. +- **Deep calls.** Functions that call themselves too deeply fail with `Maximum call stack size exceeded`. See `js_set_max_stack_size`. + +## Errors + +A thrown exception stops the query, and its message becomes the SQL error. For example, `SELECT js_eval('throw new Error("boom")');` fails with an error that contains `boom`. A syntax error fails the same way, and `js_create_` functions report it when you create the function. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md new file mode 100644 index 0000000000..5109ef9703 --- /dev/null +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -0,0 +1,79 @@ +--- +file: sqlite-js-top.html +title: JavaScript in SQLite (sqlite-js) +description: Write SQLite functions in JavaScript and call them from SQL in Terminal. +parent: terminal-top.html +prev: terminal-top.html +next: sqlite-js-functions.html +--- + +sqlite-js adds SQL functions written in JavaScript to the `sqlite3` command in Terminal. You write a JavaScript function, give it a name, and then use it in a query like any built-in SQL function. + +JavaScript is quicker to write and change than Java or Kotlin, and the code can live in the database next to the data it works on. + +You can write four kinds of function: + +- A **scalar** function returns one value for each row, such as turning a title into a web address. +- An **aggregate** function turns many rows into one value, such as a median. +- A **window** function is an aggregate that can also work over a sliding range of rows, such as a moving sum. +- A **collation** decides how text sorts. + +## Install sqlite-js + +You need an Internet connection. Open [Terminal](terminal-top.html) and enter: + +```text +apt update +apt install sqlite sqlite-js +``` + +The `sqlite` package provides the `sqlite3` command that sqlite-js works with. + +## Load sqlite-js + +Open a database in `sqlite3`, then load the extension. Load it in each session that needs it. + +```text +sqlite3 my.db +.load sqlite-js +``` + +If `sqlite3` cannot find sqlite-js, give `.load` the full path. Enter `echo $PREFIX/lib/sqlite-js` in Terminal to print it. `.load` does not understand `$PREFIX` itself. + +## Your first function + +A scalar function is JavaScript code in parentheses. It gets the SQL arguments as an array named `args`. `js_create_scalar` prints `0` when it creates the function. + +```sql +SELECT js_create_scalar('slugify', '(function (args) { + return args[0].toLowerCase().replace(/[^a-z0-9]+/g, "-").replace(/^-|-$/g, ""); +})'); +SELECT slugify('Hello, World!'); +-- hello-world +``` + +You can use the function in any query in the same session: + +```sql +WITH titles(t) AS (VALUES ('Hello, World!'), ('Code on the Go')) +SELECT t, slugify(t) FROM titles; +-- Hello, World!|hello-world +-- Code on the Go|code-on-the-go +``` + +A function you create lasts until you close `sqlite3`. To keep your functions in the database, see [sqlite-js function reference](sqlite-js-functions.html). + +## Stay safe + +- **Load sqlite-js only for databases you trust.** A database can store JavaScript, and `js_init_table(1)` runs it with your access. Through `db.exec`, that code can run any SQL, including `readfile()` and `writefile()`, which read and write your files. +- **Do not add `.load sqlite-js` to `~/.sqliterc`.** That file runs for every database you open. +- **Ctrl-C stops code that never ends.** The query fails with `interrupted`. +- **JavaScript may use 64 MB of memory per session.** Code that needs more fails with `out of memory`. + +## Learn more + +- [sqlite-js function reference](sqlite-js-functions.html): every function, with arguments, results and errors. +- [JavaScript in sqlite-js](sqlite-js-javascript.html): how SQL values and JavaScript values convert, what the engine supports, and the limits. +- [JavaScript basics](javascript-intro.html): a short introduction for new JavaScript users. + +sqlite-js is developed at [github.com/sqliteai/sqlite-js](https://github.com/sqliteai/sqlite-js) under the Apache License 2.0. The version in Code on the Go includes some fixes, described in the reference. diff --git a/docs/docdb/ADFA-7083/test_examples.py b/docs/docdb/ADFA-7083/test_examples.py new file mode 100644 index 0000000000..73d39960b6 --- /dev/null +++ b/docs/docdb/ADFA-7083/test_examples.py @@ -0,0 +1,59 @@ +#!/usr/bin/env python3 +"""Runs every ```sql example in pages/*.md against a real sqlite3 with sqlite-js +loaded, and checks that internal links resolve. + +The examples of a page run in order in one session, as a reader follows them: an +example may use what the ones before it made. The `-- text` comment lines in a +block are the output it must end with. A block with no such lines only has to +succeed. + +Usage: SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py +Prints TAP. Exits 0 only if every example passes. +""" +import glob +import os +import re +import subprocess +import sys + +HERE = os.path.dirname(os.path.abspath(__file__)) +SQLITE3 = os.environ.get("SQLITE3", "sqlite3") +EXT = os.environ.get("EXT", "sqlite-js") +# Pages of the existing help that the new pages link to. +EXISTING = {"terminal-top.html", "cogo-top.html"} + + +def main(): + n = failed = 0 + files = sorted(glob.glob(os.path.join(HERE, "pages", "*.md"))) + known = {re.search(r"^file: (.*)$", open(f, encoding="utf-8").read(), re.M).group(1) for f in files} | EXISTING + for path in files: + text = open(path, encoding="utf-8").read() + name = os.path.basename(path) + earlier = [] + for k, block in enumerate(re.findall(r"^```sql\n(.*?)^```", text, re.M | re.S), 1): + n += 1 + want = [m.group(1) for m in re.finditer(r"^-- ?(.*)$", block, re.M)] + sql = "\n".join(l for l in block.split("\n") if not l.startswith("--")) + earlier.append(sql) + all_sql = "\n".join(earlier) + r = subprocess.run([SQLITE3, "-batch", "-noheader", "-cmd", '.load "%s"' % EXT, ":memory:", all_sql], + capture_output=True, text=True, timeout=60) + got = r.stdout.rstrip("\n").split("\n") if r.stdout.strip() else [] + ok = r.returncode == 0 and (not want or got[-len(want):] == want) + print("%s %d - %s example %d" % ("ok" if ok else "not ok", n, name, k)) + if not ok: + failed += 1 + print("# exit %d, want %r, got %r, stderr %r" % (r.returncode, want, got[-len(want) or 1:], r.stderr[:200])) + for url in re.findall(r"\]\(([^)]+)\)", text): + if not re.match(r"https?://", url): + n += 1 + ok = url in known + print("%s %d - %s link %s" % ("ok" if ok else "not ok", n, name, url)) + failed += not ok + print("1..%d" % n) + print("# FAILED %d of %d" % (failed, n) if failed else "# passed all %d" % n) + sys.exit(1 if failed else 0) + + +main() From ae9ba1f91443eb4e1856c5a567bb4b4a86df9053 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 11:04:41 -0800 Subject: [PATCH 02/24] ADFA-7083: Record phone check results in CLAIMS.md Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index ae51c47e3c..469bda2038 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -6,11 +6,11 @@ | Claim | Check | |-------|-------| -| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | TODO: package names and apt behavior on a phone (ADFA-7081 phone test) | -| `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | TODO phone; same claim as sqlite-brotli's README, verified there | +| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. TODO: `apt install sqlite-js` from the stage repo (sqlite-js was installed with `dpkg -i`; production has no sqlite-js) | +| `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run | | A function lasts until `sqlite3` closes | run (functions page) | -| `js_init_table(1)` runs stored code; `db.exec` can run any SQL including `readfile()`/`writefile()` | code; TODO: `readfile`/`writefile` exist in the Termux `sqlite3` shell | +| `js_init_table(1)` runs stored code; `db.exec` can run any SQL including `readfile()`/`writefile()` | run on phone: `writefile`/`readfile` exist in the shell and work; `db.exec` ran a `readfile` query without error but returned only `{"columnCount":1}`, so the claim that it can run them is by code, not by output | | Ctrl-C stops code, query fails with `interrupted` | run (package run-tests AC8) | | 64 MB memory limit; `out of memory` | run (AC9) | | Do not put `.load` in `~/.sqliterc` | advice, follows from the above | @@ -22,11 +22,11 @@ | Each `js_` function but `js_version` works only in top-level SQL | run (AC10, flags check) | | Examples for scalar, aggregate (per-group state), window, collation, `js_eval`, `js_init_table`, `js_version` | run | | Redefining in the same session fails with the quoted message | run (package AC4) | -| `init` can be NULL | code (`init_code` optional); TODO: add a run example | +| `init` can be NULL | run on phone (NULL init, aggregate gives 2). TODO: add an example to the page | | `value`/`inverse` semantics | code, run | -| Collation: only the sign counts; a throwing collation compares as equal | code (patch history, 1.3.6 changelog); TODO run | -| `js_init_table()` columns and "whenever that table exists, each function you create is stored" | code (`js_add_to_table`: returns early if the table is missing); TODO: run a case where the table exists from an earlier session and `js_init_table()` is not called | -| `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | code; TODO run | +| Collation: only the sign counts; a throwing collation compares as equal | run on phone: a throwing collation leaves the rows in their original order, i.e. equal | +| `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | +| `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | | `js_set_memory_limit`: default 64 MB, 0 no limit, integer only, error text, returns NULL | run (package AC9), code | | `js_set_max_stack_size`: default 1 MB; 0 crashes deep recursion | code (`JS_DEFAULT_STACK_SIZE`); run by hand: size 0 + recursion segfaults, default gives "Maximum call stack size exceeded" | | `js_version(1)` is quickjs-ng | run (package AC2) | @@ -37,11 +37,11 @@ | Claim | Check | |-------|-------| -| Engine supports const/let, arrow functions, classes, template strings, destructuring, Map/Set, BigInt, regexes, JSON, Math, Date, typed arrays | run: class/destructuring/template/map example; TODO: probe Map, Set, BigInt, Date, typed arrays individually | -| `window`, `fetch`, `setTimeout`, `require`, `process` undefined | run (also `document`, `alert`, `Buffer` named in prose: TODO) | +| Engine supports const/let, arrow functions, classes, template strings, destructuring, Map/Set, BigInt, regexes, JSON, Math, Date, typed arrays | run on phone: Map, Set, BigInt (2n**70n), Date, Uint8Array, Float64Array, class, destructuring, arrow, template string | +| `window`, `fetch`, `setTimeout`, `require`, `process` undefined | run on phone: `window`, `document`, `fetch`, `alert`, `setTimeout`, `require`, `process`, `Buffer` are all undefined | | Promises never finish | run (package AC11) | | "JavaScript has no file, network or process functions of its own" | run (AC11: `std`, `os` not reachable) | -| `console.log` and `print` write to `sqlite3` standard output | run by hand (`print` TODO) | +| `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables | run (package AC6), code | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | | `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` | run | From 97fb25448b7a9a0714e03df167b2dcd66df47730 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 11:12:24 -0800 Subject: [PATCH 03/24] ADFA-7083: Credit quickjs-ng (MIT) and name the license files Co-Authored-By: Claude Sonnet 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 1 + docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 2 ++ 2 files changed, 3 insertions(+) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 469bda2038..fdb65b3283 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -31,6 +31,7 @@ | `js_set_max_stack_size`: default 1 MB; 0 crashes deep recursion | code (`JS_DEFAULT_STACK_SIZE`); run by hand: size 0 + recursion segfaults, default gives "Maximum call stack size exceeded" | | `js_version(1)` is quickjs-ng | run (package AC2) | | The six fixes listed | code, package tests | +| quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | ## sqlite-js-javascript diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index e32865e107..698a5665b6 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -199,3 +199,5 @@ The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with si ## Credits This page adapts the README of [sqlite-js](https://github.com/sqliteai/sqlite-js), which is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). It has been rewritten and changed for Code on the Go. The software is provided "as is", without warranties or conditions of any kind. + +The JavaScript engine, quickjs-ng, is built into sqlite-js and is licensed under the MIT License. The license texts are installed with the package, in `$PREFIX/share/doc/sqlite-js/`: `LICENSE` for sqlite-js and `LICENSE.quickjs` for the engine. From d91cf6405c39318c6e95c6dc0627c988b002f0ef Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 11:22:10 -0800 Subject: [PATCH 04/24] ADFA-7083: Fix what the accuracy review found Probed the host build of sqlite-js 1.3.7-1 for each claim CLAIMS.md rested on code or nothing: - js_ functions are blocked only in views and triggers stored in a database; TEMP ones can use them. - A redefinition must come before js_init_table(1), and replaces the stored version. - A BigInt result outside 64 bits wraps around silently (new tested example); NaN becomes NULL. - js_set_memory_limit rejects negatives; js_set_max_stack_size reads non-numeric text as 0, which turns the check off. - An aggregate over no rows returns NULL without running init or final. - "One kind of number" now allows for BigInt; the opening no longer says JavaScript is quicker than Java or Kotlin. CLAIMS.md records each probe; db.exec running readfile/writefile is now shown by output. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 27 ++++++++++--------- .../docdb/ADFA-7083/pages/javascript-intro.md | 2 +- .../ADFA-7083/pages/sqlite-js-functions.md | 12 ++++----- .../ADFA-7083/pages/sqlite-js-javascript.md | 13 ++++++--- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 2 +- 5 files changed, 32 insertions(+), 24 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index fdb65b3283..a6a05eb64b 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,6 +1,6 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against the sqlite-js 1.3.7-1 host build. `code` = read in the patched source. `TODO` = still to verify, for the accuracy review. +`run` = executed by `test_examples.py` against the sqlite-js 1.3.7-1 host build. `code` = read in the patched source. `probe` = run by hand against the host build during the accuracy review (2026-11-11). `TODO` = still open. ## sqlite-js-top @@ -8,9 +8,9 @@ |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. TODO: `apt install sqlite-js` from the stage repo (sqlite-js was installed with `dpkg -i`; production has no sqlite-js) | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | -| `js_create_scalar` prints 0; slugify and titles examples | run | +| `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | | A function lasts until `sqlite3` closes | run (functions page) | -| `js_init_table(1)` runs stored code; `db.exec` can run any SQL including `readfile()`/`writefile()` | run on phone: `writefile`/`readfile` exist in the shell and work; `db.exec` ran a `readfile` query without error but returned only `{"columnCount":1}`, so the claim that it can run them is by code, not by output | +| `js_init_table(1)` runs stored code; `db.exec` can run any SQL including `readfile()`/`writefile()` | probe: code stored with a side effect ran during `js_init_table(1)`, before any call; `db.exec("SELECT writefile(...)")` wrote the file and `readfile` returned a BLOB. Phone: both functions exist in the Termux shell | | Ctrl-C stops code, query fails with `interrupted` | run (package run-tests AC8) | | 64 MB memory limit; `out of memory` | run (AC9) | | Do not put `.load` in `~/.sqliterc` | advice, follows from the above | @@ -19,16 +19,16 @@ | Claim | Check | |-------|-------| -| Each `js_` function but `js_version` works only in top-level SQL | run (AC10, flags check) | +| Each `js_` function but `js_version` cannot be used in a view, trigger, CHECK, generated column or index stored in a database; TEMP views and triggers can use them | run (AC10, flags check); probe: `pragma_function_list` flags, a main-schema trigger fails with `unsafe use of js_eval()`, a TEMP view and TEMP trigger work | | Examples for scalar, aggregate (per-group state), window, collation, `js_eval`, `js_init_table`, `js_version` | run | -| Redefining in the same session fails with the quoted message | run (package AC4) | -| `init` can be NULL | run on phone (NULL init, aggregate gives 2). TODO: add an example to the page | +| Redefining in a session that has the function fails with the quoted message; redefine before `js_init_table(1)`; the new version replaces the stored one | run (package AC4); probe: after `js_init_table(1)` the redefinition fails; without it, it succeeds and the next session's `js_init_table(1)` loads the new version | +| `init` can be NULL; no rows gives NULL without running `init` or `final` | run on phone (NULL init); probe and code (`js_execute_final` returns NULL without a context). No NULL-init example on the page, deliberately: the `init` example shows the usual pattern | | `value`/`inverse` semantics | code, run | | Collation: only the sign counts; a throwing collation compares as equal | run on phone: a throwing collation leaves the rows in their original order, i.e. equal | | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | -| `js_set_memory_limit`: default 64 MB, 0 no limit, integer only, error text, returns NULL | run (package AC9), code | -| `js_set_max_stack_size`: default 1 MB; 0 crashes deep recursion | code (`JS_DEFAULT_STACK_SIZE`); run by hand: size 0 + recursion segfaults, default gives "Maximum call stack size exceeded" | +| `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error | +| `js_set_max_stack_size`: default 1 MB; 0 or non-numeric text turns the check off and deep recursion crashes | code (`JS_DEFAULT_STACK_SIZE`, `sqlite3_value_int` with no check); probe: `0` and `'abc'` each segfault on endless recursion; the default gives "Maximum call stack size exceeded" | | `js_version(1)` is quickjs-ng | run (package AC2) | | The six fixes listed | code, package tests | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | @@ -38,14 +38,15 @@ | Claim | Check | |-------|-------| +| "Memory that variables still hold counts against the limit until you clear them" | probe: two 40 MB arrays held in globals fail with `out of memory`; clearing the first lets the second succeed | | Engine supports const/let, arrow functions, classes, template strings, destructuring, Map/Set, BigInt, regexes, JSON, Math, Date, typed arrays | run on phone: Map, Set, BigInt (2n**70n), Date, Uint8Array, Float64Array, class, destructuring, arrow, template string | | `window`, `fetch`, `setTimeout`, `require`, `process` undefined | run on phone: `window`, `document`, `fetch`, `alert`, `setTimeout`, `require`, `process`, `Buffer` are all undefined | -| Promises never finish | run (package AC11) | -| "JavaScript has no file, network or process functions of its own" | run (AC11: `std`, `os` not reachable) | +| Promises never finish | run (package AC11); probe: a `.then` callback had not run in the next statement | +| "JavaScript has no file, network or process functions of its own" | run (AC11: `std`, `os` not reachable); probe: the lower-case globals are `console db decodeURI decodeURIComponent encodeURI encodeURIComponent escape eval globalThis isFinite isNaN parseFloat parseInt performance print queueMicrotask scriptArgs undefined unescape` | | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | -| SQL-to-JS and JS-to-SQL conversion tables | run (package AC6), code | +| SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` | run | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows) | run; probe: `next()` is `true` then `false` at the end; `toArray()` after one `next()` returns the remaining two of three rows | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own | run (AC3) | | Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | @@ -57,7 +58,7 @@ | Claim | Check | |-------|-------| | Every snippet and its output | run | -| `const` is never reassigned, `let` can be; arrow function syntax | JavaScript language facts | +| `const` is never reassigned, `let` can be; arrow function syntax; one kind of number for whole numbers and fractions (BigInt aside, which the intro does not cover) | JavaScript language facts | ## Format diff --git a/docs/docdb/ADFA-7083/pages/javascript-intro.md b/docs/docdb/ADFA-7083/pages/javascript-intro.md index 2ba7d4194e..17a7ca7a5b 100644 --- a/docs/docdb/ADFA-7083/pages/javascript-intro.md +++ b/docs/docdb/ADFA-7083/pages/javascript-intro.md @@ -24,7 +24,7 @@ SELECT js_eval(' ## Numbers -JavaScript has one kind of number. `Math` has the usual functions. +JavaScript uses one kind of number for whole numbers and fractions alike. `Math` has the usual functions. ```sql SELECT js_eval('Math.max(3, 9, 4)'), js_eval('Math.round(2.5)'), js_eval('10 % 3'), js_eval('2 ** 10'); diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 698a5665b6..6354256607 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -7,7 +7,7 @@ prev: sqlite-js-top.html next: sqlite-js-javascript.html --- -This page lists every SQL function that sqlite-js adds. First [load sqlite-js](sqlite-js-top.html). Each `js_` function except `js_version` works only in your own SQL statements, not inside a view, trigger, CHECK constraint, generated column or index. +This page lists every SQL function that sqlite-js adds. First [load sqlite-js](sqlite-js-top.html). Each `js_` function except `js_version` works only in your own SQL statements, not inside a view, trigger, CHECK constraint, generated column or index stored in a database. A TEMP view or trigger, which lasts only for your session, can use them. An error in your JavaScript, such as a thrown exception or a syntax error, becomes an SQL error that carries the JavaScript message. @@ -15,7 +15,7 @@ An error in your JavaScript, such as a thrown exception or a syntax error, becom In the code you pass, a **function expression** is a JavaScript function written inside parentheses, like `(function (args) { ... })`. Every `js_create_` function returns `0` when it succeeds. -A function cannot be redefined in the session that created it: you get `Function updates must be performed using a separate database connection.` Close `sqlite3`, open it again and create the new version. +A function cannot be redefined in a session that already has it: you get `Function updates must be performed using a separate database connection.` Close `sqlite3`, open it again and create the new version before you call `js_init_table(1)`, which would create the old one. If the database keeps your functions, the new version replaces the stored one. ### js_create_scalar @@ -45,7 +45,7 @@ Creates an aggregate function. - `step` is a function expression that runs once for each row, with the array of arguments. - `final` is a function expression that runs after the last row and returns the result. -The variables you set up in `init` belong to one group. Each group, and each use of the function in a query, has its own. +The variables you set up in `init` belong to one group. Each group, and each use of the function in a query, has its own. If there are no rows, the function returns NULL, and neither `init` nor `final` runs. ```sql SELECT js_create_aggregate('longest', @@ -159,7 +159,7 @@ Return the contents of a file, as TEXT or as a BLOB. An empty file gives an empt js_set_memory_limit(bytes) ``` -Sets the most memory, in bytes, that JavaScript may use in this session. The default is 64 MB. `0` means no limit. The argument must be an integer, or you get the error `The memory limit must be an integer number of bytes, or 0 for no limit`. The function returns NULL. +Sets the most memory, in bytes, that JavaScript may use in this session. The default is 64 MB. `0` means no limit. The argument must be a whole number, 0 or more, or you get the error `The memory limit must be an integer number of bytes, or 0 for no limit`. The function returns NULL. ### js_set_max_stack_size @@ -167,7 +167,7 @@ Sets the most memory, in bytes, that JavaScript may use in this session. The def js_set_max_stack_size(bytes) ``` -Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Do not use `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. +Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Give a whole number of bytes, such as `1048576`. Do not use `0`, or text that is not a number, which counts as `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. ## Version @@ -191,7 +191,7 @@ The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with si - Ctrl-C stops JavaScript that never ends. - JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. -- The functions that run code, read files or change limits cannot be used in a view, trigger or index. +- The functions that run code, read files or change limits cannot be used in a view, trigger or index stored in a database. - A function can return a BLOB, by returning an `ArrayBuffer` or a typed array such as `Uint8Array`. - A whole number comes back as an INTEGER, even beyond the 32-bit range. - An exception in an aggregate or window function fails the query. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index e4186e006b..d452b2ecc6 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -60,8 +60,8 @@ Your function receives its SQL arguments in an array, so the first argument is ` | `null`, `undefined` | NULL | | `true`, `false` | INTEGER 1 or 0 | | a whole number from -(2^53 - 1) to 2^53 - 1 | INTEGER | -| any other number | REAL | -| a BigInt | INTEGER | +| any other number | REAL (`NaN` becomes NULL) | +| a BigInt | INTEGER, if it fits (see below) | | a string | TEXT | | an `ArrayBuffer`, or a typed array such as `Uint8Array` | BLOB | | any other object | NULL | @@ -73,7 +73,7 @@ SELECT quote(js_eval('true')), quote(js_eval('new Uint8Array([1, 2, 3])')), quot -- 1|X'010203'|NULL|{"a":1} ``` -JavaScript has one number type, so it cannot tell 3.0 from 3. A REAL that holds a whole number, such as 3.0, comes back as the INTEGER 3. An INTEGER larger than 2^53 - 1 loses digits in JavaScript. To keep every digit, pass it as TEXT and convert it with `BigInt`: +Apart from BigInt, JavaScript has one number type, so it cannot tell 3.0 from 3. A REAL that holds a whole number, such as 3.0, comes back as the INTEGER 3. An INTEGER larger than 2^53 - 1 loses digits in JavaScript. To keep every digit, pass it as TEXT and convert it with `BigInt`: ```sql SELECT js_create_scalar('big_next', '(function (args) { return BigInt(args[0]) + 1n; })'); @@ -81,6 +81,13 @@ SELECT big_next(CAST(9223372036854775806 AS TEXT)); -- 9223372036854775807 ``` +A BigInt result must fit in an INTEGER, from -2^63 to 2^63 - 1. A larger one wraps around without an error: + +```sql +SELECT big_next(CAST(9223372036854775807 AS TEXT)); +-- -9223372036854775808 +``` + ## Run SQL from JavaScript The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 5109ef9703..87f3fb9405 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -9,7 +9,7 @@ next: sqlite-js-functions.html sqlite-js adds SQL functions written in JavaScript to the `sqlite3` command in Terminal. You write a JavaScript function, give it a name, and then use it in a query like any built-in SQL function. -JavaScript is quicker to write and change than Java or Kotlin, and the code can live in the database next to the data it works on. +A small JavaScript function is quick to write and change, and the code can live in the database next to the data it works on. You can write four kinds of function: From fd83516eb7bab93397f99b79dc4d581ba61d7629 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 11:27:15 -0800 Subject: [PATCH 05/24] ADFA-7083: Record apt install from the stage repo Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index a6a05eb64b..23a6ce34e7 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -6,7 +6,7 @@ | Claim | Check | |-------|-------| -| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. TODO: `apt install sqlite-js` from the stage repo (sqlite-js was installed with `dpkg -i`; production has no sqlite-js) | +| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. Production gets it when the package is published there | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | | A function lasts until `sqlite3` closes | run (functions page) | From 7a3b8c78456fed475a8cbd8350a10f27c5c5cbce Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 11:59:44 -0800 Subject: [PATCH 06/24] ADFA-7083: Document db.exec's deferred run and silent errors, and 1.3.7-2 - db.exec runs its statement only at next() or toArray(); a new tested example shows that an INSERT without next() adds nothing. - A run-time error (a failed UNIQUE constraint, an integer overflow) makes next() return false and toArray() return the rows before it, with no error; SQL errors are reported (F20, filed on ADFA-7081). - js_set_max_stack_size: as in sqlite-js 1.3.7-2 (patch 0007), sizes the stack cannot hold are errors that name the largest size; 0 still turns the check off. Seven fixes, not six. These pages now describe 1.3.7-2, so they should ship with it. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 6 +++--- .../ADFA-7083/pages/sqlite-js-functions.md | 5 +++-- .../ADFA-7083/pages/sqlite-js-javascript.md | 18 +++++++++++++++--- 3 files changed, 21 insertions(+), 8 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 23a6ce34e7..43ae6ebf90 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -28,9 +28,9 @@ | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | | `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error | -| `js_set_max_stack_size`: default 1 MB; 0 or non-numeric text turns the check off and deep recursion crashes | code (`JS_DEFAULT_STACK_SIZE`, `sqlite3_value_int` with no check); probe: `0` and `'abc'` each segfault on endless recursion; the default gives "Maximum call stack size exceeded" | +| `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. TODO: phone test of 1.3.7-2 | | `js_version(1)` is quickjs-ng | run (package AC2) | -| The six fixes listed | code, package tests | +| The seven fixes listed | code, package tests (patches 0001 to 0007) | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | @@ -46,7 +46,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows) | run; probe: `next()` is `true` then `false` at the end; `toArray()` after one `next()` returns the remaining two of three rows | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` return `false` and `toArray()` return the rows before it; SQL errors are reported | run (the notes example: an INSERT without `next()` adds nothing); probe: `next()` is `true` then `false` at the end; `toArray()` after one `next()` returns the remaining two of three rows; a failed UNIQUE INSERT and `abs(-9223372036854775808)` give `next()` = `false` with no error; `toArray()` returned `[[1]]` for a 3-row query failing on row 2; `db.exec("SELECT nosuchfunc()")` throws. Code: `js_sqlite_exec` only prepares; `js_rowset_next`/`js_rowset_to_array` treat anything but `SQLITE_ROW` as the end | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own | run (AC3) | | Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 6354256607..8a497bbf4d 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -167,7 +167,7 @@ Sets the most memory, in bytes, that JavaScript may use in this session. The def js_set_max_stack_size(bytes) ``` -Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Give a whole number of bytes, such as `1048576`. Do not use `0`, or text that is not a number, which counts as `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. +Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Give a whole number of bytes, such as `1048576`, no larger than the stack of `sqlite3` allows. Anything else is an error, and the error message names the largest size allowed. Do not use `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. ## Version @@ -187,7 +187,7 @@ SELECT js_version(); ## What is different from the upstream project -The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with six fixes, which were also sent to the upstream project. With them: +The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with seven fixes, which were also sent to the upstream project. With them: - Ctrl-C stops JavaScript that never ends. - JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. @@ -195,6 +195,7 @@ The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with si - A function can return a BLOB, by returning an `ArrayBuffer` or a typed array such as `Uint8Array`. - A whole number comes back as an INTEGER, even beyond the 32-bit range. - An exception in an aggregate or window function fails the query. +- `js_set_max_stack_size` accepts only a size the stack can hold, so code that calls itself without end fails instead of crashing `sqlite3`. ## Credits diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index d452b2ecc6..e3ef489fbd 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -90,15 +90,15 @@ SELECT big_next(CAST(9223372036854775807 AS TEXT)); ## Run SQL from JavaScript -The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. +The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs only when you call `next()` or `toArray()`. | Result set | What it does | |------------|--------------| -| `next()` | Moves to the next row. It returns `true`, or `false` when there are no more rows. | +| `next()` | Runs the statement to its next row. It returns `true`, or `false` when there are no more rows, and also when the statement fails. | | `get(i)` | Returns column `i` of the current row, counting from 0. | | `name(i)` | Returns the name of column `i`. | | `columnCount` | The number of columns. | -| `toArray()` | Returns all the remaining rows, each as an array. | +| `toArray()` | Returns all the remaining rows, each as an array. If the statement fails partway, you get the rows before the failure. | ```sql SELECT js_eval('(function () { @@ -114,6 +114,18 @@ SELECT js_eval('JSON.stringify(db.exec("SELECT 1 AS a, 2 AS b UNION ALL SELECT 3 Parameters convert like function results do, so a `Uint8Array` is bound as a BLOB. +To change data, call `next()` once on the result set. Without it, nothing happens. + +```sql +CREATE TABLE notes(t TEXT); +SELECT js_eval('db.exec("INSERT INTO notes VALUES (?)", "kept"); db.exec("INSERT INTO notes VALUES (?)", "run").next(); 0'); +SELECT group_concat(t) FROM notes; +-- 0 +-- run +``` + +> An error while the statement runs, such as a failed `UNIQUE` constraint, is not reported: `next()` just returns `false`. An error in the SQL itself, such as a misspelled name, is reported. To be sure a change was made, read the data back. + ## Variables and state - Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. From 8dca7596032ff14dfa6c4f9a7fe3880e4c31e498 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 17:51:16 -0800 Subject: [PATCH 07/24] ADFA-7083: Say db.exec errors are thrown (sqlite-js 1.3.7-3), eight fixes Patch 0008 (terminal-packages PR #6, sqlite-js 1.3.7-3) makes next() and toArray() throw when the statement fails. Before it, next() returned false and toArray() returned the rows before the error. The javascript page now says so, with a tested example that catches a failed UNIQUE insert. The example fails on a host build without 0008. The functions page lists the eighth fix. CLAIMS.md: test_examples.py passes 35 of 35 on a host build of 1.3.7-4. Phone results: 1.3.7-2 passed 87 of 87, which closes the js_set_max_stack_size TODO. 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and on arm, run as a 32-bit process. README: the pages document 1.3.7-3 and later. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 10 +++++----- docs/docdb/ADFA-7083/README.md | 2 +- .../ADFA-7083/pages/sqlite-js-functions.md | 3 ++- .../ADFA-7083/pages/sqlite-js-javascript.md | 20 ++++++++++++++++--- 4 files changed, 25 insertions(+), 10 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 43ae6ebf90..def2d1bbf2 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,12 +1,12 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against the sqlite-js 1.3.7-1 host build. `code` = read in the patched source. `probe` = run by hand against the host build during the accuracy review (2026-11-11). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (35 of 35 on 1.3.7-4, patches 0001 to 0008, 2026-11-12; earlier, 1.3.7-1 and 1.3.7-2). `code` = read in the patched source. `probe` = run by hand against the host build during the accuracy review (2026-11-11). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| -| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. Production gets it when the package is published there | +| `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | | A function lasts until `sqlite3` closes | run (functions page) | @@ -28,9 +28,9 @@ | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | | `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error | -| `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. TODO: phone test of 1.3.7-2 | +| `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | | `js_version(1)` is quickjs-ng | run (package AC2) | -| The seven fixes listed | code, package tests (patches 0001 to 0007) | +| The eight fixes listed | code, package tests (patches 0001 to 0008) | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | @@ -46,7 +46,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` return `false` and `toArray()` return the rows before it; SQL errors are reported | run (the notes example: an INSERT without `next()` adds nothing); probe: `next()` is `true` then `false` at the end; `toArray()` after one `next()` returns the remaining two of three rows; a failed UNIQUE INSERT and `abs(-9223372036854775808)` give `next()` = `false` with no error; `toArray()` returned `[[1]]` for a 3-row query failing on row 2; `db.exec("SELECT nosuchfunc()")` throws. Code: `js_sqlite_exec` only prepares; `js_rowset_next`/`js_rowset_to_array` treat anything but `SQLITE_ROW` as the end | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.3.7-3, patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE` | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own | run (AC3) | | Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 69f8881570..54e3fb89b7 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -9,7 +9,7 @@ Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached fr | JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | | JavaScript basics | `i/javascript-intro.html` | -They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-1. Pages for apps follow ADFA-7082. +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 8a497bbf4d..60c364145e 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -187,7 +187,7 @@ SELECT js_version(); ## What is different from the upstream project -The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with seven fixes, which were also sent to the upstream project. With them: +The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with eight fixes, which were also sent to the upstream project. With them: - Ctrl-C stops JavaScript that never ends. - JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. @@ -196,6 +196,7 @@ The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with se - A whole number comes back as an INTEGER, even beyond the 32-bit range. - An exception in an aggregate or window function fails the query. - `js_set_max_stack_size` accepts only a size the stack can hold, so code that calls itself without end fails instead of crashing `sqlite3`. +- An error while `db.exec` runs a statement, such as a failed constraint, is thrown instead of being ignored. ## Credits diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index e3ef489fbd..0169fd2be4 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -94,11 +94,11 @@ The global object `db` runs SQL on the same database. `db.exec(sql, ...parameter | Result set | What it does | |------------|--------------| -| `next()` | Runs the statement to its next row. It returns `true`, or `false` when there are no more rows, and also when the statement fails. | +| `next()` | Runs the statement to its next row. It returns `true`, or `false` when there are no more rows. If the statement fails, it throws an error. | | `get(i)` | Returns column `i` of the current row, counting from 0. | | `name(i)` | Returns the name of column `i`. | | `columnCount` | The number of columns. | -| `toArray()` | Returns all the remaining rows, each as an array. If the statement fails partway, you get the rows before the failure. | +| `toArray()` | Returns all the remaining rows, each as an array. If the statement fails partway, it throws an error. | ```sql SELECT js_eval('(function () { @@ -124,7 +124,21 @@ SELECT group_concat(t) FROM notes; -- run ``` -> An error while the statement runs, such as a failed `UNIQUE` constraint, is not reported: `next()` just returns `false`. An error in the SQL itself, such as a misspelled name, is reported. To be sure a change was made, read the data back. +If the statement fails, for example on a `UNIQUE` constraint, `next()` or `toArray()` throws an error with SQLite's message. Unless your JavaScript catches it, the SQL that called the JavaScript fails with that message. An error in the SQL itself, such as a misspelled name, is thrown by `db.exec`. + +```sql +CREATE TABLE colors(name TEXT UNIQUE); +INSERT INTO colors VALUES ('red'); +SELECT js_eval('(function () { + try { + db.exec("INSERT INTO colors VALUES (?)", "red").next(); + return "added"; + } catch (e) { + return "not added: " + e.message; + } +})()'); +-- not added: UNIQUE constraint failed: colors.name +``` ## Variables and state From 7a24e2bcb2c2032fb075b00dc4f65333bfd29e86 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 18:20:29 -0800 Subject: [PATCH 08/24] ADFA-7083: Address the reviews of PR 2241 Independent review and CodeRabbit: - Examples with const, let or class run inside braces, so a reader can run them again in the same session without `redeclaration of`. The intro and the javascript page say why. - The redefinition steps no longer end in js_init_table(1) failing: it runs once per session and fails if a stored function already exists. - Aggregate and window functions cannot see js_eval variables (said). - Months count from 0; true shows as 1; the stack-size summary says 0 is accepted; `.shell echo $PREFIX/lib/sqlite-js` from the sqlite3 prompt; the upstream address shown once; the top page names the version the pages describe and how to check it. - apply.py refuses a database below version 2, stamps LastChange's content-i and wholedb rows instead of adding rows, and only when something changed, so a second run changes nothing. The Terminal page's link gets a heading of its own, since its other sections start collapsed. - test_examples.py also runs each example that creates nothing a second time, checks the parent, prev and next links, and prints the right output on failure. test_examples.py: 64 of 64 on a 1.3.7-4 host build. The second-run checks failed on the five examples before the braces; a misspelled next link fails. apply.py on copies of documentation.db 3.0.0: the second run leaves the file byte-identical; a database declaring 1.9.0 is refused unchanged. All pages render with Pebble 4.1.1 from the resulting database. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 16 ++++-- docs/docdb/ADFA-7083/README.md | 4 +- docs/docdb/ADFA-7083/apply.py | 54 +++++++++++++++---- .../docdb/ADFA-7083/pages/javascript-intro.md | 22 +++++--- .../ADFA-7083/pages/sqlite-js-functions.md | 6 +-- .../ADFA-7083/pages/sqlite-js-javascript.md | 7 +-- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 6 ++- docs/docdb/ADFA-7083/test_examples.py | 43 ++++++++++----- 8 files changed, 113 insertions(+), 45 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index def2d1bbf2..bb74dce05c 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,12 +1,14 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (35 of 35 on 1.3.7-4, patches 0001 to 0008, 2026-11-12; earlier, 1.3.7-1 and 1.3.7-2). `code` = read in the patched source. `probe` = run by hand against the host build during the accuracy review (2026-11-11). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.3.7-4, patches 0001 to 0008, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-1 and 1.3.7-2). `code` = read in the patched source. `probe` = run by hand against the host build, in the accuracy review (2026-11-11) or the review of PR 2241 (2026-11-12). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | +| These pages describe 1.3.7-3 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | +| `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | | A function lasts until `sqlite3` closes | run (functions page) | @@ -21,7 +23,8 @@ |-------|-------| | Each `js_` function but `js_version` cannot be used in a view, trigger, CHECK, generated column or index stored in a database; TEMP views and triggers can use them | run (AC10, flags check); probe: `pragma_function_list` flags, a main-schema trigger fails with `unsafe use of js_eval()`, a TEMP view and TEMP trigger work | | Examples for scalar, aggregate (per-group state), window, collation, `js_eval`, `js_init_table`, `js_version` | run | -| Redefining in a session that has the function fails with the quoted message; redefine before `js_init_table(1)`; the new version replaces the stored one | run (package AC4); probe: after `js_init_table(1)` the redefinition fails; without it, it succeeds and the next session's `js_init_table(1)` loads the new version | +| Redefining in a session that has the function fails with the quoted message; create the new version first in a new session, without `js_init_table(1)`; it replaces the stored one, and the next session's `js_init_table(1)` creates it | run (package AC4); probe: after `js_init_table(1)` the redefinition fails; without it, it succeeds and the next session's `js_init_table(1)` loads the new version. Probe on 1.3.7-4 (2026-11-12), file database: in a new session the redefinition succeeded and `f()` gave the new version; the next session's `js_init_table(1)` gave the new `f` and the unchanged `g` | +| `js_init_table(1)` runs once per session and fails with the same message if a stored function already exists in it | probe on 1.3.7-4: a second `js_init_table(1)`, and `js_init_table(1)` after creating any function, fail with the message. Code: `sqlite3_create_function_v2` returns `SQLITE_BUSY` when it replaces a function while the calling statement runs, and sqlite-js reports that as the message | | `init` can be NULL; no rows gives NULL without running `init` or `final` | run on phone (NULL init); probe and code (`js_execute_final` returns NULL without a context). No NULL-init example on the page, deliberately: the `init` example shows the usual pattern | | `value`/`inverse` semantics | code, run | | Collation: only the sign counts; a throwing collation compares as equal | run on phone: a throwing collation leaves the rows in their original order, i.e. equal | @@ -48,7 +51,8 @@ | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | | `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.3.7-3, patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE` | | Parameters convert like results | run (package AC6) | -| Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own | run (AC3) | +| Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | +| Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | | Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | | Stack overflow message `Maximum call stack size exceeded` | run by hand | | `js_create_` reports syntax errors when the function is created | run (package AC5) | @@ -57,12 +61,14 @@ | Claim | Check | |-------|-------| -| Every snippet and its output | run | +| Every snippet and its output, also when run a second time | run | +| Months count from 0; `true` shows as 1 | run (the Dates and Text examples) | | `const` is never reassigned, `let` can be; arrow function syntax; one kind of number for whole numbers and fractions (BigInt aside, which the intro does not cover) | JavaScript language facts | ## Format | Claim | Check | |-------|-------| -| Pages render through `helpmanual-topic.peb` | rendered with Jinja2 as a Pebble stand-in: title, body, breadcrumbs present. TODO: render in the app | +| Pages render through `helpmanual-topic.peb` | rendered with Pebble 4.1.1, the app's engine before it was vendored, from a test database built by `apply.py`: the four pages, `terminal-top` and `cogo-top` render, with titles, unescaped bodies and working links. TODO: render in the app | +| `apply.py`: a second run changes nothing; it refuses a database below version 2; the Terminal page's link has a heading of its own | run on copies of documentation.db 3.0.0: the second run printed `unchanged` for each page and left the file byte-identical; one declaring 1.9.0 was refused with no row changed; `LastChange`'s `content-i` and `wholedb` rows were stamped, not added | | Phone layout of tables and code blocks | body-only render in headless Chrome (code blocks wrap, tables fit); TODO: on a device at font scale 1.0 and 2.0 | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 54e3fb89b7..980df1ebf4 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -13,8 +13,8 @@ They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.n - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. -- `apply.py DB`: adds the pages to a `documentation.db` in one transaction, compressed against its `CompressionDictionary`, and appends a link to `i/terminal-top.html`. Run it on a copy first; the master is edited by a `docdb-studio` maintainer. Needs the `brotli` command. -- `test_examples.py`: runs every ` ```sql ` example against a real `sqlite3` with sqlite-js loaded, and checks that links resolve: `SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py`. +- `apply.py DB`: adds the pages to a `documentation.db` in one transaction, compressed against its `CompressionDictionary`, and appends a section with a link to `i/terminal-top.html`. A second run changes nothing. It refuses a database that declares a version below 2. Run it on a copy first; the master is edited by a `docdb-studio` maintainer. Needs the `brotli` command. +- `test_examples.py`: runs every ` ```sql ` example against a real `sqlite3` with sqlite-js loaded, runs again each one that creates nothing, as a reader would, and checks that links, including each page's parent, prev and next, resolve: `SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py`. - `CLAIMS.md`: every factual claim in the pages, and how each was checked. ## Not done here diff --git a/docs/docdb/ADFA-7083/apply.py b/docs/docdb/ADFA-7083/apply.py index f6b5dece9b..5230c428ae 100644 --- a/docs/docdb/ADFA-7083/apply.py +++ b/docs/docdb/ADFA-7083/apply.py @@ -6,11 +6,17 @@ Builds the pages (build.py), compresses each with Brotli against the database's CompressionDictionary (quality 11, lgwin 24, as the documentation pipeline does), and in ONE transaction: - - upserts a Content row per page, path i/, templateId of - helpmanual-topic.peb, text/html, EN-us; - - appends a link to the new pages to i/terminal-top.html, unless it has one; - - records the change in LastChange (content-i and wholedb). -Any failure rolls everything back. Safe to run again: it replaces its rows. + - upserts a Content row per page whose row is missing or differs, path + i/, templateId of helpmanual-topic.peb, text/html, EN-us; + - appends a section that links to the new pages to i/terminal-top.html, + unless it has one. The page's other sections start collapsed, so the link + gets a heading of its own, where readers see it; + - if a page or the link changed, stamps the content-i and wholedb rows of + LastChange, inserting any that is missing, as docdb-studio does. +Any failure rolls everything back. A second run changes nothing. + +It refuses a database that does not declare version 2 or later: the app +decodes rows with the dictionary only from version 2. Needs the `brotli` command-line tool. Run it on a copy first: this edits the file in place, and the master lives in OfflineDocumentationTools' workflow. @@ -26,7 +32,9 @@ HERE = os.path.dirname(os.path.abspath(__file__)) LINK_FILE = "terminal-top.html" LINK_TARGET = "sqlite-js-top.html" -LINK_HTML = ('

To write SQL functions in JavaScript, see ' +LINK_HTML = ('

' + 'SQL functions in JavaScript

\r\n' + '

To write SQL functions in JavaScript, see ' 'JavaScript in SQLite (sqlite-js).

\r\n') WHO = "ADFA-7083" @@ -64,23 +72,41 @@ def main(): if not (template and html_type and lang and dictionary): sys.exit("this does not look like a documentation.db: a template, content type, language or " "dictionary is missing") + # The app's own query and order (DatabaseVersionResolver): the newest row's version counts. + try: + version = db.execute("SELECT major FROM DocumentationDatabaseVersion " + "ORDER BY changeTime DESC, rowid DESC LIMIT 1").fetchone() + except sqlite3.OperationalError: + version = None + if not version or version[0] < 2: + sys.exit("this database does not declare version 2 or later, so the app would not decode " + "pages compressed against its dictionary") dict_path = os.path.join(work, "dict") with open(dict_path, "wb") as f: f.write(dictionary[0]) db.execute("BEGIN IMMEDIATE") try: + changed = False for name in sorted(os.listdir(out_dir)): with open(os.path.join(out_dir, name), "rb") as f: data = f.read() + path = "i/" + name[:-len(".json")] blob = brotli(["-q", "11", "-w", "24", "-D", dict_path], data, work) + stored = db.execute("SELECT languageID, content, contentTypeID, templateId FROM Content " + "WHERE path = ?", (path,)).fetchone() + if stored and (stored[0], bytes(stored[1]), stored[2], stored[3]) == \ + (lang[0], blob, html_type[0], template[0]): + print("%s unchanged" % path) + continue db.execute( "INSERT INTO Content (path, languageID, content, contentTypeID, templateId) " "VALUES (?, ?, ?, ?, ?) ON CONFLICT(path) DO UPDATE SET " "languageID = excluded.languageID, content = excluded.content, " "contentTypeID = excluded.contentTypeID, templateId = excluded.templateId", - ("i/" + name[:-len(".json")], lang[0], blob, html_type[0], template[0])) - print("upserted i/%s (%d bytes)" % (name[:-len(".json")], len(blob))) + (path, lang[0], blob, html_type[0], template[0])) + changed = True + print("upserted %s (%d bytes)" % (path, len(blob))) row = db.execute("SELECT content FROM Content WHERE path = ?", ("i/" + LINK_FILE,)).fetchone() if row is None: @@ -93,10 +119,18 @@ def main(): blob = brotli(["-q", "11", "-w", "24", "-D", dict_path], json.dumps(page, ensure_ascii=False).encode("utf-8"), work) db.execute("UPDATE Content SET content = ? WHERE path = ?", (blob, "i/" + LINK_FILE)) + changed = True print("added a link to i/%s" % LINK_FILE) - for doc_set in ("content-i", "wholedb"): - db.execute("INSERT INTO LastChange (documentationSet, who) VALUES (?, ?)", (doc_set, WHO)) + # Stamp the sets' rows rather than add rows: the app reads wholedb with LIMIT 1 and no order. + if changed: + for doc_set in ("content-i", "wholedb"): + if db.execute("UPDATE LastChange SET changeTime = CURRENT_TIMESTAMP, who = ? " + "WHERE documentationSet = ?", (WHO, doc_set)).rowcount == 0: + db.execute("INSERT INTO LastChange (documentationSet, who) VALUES (?, ?)", (doc_set, WHO)) + print("stamped LastChange: content-i, wholedb") + else: + print("nothing changed") db.execute("COMMIT") except BaseException: db.execute("ROLLBACK") diff --git a/docs/docdb/ADFA-7083/pages/javascript-intro.md b/docs/docdb/ADFA-7083/pages/javascript-intro.md index 17a7ca7a5b..89b3f0cdc0 100644 --- a/docs/docdb/ADFA-7083/pages/javascript-intro.md +++ b/docs/docdb/ADFA-7083/pages/javascript-intro.md @@ -8,17 +8,19 @@ prev: sqlite-js-javascript.html This page teaches enough JavaScript to write [sqlite-js functions](sqlite-js-top.html). Each example runs in `sqlite3` after you `.load sqlite-js`. The `js_eval` function runs JavaScript and returns the value of the last expression. +A name you declare with `const`, `let` or `class` lasts until you close `sqlite3`, and declaring it again fails with `redeclaration of`. That is why the longer examples put their code in braces, `{ ... }`: the names then last for one run, so you can run the example again. + ## Values and variables `const` makes a variable that you never reassign. `let` makes one that you can. Text goes in quotes, and a template string, in backticks, can include values with `${...}`. ```sql -SELECT js_eval(' +SELECT js_eval('{ const name = "Code on the Go"; let count = 3; count = count + 1; `${name} has ${count} tools`; -'); +}'); -- Code on the Go has 4 tools ``` @@ -33,6 +35,8 @@ SELECT js_eval('Math.max(3, 9, 4)'), js_eval('Math.round(2.5)'), js_eval('10 % 3 ## Text +A test such as `includes` gives `true` or `false`, which SQL shows as 1 or 0. + ```sql SELECT js_eval('" Hello, World ".trim().toUpperCase()'), js_eval('"a,b,c".split(",").length'), js_eval('"abcdef".slice(1, 4)'), js_eval('"hello".includes("ell")'); -- HELLO, WORLD|3|bcd|1 @@ -62,13 +66,13 @@ SELECT js_eval('JSON.stringify({ name: "ada", langs: ["js", "sql"] })'), ## Decisions and loops ```sql -SELECT js_eval(' +SELECT js_eval('{ let total = 0; for (let i = 1; i <= 5; i++) { if (i % 2 === 1) total += i; } total; -'); +}'); -- 9 ``` @@ -77,16 +81,18 @@ SELECT js_eval(' A function takes values and returns one. An arrow function is a short way to write one. ```sql -SELECT js_eval(' +SELECT js_eval('{ function area(w, h) { return w * h; } const half = n => n / 2; half(area(3, 4)); -'); +}'); -- 6 ``` ## Dates +Months count from 0, so 10 is November. + ```sql SELECT js_eval('new Date(Date.UTC(2026, 10, 11)).toISOString().slice(0, 10)'); -- 2026-11-11 @@ -97,11 +103,11 @@ SELECT js_eval('new Date(Date.UTC(2026, 10, 11)).toISOString().slice(0, 10)'); `try` and `catch` let your code recover when something throws an exception. ```sql -SELECT js_eval(' +SELECT js_eval('{ let result; try { result = JSON.parse("not json"); } catch (e) { result = "bad input"; } result; -'); +}'); -- bad input ``` diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 60c364145e..8b4b805775 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -15,7 +15,7 @@ An error in your JavaScript, such as a thrown exception or a syntax error, becom In the code you pass, a **function expression** is a JavaScript function written inside parentheses, like `(function (args) { ... })`. Every `js_create_` function returns `0` when it succeeds. -A function cannot be redefined in a session that already has it: you get `Function updates must be performed using a separate database connection.` Close `sqlite3`, open it again and create the new version before you call `js_init_table(1)`, which would create the old one. If the database keeps your functions, the new version replaces the stored one. +A function cannot be redefined in a session that already has it: you get `Function updates must be performed using a separate database connection.` To change one, close `sqlite3`, open it again and create the new version first, without calling `js_init_table(1)`, which would create the old one. If the database keeps your functions, the new version replaces the stored one, and `js_init_table(1)` creates it in your next session. ### js_create_scalar @@ -129,7 +129,7 @@ js_init_table(1) `js_init_table()` creates the table `js_functions`, with the columns `name`, `kind`, `init_code`, `step_code`, `final_code`, `value_code` and `inverse_code`. Whenever that table exists, each function you create is stored in it, under the same name. -`js_init_table(1)` also creates every function stored in the table, which is how a new session gets them back. +`js_init_table(1)` also creates every function stored in the table, which is how a new session gets them back. Call it once, at the start of the session: if a stored function already exists in the session, because you created it or already called `js_init_table(1)`, it fails with `Function updates must be performed using a separate database connection.` ```sql SELECT js_init_table(); @@ -195,7 +195,7 @@ The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with ei - A function can return a BLOB, by returning an `ArrayBuffer` or a typed array such as `Uint8Array`. - A whole number comes back as an INTEGER, even beyond the 32-bit range. - An exception in an aggregate or window function fails the query. -- `js_set_max_stack_size` accepts only a size the stack can hold, so code that calls itself without end fails instead of crashing `sqlite3`. +- `js_set_max_stack_size` accepts only a size the stack can hold, or 0, which turns the check off. With a size set, code that calls itself without end fails instead of crashing `sqlite3`. - An error while `db.exec` runs a statement, such as a failed constraint, is thrown instead of being ignored. ## Credits diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index 0169fd2be4..850b38af15 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -14,11 +14,11 @@ sqlite-js runs your code with quickjs-ng, a small JavaScript engine. This page e The engine supports modern JavaScript: `const` and `let`, arrow functions, classes, template strings, destructuring, `Map` and `Set`, `BigInt`, regular expressions, `JSON`, `Math`, `Date`, typed arrays and the standard array and string methods. ```sql -SELECT js_eval(' +SELECT js_eval('{ class Point { constructor(x, y) { this.x = x; this.y = y; } get sum() { return this.x + this.y; } } const { sum } = new Point(3, 4); `${[1, 2, 3].map(n => n * n).join("+")} and ${sum}`; -'); +}'); -- 1+4+9 and 7 ``` @@ -142,7 +142,8 @@ SELECT js_eval('(function () { ## Variables and state -- Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. +- Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. Aggregate and window functions cannot see them: define what they need in their `init` code. +- A name declared with `const`, `let` or `class` at the top level of `js_eval` code also lasts for the session, so running the same code again fails with `redeclaration of`. Put the code in braces, `{ ... }`, to keep its names to one run. `js_eval` still returns the value of the last expression. - Each running aggregate or window function has variables of its own, which its `init` code sets up. That is why two groups never mix their state. ## Limits diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 87f3fb9405..f83c526410 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -29,6 +29,8 @@ apt install sqlite sqlite-js The `sqlite` package provides the `sqlite3` command that sqlite-js works with. +These pages describe sqlite-js 1.3.7-3 and later. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. + ## Load sqlite-js Open a database in `sqlite3`, then load the extension. Load it in each session that needs it. @@ -38,7 +40,7 @@ sqlite3 my.db .load sqlite-js ``` -If `sqlite3` cannot find sqlite-js, give `.load` the full path. Enter `echo $PREFIX/lib/sqlite-js` in Terminal to print it. `.load` does not understand `$PREFIX` itself. +If `sqlite3` cannot find sqlite-js, give `.load` the full path. To print it, enter `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3`, or `echo $PREFIX/lib/sqlite-js` in Terminal. `.load` does not understand `$PREFIX` itself. ## Your first function @@ -76,4 +78,4 @@ A function you create lasts until you close `sqlite3`. To keep your functions in - [JavaScript in sqlite-js](sqlite-js-javascript.html): how SQL values and JavaScript values convert, what the engine supports, and the limits. - [JavaScript basics](javascript-intro.html): a short introduction for new JavaScript users. -sqlite-js is developed at [github.com/sqliteai/sqlite-js](https://github.com/sqliteai/sqlite-js) under the Apache License 2.0. The version in Code on the Go includes some fixes, described in the reference. +sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. The version in Code on the Go includes some fixes, described in the reference. diff --git a/docs/docdb/ADFA-7083/test_examples.py b/docs/docdb/ADFA-7083/test_examples.py index 73d39960b6..6ddd61fd9a 100644 --- a/docs/docdb/ADFA-7083/test_examples.py +++ b/docs/docdb/ADFA-7083/test_examples.py @@ -1,11 +1,14 @@ #!/usr/bin/env python3 """Runs every ```sql example in pages/*.md against a real sqlite3 with sqlite-js -loaded, and checks that internal links resolve. +loaded, and checks that internal links, including each page's parent, prev and +next, resolve. The examples of a page run in order in one session, as a reader follows them: an example may use what the ones before it made. The `-- text` comment lines in a block are the output it must end with. A block with no such lines only has to -succeed. +succeed. An example that creates nothing, no table, row or function, also runs a +second time in the same session, as a reader trying it again would, and must +give the same output. Usage: SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py Prints TAP. Exits 0 only if every example passes. @@ -21,6 +24,14 @@ EXT = os.environ.get("EXT", "sqlite-js") # Pages of the existing help that the new pages link to. EXISTING = {"terminal-top.html", "cogo-top.html"} +# An example that makes something cannot simply run again: a second CREATE TABLE fails. +MAKES_SOMETHING = re.compile(r"\b(CREATE|INSERT|js_create_\w+|js_init_table)\b", re.I) + + +def run(sql): + r = subprocess.run([SQLITE3, "-batch", "-noheader", "-cmd", '.load "%s"' % EXT, ":memory:", sql], + capture_output=True, text=True, timeout=60) + return r, (r.stdout.rstrip("\n").split("\n") if r.stdout.strip() else []) def main(): @@ -32,19 +43,27 @@ def main(): name = os.path.basename(path) earlier = [] for k, block in enumerate(re.findall(r"^```sql\n(.*?)^```", text, re.M | re.S), 1): - n += 1 want = [m.group(1) for m in re.finditer(r"^-- ?(.*)$", block, re.M)] sql = "\n".join(l for l in block.split("\n") if not l.startswith("--")) earlier.append(sql) - all_sql = "\n".join(earlier) - r = subprocess.run([SQLITE3, "-batch", "-noheader", "-cmd", '.load "%s"' % EXT, ":memory:", all_sql], - capture_output=True, text=True, timeout=60) - got = r.stdout.rstrip("\n").split("\n") if r.stdout.strip() else [] - ok = r.returncode == 0 and (not want or got[-len(want):] == want) - print("%s %d - %s example %d" % ("ok" if ok else "not ok", n, name, k)) - if not ok: - failed += 1 - print("# exit %d, want %r, got %r, stderr %r" % (r.returncode, want, got[-len(want) or 1:], r.stderr[:200])) + runs = [("", "\n".join(earlier))] + if not MAKES_SOMETHING.search(sql): + runs.append((" run again", "\n".join(earlier + [sql]))) + for label, all_sql in runs: + n += 1 + r, got = run(all_sql) + ok = r.returncode == 0 and (not want or got[-len(want):] == want) + print("%s %d - %s example %d%s" % ("ok" if ok else "not ok", n, name, k, label)) + if not ok: + failed += 1 + print("# exit %d, want %r, got %r, stderr %r" + % (r.returncode, want, got[-(len(want) or 1):], r.stderr[:200])) + front = re.match(r"---\n(.*?)\n---\n", text, re.S).group(1) + for key, url in re.findall(r"^(parent|prev|next): (.*)$", front, re.M): + n += 1 + ok = url in known + print("%s %d - %s %s %s" % ("ok" if ok else "not ok", n, name, key, url)) + failed += not ok for url in re.findall(r"\]\(([^)]+)\)", text): if not re.match(r"https?://", url): n += 1 From 6a2ab84bfa7870c964d2151517aa7d7ddeb405c0 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Wed, 11 Nov 2026 18:25:31 -0800 Subject: [PATCH 09/24] ADFA-7083: Compare only the output of the run under test CodeRabbit on PR 2241: the second-run check compared the end of the combined output, so a repeat that printed nothing passed on the first run's output. test_examples.py now prints a boundary just before the run under test, in both runs, and compares only what follows it. Checked: CodeRabbit's example (a query that prints 42 once and nothing when repeated) passes the old check and fails the new one; the pages pass 64 of 64 on a 1.3.7-4 host build. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/test_examples.py | 14 +++++++++----- 1 file changed, 9 insertions(+), 5 deletions(-) diff --git a/docs/docdb/ADFA-7083/test_examples.py b/docs/docdb/ADFA-7083/test_examples.py index 6ddd61fd9a..8ade976f60 100644 --- a/docs/docdb/ADFA-7083/test_examples.py +++ b/docs/docdb/ADFA-7083/test_examples.py @@ -26,6 +26,8 @@ EXISTING = {"terminal-top.html", "cogo-top.html"} # An example that makes something cannot simply run again: a second CREATE TABLE fails. MAKES_SOMETHING = re.compile(r"\b(CREATE|INSERT|js_create_\w+|js_init_table)\b", re.I) +# Printed just before the run under test, so that only its own output is compared. +BOUNDARY = "test_examples: run under test" def run(sql): @@ -45,14 +47,16 @@ def main(): for k, block in enumerate(re.findall(r"^```sql\n(.*?)^```", text, re.M | re.S), 1): want = [m.group(1) for m in re.finditer(r"^-- ?(.*)$", block, re.M)] sql = "\n".join(l for l in block.split("\n") if not l.startswith("--")) - earlier.append(sql) - runs = [("", "\n".join(earlier))] + mark = "SELECT '%s';" % BOUNDARY + runs = [("", "\n".join(earlier + [mark, sql]))] if not MAKES_SOMETHING.search(sql): - runs.append((" run again", "\n".join(earlier + [sql]))) + runs.append((" run again", "\n".join(earlier + [sql, mark, sql]))) + earlier.append(sql) for label, all_sql in runs: n += 1 - r, got = run(all_sql) - ok = r.returncode == 0 and (not want or got[-len(want):] == want) + r, out = run(all_sql) + got = out[len(out) - out[::-1].index(BOUNDARY):] if BOUNDARY in out else [] + ok = r.returncode == 0 and BOUNDARY in out and (not want or got[-len(want):] == want) print("%s %d - %s example %d%s" % ("ok" if ok else "not ok", n, name, k, label)) if not ok: failed += 1 From 6694952e39750c5b0f0dda5b0ca4d4fb14827e5b Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 00:14:53 -0800 Subject: [PATCH 10/24] ADFA-7083: Describe sqlite-js 1.3.7-p20261112, built from upstream main The terminal package is now built from upstream's main branch of 2026-11-12 (c8ae85a), which merged our fixes, instead of 1.3.7 with patches. The pages said it was "version 1.3.7 of the upstream project with eight fixes, which were also sent to the upstream project". - Functions page: the last section compares with 1.3.7 and says the code is upstream's of 12 November 2026, with fixes from App Dev For All; js_version notes that the newer package still reports 1.3.7; and js_set_memory_limit says a limit below memory that variables still hold leaves no room until raised. - Overview page: 1.3.7-p20261112 is named as a later version, and the credits line no longer says "some fixes". - CLAIMS.md: each changed claim and its check. README.md: the version. Checked on host builds of 1.3.7-p20261112 and 1.3.7-5: test_examples.py passes 64 of 64 on both; the lowered-limit sentence holds on both; js_version() gives 1.3.7; dpkg sorts 1.3.7-p20261112 after 1.3.7-3. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 10 ++++++---- docs/docdb/ADFA-7083/README.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 8 +++++--- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 4 ++-- 4 files changed, 14 insertions(+), 10 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index bb74dce05c..3037e19c79 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,13 +1,13 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.3.7-4, patches 0001 to 0008, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-1 and 1.3.7-2). `code` = read in the patched source. `probe` = run by hand against the host build, in the accuracy review (2026-11-11) or the review of PR 2241 (2026-11-12). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.3.7-p20261112, upstream main at c8ae85a with no patches, and on 1.3.7-5, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-1, 1.3.7-2 and 1.3.7-4, patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, upstream main from 1.3.7-p20261112. `probe` = run by hand against the host build, in the accuracy review (2026-11-11) or the review of PR 2241 (2026-11-12). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.3.7-3 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | +| These pages describe 1.3.7-3 and later, such as 1.3.7-p20261112; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; `dpkg --compare-versions 1.3.7-p20261112 gt 1.3.7-3` is true; the examples pass on 1.3.7-5 and 1.3.7-p20261112; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | @@ -16,6 +16,7 @@ | Ctrl-C stops code, query fails with `interrupted` | run (package run-tests AC8) | | 64 MB memory limit; `out of memory` | run (AC9) | | Do not put `.load` in `~/.sqliterc` | advice, follows from the above | +| The version in Code on the Go is newer than the 1.3.7 release and includes fixes from App Dev For All | code: 1.3.7-p20261112's `build.sh` (terminal-packages 428a196) builds upstream main at c8ae85a, committed 2026-11-12; the 1.3.7 tag is its ancestor, and it merges App Dev For All's PRs #11 to #19 (git log 1.3.7..c8ae85a) | ## sqlite-js-functions @@ -30,10 +31,11 @@ | Collation: only the sign counts; a throwing collation compares as equal | run on phone: a throwing collation leaves the rows in their original order, i.e. equal | | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | -| `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error | +| `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL; a limit below the memory variables still hold leaves no room for any JavaScript until raised | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error; probe on 1.3.7-5 and 1.3.7-p20261112: with 40 MB held in a global and the limit at 4 MB, `js_eval('1 + 1')` and `js_eval('2 + 2')` fail with `out of memory`, and after raising it to 64 MB `js_eval('3 + 3')` gives 6 | | `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | | `js_version(1)` is quickjs-ng | run (package AC2) | -| The eight fixes listed | code, package tests (patches 0001 to 0008) | +| `js_version()` gives `1.3.7`; a package built from newer upstream code, such as 1.3.7-p20261112, still reports `1.3.7`; `dpkg -s sqlite-js` shows the package version | run; probe on the 1.3.7-p20261112 host build: `SELECT js_version()` gives 1.3.7, and `SQLITE_JS_VERSION` is "1.3.7" at c8ae85a | +| Built from upstream code of 12 November 2026, newer than 1.3.7, which includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: upstream main c8ae85a (2026-11-12) merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 980df1ebf4..7fea1df1a9 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -9,7 +9,7 @@ Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached fr | JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | | JavaScript basics | `i/javascript-intro.html` | -They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later, including 1.3.7-p20261112, which is upstream's main branch after 1.3.7 with our fixes merged. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 8b4b805775..a4b01ec9a9 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -159,7 +159,7 @@ Return the contents of a file, as TEXT or as a BLOB. An empty file gives an empt js_set_memory_limit(bytes) ``` -Sets the most memory, in bytes, that JavaScript may use in this session. The default is 64 MB. `0` means no limit. The argument must be a whole number, 0 or more, or you get the error `The memory limit must be an integer number of bytes, or 0 for no limit`. The function returns NULL. +Sets the most memory, in bytes, that JavaScript may use in this session. The default is 64 MB. `0` means no limit. The argument must be a whole number, 0 or more, or you get the error `The memory limit must be an integer number of bytes, or 0 for no limit`. The function returns NULL. A limit below the memory that variables still hold leaves no room for any JavaScript until you raise it again. ### js_set_max_stack_size @@ -185,9 +185,11 @@ SELECT js_version(); -- 1.3.7 ``` -## What is different from the upstream project +A package built from upstream code newer than 1.3.7, such as 1.3.7-p20261112, still reports `1.3.7`. To see the package version, enter `dpkg -s sqlite-js` in Terminal. -The sqlite-js in Code on the Go is version 1.3.7 of the upstream project with eight fixes, which were also sent to the upstream project. With them: +## What is different from sqlite-js 1.3.7 + +The sqlite-js in Code on the Go is built from the upstream project's code of 12 November 2026, which is newer than its 1.3.7 release. That code includes fixes that App Dev For All sent to the upstream project. Compared with 1.3.7: - Ctrl-C stops JavaScript that never ends. - JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index f83c526410..4332b89978 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -29,7 +29,7 @@ apt install sqlite sqlite-js The `sqlite` package provides the `sqlite3` command that sqlite-js works with. -These pages describe sqlite-js 1.3.7-3 and later. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. +These pages describe sqlite-js 1.3.7-3 and later versions, such as 1.3.7-p20261112. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. ## Load sqlite-js @@ -78,4 +78,4 @@ A function you create lasts until you close `sqlite3`. To keep your functions in - [JavaScript in sqlite-js](sqlite-js-javascript.html): how SQL values and JavaScript values convert, what the engine supports, and the limits. - [JavaScript basics](javascript-intro.html): a short introduction for new JavaScript users. -sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. The version in Code on the Go includes some fixes, described in the reference. +sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. The version in Code on the Go is newer than the 1.3.7 release and includes fixes from App Dev For All, described in the reference. From b159b3dcbba7eee4ae37b47fc4e2058d7533c48e Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 00:45:57 -0800 Subject: [PATCH 11/24] ADFA-7083: Give the version as dpkg shows it, 1.3.7-p20261112-0 termux-packages appends -0 to a version that contains a hyphen, so the published package, and `dpkg -s sqlite-js`, show 1.3.7-p20261112-0. Stage's Packages index lists that for aarch64 and arm (CI run 43183024875). The overview page, the js_version note, README.md and the two CLAIMS.md rows now give that version, with that evidence. test_examples.py: 64 of 64 on the host build. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 4 ++-- docs/docdb/ADFA-7083/README.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 2 +- 4 files changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 3037e19c79..5802af7afe 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -7,7 +7,7 @@ | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.3.7-3 and later, such as 1.3.7-p20261112; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; `dpkg --compare-versions 1.3.7-p20261112 gt 1.3.7-3` is true; the examples pass on 1.3.7-5 and 1.3.7-p20261112; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | +| These pages describe 1.3.7-3 and later, such as 1.3.7-p20261112-0; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; stage's Packages index lists sqlite-js at `Version: 1.3.7-p20261112-0` for aarch64 and arm, published by CI run 43183024875 (termux-packages appends `-0` to a version that contains a hyphen, `termux_step_start_build.sh`); `dpkg --compare-versions 1.3.7-p20261112-0 gt 1.3.7-3` is true; the examples pass on 1.3.7-5 and 1.3.7-p20261112; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | @@ -34,7 +34,7 @@ | `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL; a limit below the memory variables still hold leaves no room for any JavaScript until raised | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error; probe on 1.3.7-5 and 1.3.7-p20261112: with 40 MB held in a global and the limit at 4 MB, `js_eval('1 + 1')` and `js_eval('2 + 2')` fail with `out of memory`, and after raising it to 64 MB `js_eval('3 + 3')` gives 6 | | `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | | `js_version(1)` is quickjs-ng | run (package AC2) | -| `js_version()` gives `1.3.7`; a package built from newer upstream code, such as 1.3.7-p20261112, still reports `1.3.7`; `dpkg -s sqlite-js` shows the package version | run; probe on the 1.3.7-p20261112 host build: `SELECT js_version()` gives 1.3.7, and `SQLITE_JS_VERSION` is "1.3.7" at c8ae85a | +| `js_version()` gives `1.3.7`; a package built from newer upstream code, such as 1.3.7-p20261112-0, still reports `1.3.7`; `dpkg -s sqlite-js` shows the package version | run; probe on the 1.3.7-p20261112 host build: `SELECT js_version()` gives 1.3.7, and `SQLITE_JS_VERSION` is "1.3.7" at c8ae85a; the published package's version is `1.3.7-p20261112-0` (stage's Packages index, CI run 43183024875) | | Built from upstream code of 12 November 2026, newer than 1.3.7, which includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: upstream main c8ae85a (2026-11-12) merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 7fea1df1a9..9c60caa867 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -9,7 +9,7 @@ Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached fr | JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | | JavaScript basics | `i/javascript-intro.html` | -They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later, including 1.3.7-p20261112, which is upstream's main branch after 1.3.7 with our fixes merged. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later, including 1.3.7-p20261112-0, which is upstream's main branch after 1.3.7 with our fixes merged. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index a4b01ec9a9..5ff04291f2 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -185,7 +185,7 @@ SELECT js_version(); -- 1.3.7 ``` -A package built from upstream code newer than 1.3.7, such as 1.3.7-p20261112, still reports `1.3.7`. To see the package version, enter `dpkg -s sqlite-js` in Terminal. +A package built from upstream code newer than 1.3.7, such as 1.3.7-p20261112-0, still reports `1.3.7`. To see the package version, enter `dpkg -s sqlite-js` in Terminal. ## What is different from sqlite-js 1.3.7 diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 4332b89978..0a373c162d 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -29,7 +29,7 @@ apt install sqlite sqlite-js The `sqlite` package provides the `sqlite3` command that sqlite-js works with. -These pages describe sqlite-js 1.3.7-3 and later versions, such as 1.3.7-p20261112. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. +These pages describe sqlite-js 1.3.7-3 and later versions, such as 1.3.7-p20261112-0. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. ## Load sqlite-js From f6999f99d787f0570ba9a7034d631e4646847acd Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 15:31:44 -0800 Subject: [PATCH 12/24] ADFA-7083: Describe sqlite-js 1.4.1, the release with our fixes Upstream released 1.4.0 and 1.4.1 on 2026-11-12. Both contain c8ae85a, and their C code differs from it only in the version string, so the terminal package moves to 1.4.1 (terminal-packages #6). That answers both comments from the independent review: - Overview page, README.md and CLAIMS.md: the pages now describe 1.4.1 and later. "1.3.7-3 and later" was wrong: 1.3.7-3 to 1.3.7-5 lack the fix for a lowered memory limit (upstream ef06558). - Functions page: js_version() gives 1.4.1, so the note that a newer package still reports 1.3.7 is gone. The last section is now "What changed in sqlite-js 1.4" and says that 1.4.0 includes our fixes. - CLAIMS.md: the probes that cited older builds were run again on 1.4.1, with the same results: redefinition, js_init_table(1) twice, js_set_max_stack_size's inputs and recursion, the held-memory limit, db.exec errors and variable sharing. test_examples.py: 64 of 64 on a host build of 1.4.1. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 22 +++++++++---------- docs/docdb/ADFA-7083/README.md | 2 +- .../ADFA-7083/pages/sqlite-js-functions.md | 8 +++---- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 4 ++-- 4 files changed, 17 insertions(+), 19 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 5802af7afe..8225925730 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,13 +1,13 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.3.7-p20261112, upstream main at c8ae85a with no patches, and on 1.3.7-5, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-1, 1.3.7-2 and 1.3.7-4, patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, upstream main from 1.3.7-p20261112. `probe` = run by hand against the host build, in the accuracy review (2026-11-11) or the review of PR 2241 (2026-11-12). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.4.1, upstream's release with no patches, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`. `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.3.7-3 and later, such as 1.3.7-p20261112-0; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: 0008, which the javascript page describes, is in 1.3.7-3; stage's Packages index lists sqlite-js at `Version: 1.3.7-p20261112-0` for aarch64 and arm, published by CI run 43183024875 (termux-packages appends `-0` to a version that contains a hyphen, `termux_step_start_build.sh`); `dpkg --compare-versions 1.3.7-p20261112-0 gt 1.3.7-3` is true; the examples pass on 1.3.7-5 and 1.3.7-p20261112; host: `dpkg -s` prints a `Version:` line. TODO: `dpkg -s sqlite-js` on the phone | +| These pages describe 1.4.1 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) builds the 1.4.1 tag with no revision, so its version is `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.1 gt 1.3.7-p20261112-0` is true; the examples pass on 1.4.1; host: `dpkg -s` prints a `Version:` line. TODO: the 1.4.1 package on stage, and `dpkg -s sqlite-js` on the phone | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | @@ -16,7 +16,7 @@ | Ctrl-C stops code, query fails with `interrupted` | run (package run-tests AC8) | | 64 MB memory limit; `out of memory` | run (AC9) | | Do not put `.load` in `~/.sqliterc` | advice, follows from the above | -| The version in Code on the Go is newer than the 1.3.7 release and includes fixes from App Dev For All | code: 1.3.7-p20261112's `build.sh` (terminal-packages 428a196) builds upstream main at c8ae85a, committed 2026-11-12; the 1.3.7 tag is its ancestor, and it merges App Dev For All's PRs #11 to #19 (git log 1.3.7..c8ae85a) | +| Version 1.4.0 and later include fixes from App Dev For All | code: upstream's tags 1.4.0 (44528d0) and 1.4.1 (9923fe3) contain c8ae85a, which merges App Dev For All's PRs #11 to #19 (git log 1.3.7..1.4.1); 1.4.0's CHANGELOG lists the changes | ## sqlite-js-functions @@ -24,18 +24,18 @@ |-------|-------| | Each `js_` function but `js_version` cannot be used in a view, trigger, CHECK, generated column or index stored in a database; TEMP views and triggers can use them | run (AC10, flags check); probe: `pragma_function_list` flags, a main-schema trigger fails with `unsafe use of js_eval()`, a TEMP view and TEMP trigger work | | Examples for scalar, aggregate (per-group state), window, collation, `js_eval`, `js_init_table`, `js_version` | run | -| Redefining in a session that has the function fails with the quoted message; create the new version first in a new session, without `js_init_table(1)`; it replaces the stored one, and the next session's `js_init_table(1)` creates it | run (package AC4); probe: after `js_init_table(1)` the redefinition fails; without it, it succeeds and the next session's `js_init_table(1)` loads the new version. Probe on 1.3.7-4 (2026-11-12), file database: in a new session the redefinition succeeded and `f()` gave the new version; the next session's `js_init_table(1)` gave the new `f` and the unchanged `g` | -| `js_init_table(1)` runs once per session and fails with the same message if a stored function already exists in it | probe on 1.3.7-4: a second `js_init_table(1)`, and `js_init_table(1)` after creating any function, fail with the message. Code: `sqlite3_create_function_v2` returns `SQLITE_BUSY` when it replaces a function while the calling statement runs, and sqlite-js reports that as the message | +| Redefining in a session that has the function fails with the quoted message; create the new version first in a new session, without `js_init_table(1)`; it replaces the stored one, and the next session's `js_init_table(1)` creates it | run (package AC4); probe: after `js_init_table(1)` the redefinition fails; without it, it succeeds and the next session's `js_init_table(1)` loads the new version. Probe on 1.3.7-4 and on 1.4.1 (2026-11-12), file database: in a new session the redefinition succeeded and `f()` gave the new version; the next session's `js_init_table(1)` gave the new `f` and the unchanged `g` | +| `js_init_table(1)` runs once per session and fails with the same message if a stored function already exists in it | probe on 1.3.7-4 and 1.4.1: a second `js_init_table(1)`, and `js_init_table(1)` after creating any function, fail with the message. Code: `sqlite3_create_function_v2` returns `SQLITE_BUSY` when it replaces a function while the calling statement runs, and sqlite-js reports that as the message | | `init` can be NULL; no rows gives NULL without running `init` or `final` | run on phone (NULL init); probe and code (`js_execute_final` returns NULL without a context). No NULL-init example on the page, deliberately: the `init` example shows the usual pattern | | `value`/`inverse` semantics | code, run | | Collation: only the sign counts; a throwing collation compares as equal | run on phone: a throwing collation leaves the rows in their original order, i.e. equal | | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | -| `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL; a limit below the memory variables still hold leaves no room for any JavaScript until raised | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error; probe on 1.3.7-5 and 1.3.7-p20261112: with 40 MB held in a global and the limit at 4 MB, `js_eval('1 + 1')` and `js_eval('2 + 2')` fail with `out of memory`, and after raising it to 64 MB `js_eval('3 + 3')` gives 6 | -| `js_set_max_stack_size` (1.3.7-2, patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (patch 0007); probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | +| `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL; a limit below the memory variables still hold leaves no room for any JavaScript until raised | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error; probe on 1.3.7-5, 1.3.7-p20261112 and 1.4.1: with 40 MB held in a global and the limit at 4 MB, `js_eval('1 + 1')` and `js_eval('2 + 2')` fail with `out of memory`, and after raising it to 64 MB `js_eval('3 + 3')` gives 6 | +| `js_set_max_stack_size` (1.4.0; 1.3.7-2 to 1.3.7-5 as patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (1.4.1; patch 0007 before); probe on the 1.4.1 host build (2026-11-12): the same inputs give the error, naming 8131952 as the largest size, `262144`, `1048576` and `0` are accepted, and endless recursion at 4 KB below the largest fails with `Maximum call stack size exceeded`; probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | | `js_version(1)` is quickjs-ng | run (package AC2) | -| `js_version()` gives `1.3.7`; a package built from newer upstream code, such as 1.3.7-p20261112-0, still reports `1.3.7`; `dpkg -s sqlite-js` shows the package version | run; probe on the 1.3.7-p20261112 host build: `SELECT js_version()` gives 1.3.7, and `SQLITE_JS_VERSION` is "1.3.7" at c8ae85a; the published package's version is `1.3.7-p20261112-0` (stage's Packages index, CI run 43183024875) | -| Built from upstream code of 12 November 2026, newer than 1.3.7, which includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: upstream main c8ae85a (2026-11-12) merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build | +| `js_version()` gives `1.4.1` | run (the example's `-- 1.4.1`); `SQLITE_JS_VERSION` is "1.4.1" in the 1.4.1 tag's `src/sqlitejs.h` | +| sqlite-js 1.4.0 includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: 1.4.0 contains c8ae85a (2026-11-12), which merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19, and its CHANGELOG lists the same changes; 1.4.1's C code is c8ae85a's; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build, the same C code | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | @@ -51,9 +51,9 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.3.7-3, patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE` | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE` | | Parameters convert like results | run (package AC6) | -| Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | +| Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | | Ctrl-C works "even code that catches every error"; "`sqlite3` keeps running if you were typing commands" | run (AC8) | | Stack overflow message `Maximum call stack size exceeded` | run by hand | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 9c60caa867..66f4f6b208 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -9,7 +9,7 @@ Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached fr | JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | | JavaScript basics | `i/javascript-intro.html` | -They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.3.7-3 and later, including 1.3.7-p20261112-0, which is upstream's main branch after 1.3.7 with our fixes merged. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.4.1 and later: upstream's 1.4.0 release merged our fixes. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 5ff04291f2..3578cb59c3 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -182,14 +182,12 @@ js_version(1) ```sql SELECT js_version(); --- 1.3.7 +-- 1.4.1 ``` -A package built from upstream code newer than 1.3.7, such as 1.3.7-p20261112-0, still reports `1.3.7`. To see the package version, enter `dpkg -s sqlite-js` in Terminal. +## What changed in sqlite-js 1.4 -## What is different from sqlite-js 1.3.7 - -The sqlite-js in Code on the Go is built from the upstream project's code of 12 November 2026, which is newer than its 1.3.7 release. That code includes fixes that App Dev For All sent to the upstream project. Compared with 1.3.7: +sqlite-js 1.4.0 includes fixes that App Dev For All sent to the upstream project. Compared with 1.3.7: - Ctrl-C stops JavaScript that never ends. - JavaScript may use 64 MB of memory, and `js_set_memory_limit` changes the limit. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 0a373c162d..2362fb34ab 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -29,7 +29,7 @@ apt install sqlite sqlite-js The `sqlite` package provides the `sqlite3` command that sqlite-js works with. -These pages describe sqlite-js 1.3.7-3 and later versions, such as 1.3.7-p20261112-0. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. +These pages describe sqlite-js 1.4.1 and later versions. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. ## Load sqlite-js @@ -78,4 +78,4 @@ A function you create lasts until you close `sqlite3`. To keep your functions in - [JavaScript in sqlite-js](sqlite-js-javascript.html): how SQL values and JavaScript values convert, what the engine supports, and the limits. - [JavaScript basics](javascript-intro.html): a short introduction for new JavaScript users. -sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. The version in Code on the Go is newer than the 1.3.7 release and includes fixes from App Dev For All, described in the reference. +sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. Version 1.4.0 and later include fixes from App Dev For All, described in the reference. From c32e76ad9ed544bbd23f5043275d5d5e12ca3f60 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 15:53:27 -0800 Subject: [PATCH 13/24] ADFA-7083: Note that the examples pass with the package's two patches The terminal package is 1.4.1 with two patches (terminal-packages #6): the stack check and garbage collection. test_examples.py passes 64 of 64 on a host build with them, as on 1.4.1 without them, and no page claim changes. CLAIMS says so. The pages and the test database are unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 8225925730..0e827c3399 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,6 +1,6 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.4.1, upstream's release with no patches, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`. `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.4.1 with the package's two patches, stack and garbage collection, which change none of the pages' claims, and on 1.4.1 without them, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`, with the package's patches 0001 and 0002 (sqliteai/sqlite-js #22 and #23). `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. ## sqlite-js-top From 9931e0c3dee491674a0b24e9b9622d03c0fd1e21 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 19:54:21 -0800 Subject: [PATCH 14/24] ADFA-7083: Say that a PRAGMA can take effect in db.exec itself CodeRabbit: db.exec prepares its statement at once, and a PRAGMA that changes a setting of the connection takes effect then, without next() or toArray(). Checked on 1.4.1 (sqlite3 3.46.1): busy_timeout, foreign_keys and cache_size take effect, user_version and application_id do not. The JavaScript page and CLAIMS say so. test_examples.py passes 64 of 64, and the test database is rebuilt. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 0e827c3399..ddbae9803f 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -51,7 +51,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE` | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except a PRAGMA that changes a connection setting, which takes effect when `db.exec` prepares it; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index 850b38af15..cc196738cb 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -90,7 +90,7 @@ SELECT big_next(CAST(9223372036854775807 AS TEXT)); ## Run SQL from JavaScript -The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs only when you call `next()` or `toArray()`. +The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: a `PRAGMA` that changes a setting of the connection, such as `PRAGMA foreign_keys = ON`, takes effect already in `db.exec`. | Result set | What it does | |------------|--------------| From 03f6b1d9ac1716deffd9b26f2efd8d865533fe03 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 20:27:15 -0800 Subject: [PATCH 15/24] ADFA-7083: Say that only some PRAGMAs take effect in db.exec CodeRabbit: "a PRAGMA that changes a setting of the connection" was too broad. journal_mode is such a setting, and takes effect only when the statement is stepped: with the C API on SQLite 3.49.1, PRAGMA journal_mode=OFF prepared but not stepped leaves the mode delete. The page now says that some PRAGMAs take effect already in db.exec, and that which ones do depends on the PRAGMA; CLAIMS has the probe. test_examples.py passes 64 of 64, and the test database is rebuilt. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index ddbae9803f..eb1ba979ee 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -51,7 +51,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except a PRAGMA that changes a connection setting, which takes effect when `db.exec` prepares it; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete` | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index cc196738cb..f4a54baf6e 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -90,7 +90,7 @@ SELECT big_next(CAST(9223372036854775807 AS TEXT)); ## Run SQL from JavaScript -The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: a `PRAGMA` that changes a setting of the connection, such as `PRAGMA foreign_keys = ON`, takes effect already in `db.exec`. +The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: some `PRAGMA` statements, such as `PRAGMA foreign_keys = ON`, take effect already in `db.exec`. Which ones do depends on the `PRAGMA`. | Result set | What it does | |------------|--------------| From 898848f63f8e6f46d7b88123304b8185f1c45798 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 20:42:36 -0800 Subject: [PATCH 16/24] ADFA-7083: Say that PRAGMA foreign_keys does nothing inside a transaction CodeRabbit: the page gives PRAGMA foreign_keys = ON as an example of a PRAGMA that takes effect in db.exec, and SQLite makes it a no-op inside a transaction or savepoint, with no error. Checked on 1.4.1 (sqlite3 3.46.1): through db.exec it stays 0 after BEGIN or SAVEPOINT, and becomes 1 outside a transaction. The page now says to set it before BEGIN or SAVEPOINT, as CodeRabbit suggested; CLAIMS has the probe. test_examples.py passes 64 of 64, and the test database is rebuilt. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index eb1ba979ee..c8894d6941 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -51,7 +51,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete` | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index f4a54baf6e..88d4b1455c 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -90,7 +90,7 @@ SELECT big_next(CAST(9223372036854775807 AS TEXT)); ## Run SQL from JavaScript -The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: some `PRAGMA` statements, such as `PRAGMA foreign_keys = ON`, take effect already in `db.exec`. Which ones do depends on the `PRAGMA`. +The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: some `PRAGMA` statements, such as `PRAGMA foreign_keys = ON`, take effect already in `db.exec`. Which ones do depends on the `PRAGMA`. `PRAGMA foreign_keys` has no effect inside a transaction or savepoint, so set it before you run `BEGIN` or `SAVEPOINT`. | Result set | What it does | |------------|--------------| From 32f230f9d9ae0f9e31631ca445d395a8a91c2e92 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Thu, 12 Nov 2026 23:55:46 -0800 Subject: [PATCH 17/24] ADFA-7083: Let apply.py add the link to the current documentation.db The phone test found that apply.py failed on the database a current build ships (wholedb 2026-11-02). It reads i/terminal-top.html as JSON, the template context of databases up to 2026-10-16, but every page there is now a finished Help+Manual page (HTML). Those databases also store i/jstopics/.js (365 of them), which the help viewer loads to move between pages. So even a terminal-top.html with the link would not show it in the app. apply.py now: - adds the section to the HTML page at the end of its topic body (before the skin's hmpagebody_scroller end), keeping the BOM; - adds it to a stored i/jstopics/terminal-top.js too, at the end of its hmBody string, escaped as Help+Manual writes it; - leaves JSON databases as before: the result is byte-identical, and the app renders their jstopics files from the page; - finds an existing section by its anchor, id="sqlite-js", so a second run changes nothing in either form. The new pages stay template contexts, which the app renders with helpmanual-topic.peb (id 3607 there), and their jstopics files are rendered from them. Checked on a copy of the phone's database: both files get the section, the jstopics file evaluates in Node with it at the end of hmBody, and a second run changes nothing. In the app on the phone, the Terminal page shows the section and its link opens the pages. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/README.md | 2 +- docs/docdb/ADFA-7083/apply.py | 68 +++++++++++++++++++++++++++++++--- 2 files changed, 63 insertions(+), 7 deletions(-) diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 66f4f6b208..5b8dcc1a0d 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -13,7 +13,7 @@ They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.n - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. -- `apply.py DB`: adds the pages to a `documentation.db` in one transaction, compressed against its `CompressionDictionary`, and appends a section with a link to `i/terminal-top.html`. A second run changes nothing. It refuses a database that declares a version below 2. Run it on a copy first; the master is edited by a `docdb-studio` maintainer. Needs the `brotli` command. +- `apply.py DB`: adds the pages to a `documentation.db` in one transaction, compressed against its `CompressionDictionary`, and appends a section with a link to `i/terminal-top.html`. It handles both forms of that page: a template context (JSON), as in databases up to `wholedb` 2026-10-16, and a finished Help+Manual page (HTML), as from 2026-11-02. A database of the second form also stores `i/jstopics/terminal-top.js`, which the help viewer loads to move between pages, and the section goes into it too. A second run changes nothing. It refuses a database that declares a version below 2. Run it on a copy first; the master is edited by a `docdb-studio` maintainer. Needs the `brotli` command. - `test_examples.py`: runs every ` ```sql ` example against a real `sqlite3` with sqlite-js loaded, runs again each one that creates nothing, as a reader would, and checks that links, including each page's parent, prev and next, resolve: `SQLITE3=sqlite3 EXT=/path/to/sqlite-js.so python3 test_examples.py`. - `CLAIMS.md`: every factual claim in the pages, and how each was checked. diff --git a/docs/docdb/ADFA-7083/apply.py b/docs/docdb/ADFA-7083/apply.py index 5230c428ae..dda88a61c8 100644 --- a/docs/docdb/ADFA-7083/apply.py +++ b/docs/docdb/ADFA-7083/apply.py @@ -10,7 +10,13 @@ i/, templateId of helpmanual-topic.peb, text/html, EN-us; - appends a section that links to the new pages to i/terminal-top.html, unless it has one. The page's other sections start collapsed, so the link - gets a heading of its own, where readers see it; + gets a heading of its own, where readers see it. The page is either a + template context (JSON, rendered by helpmanual-topic.peb), as in databases + up to wholedb 2026-10-16, or a finished Help+Manual page (HTML), as from + wholedb 2026-11-02; the section goes at the end of the topic's body in both. + A database of the second kind also stores the page's i/jstopics/ + terminal-top.js, which the help viewer loads to move between pages, so the + section goes into that file's hmBody too; - if a page or the link changed, stamps the content-i and wholedb rows of LastChange, inserting any that is missing, as docdb-studio does. Any failure rolls everything back. A second run changes nothing. @@ -23,6 +29,7 @@ """ import json import os +import re import shutil import sqlite3 import subprocess @@ -37,6 +44,43 @@ '

To write SQL functions in JavaScript, see ' 'JavaScript in SQLite (sqlite-js).

\r\n') WHO = "ADFA-7083" +# A finished page's topic body ends where the skin closes its scroller div. +HTML_BODY_END = re.compile(r"\n\s*\s*") +# A stored jstopics file is hmLoadTopic({...}) whose last property is the hmBody string. +JSTOPIC_BODY_END = re.compile(r'"\s*\}\)\s*$') + + +def add_link(text): + """The page with the link section added, or None if it already has it.""" + if text.lstrip("\ufeff \t\r\n").startswith("{"): + page = json.loads(text) + if 'id="sqlite-js"' in page["hmBody"]: + return None + page["hmBody"] += LINK_HTML + return json.dumps(page, ensure_ascii=False) + if 'id="sqlite-js"' in text: + return None + ends = list(HTML_BODY_END.finditer(text)) + if len(ends) != 1: + raise RuntimeError("i/%s is HTML, but its topic body's end was found %d times, not once" + % (LINK_FILE, len(ends))) + at = ends[0].start() + return text[:at] + "\n" + LINK_HTML.replace("\r\n", "\n").rstrip("\n") + text[at:] + + +def add_link_to_jstopic(text): + """The jstopics file with the link section added to its hmBody, or None if it has it.""" + if 'id=\\"sqlite-js\\"' in text: + return None + ends = list(JSTOPIC_BODY_END.finditer(text)) + body = text.rfind("hmBody:") + if len(ends) != 1 or body < 0 or re.search(r"\n\s*hm\w+:", text[body:]): + raise RuntimeError("i/jstopics/%s does not end with its hmBody string" % LINK_FILE.replace(".html", ".js")) + # escaped as Help+Manual writes these strings: \" for quotes, <\/ for Date: Thu, 12 Nov 2026 23:55:46 -0800 Subject: [PATCH 18/24] ADFA-7083: Keep the pages whole at font scale 2.0 Read through in the app on a phone at font scale 1.0 and 2.0. At 2.0, three long addresses ran off the right edge, and part of each was cut off: github.com/sqliteai/sqlite-js on the top page, the Apache license URL in the reference's credits, and the MDN guide's URL on the basics page. They are reworded to name the project, the license and the site without the long addresses. The license texts are installed with the package, as the credits say. The result-set table on the JavaScript page broke the method names mid-word, at 1.0 too ("toArra" / "y()"). It is now a list, which keeps each name whole at both scales. test_examples.py passes 64 of 64. CLAIMS records the phone check and the apply.py run on the current database. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 4 ++-- docs/docdb/ADFA-7083/pages/javascript-intro.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 14 +++++++------- docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 2 +- 5 files changed, 12 insertions(+), 12 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index c8894d6941..10f6cbff23 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -71,6 +71,6 @@ | Claim | Check | |-------|-------| -| Pages render through `helpmanual-topic.peb` | rendered with Pebble 4.1.1, the app's engine before it was vendored, from a test database built by `apply.py`: the four pages, `terminal-top` and `cogo-top` render, with titles, unescaped bodies and working links. TODO: render in the app | -| `apply.py`: a second run changes nothing; it refuses a database below version 2; the Terminal page's link has a heading of its own | run on copies of documentation.db 3.0.0: the second run printed `unchanged` for each page and left the file byte-identical; one declaring 1.9.0 was refused with no row changed; `LastChange`'s `content-i` and `wholedb` rows were stamped, not added | +| Pages render through `helpmanual-topic.peb` | rendered with Pebble 4.1.1, the app's engine before it was vendored, from a test database built by `apply.py`: the four pages, `terminal-top` and `cogo-top` render, with titles, unescaped bodies and working links. TODO: render in the app; on 2026-11-13, in the app on a Galaxy Note20 Ultra (debug build C-d-1113-0401) with the test database built from its installed one: the Terminal page shows the new section, its link and the links between the four pages work in the help viewer, and each page was read through at font scale 1.0 and 2.0 | +| `apply.py`: a second run changes nothing; it refuses a database below version 2; the Terminal page's link has a heading of its own | run on copies of documentation.db 3.0.0: the second run printed `unchanged` for each page and left the file byte-identical; one declaring 1.9.0 was refused with no row changed; `LastChange`'s `content-i` and `wholedb` rows were stamped, not added; on 2026-11-13 also run on the phone's installed database (`wholedb` 2026-11-02, every page finished HTML, 365 stored `jstopics` files): it adds the section to `terminal-top.html` and to `jstopics/terminal-top.js`, which evaluates in Node with the section at the end of `hmBody`, and a second run changes nothing. Before this, it failed on that database: it read the page as JSON | | Phone layout of tables and code blocks | body-only render in headless Chrome (code blocks wrap, tables fit); TODO: on a device at font scale 1.0 and 2.0 | diff --git a/docs/docdb/ADFA-7083/pages/javascript-intro.md b/docs/docdb/ADFA-7083/pages/javascript-intro.md index 89b3f0cdc0..867b327e52 100644 --- a/docs/docdb/ADFA-7083/pages/javascript-intro.md +++ b/docs/docdb/ADFA-7083/pages/javascript-intro.md @@ -122,4 +122,4 @@ SELECT js_eval('"order 66 shipped on 2026-11-11".match(/\d{4}-\d{2}-\d{2}/)[0]') ## Keep learning -The [MDN JavaScript Guide](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide) teaches the whole language. In sqlite-js, ignore the parts about web pages, such as the DOM, `fetch` and timers: see [JavaScript in sqlite-js](sqlite-js-javascript.html) for what is available. +The JavaScript Guide on MDN, at developer.mozilla.org, teaches the whole language. In sqlite-js, ignore the parts about web pages, such as the DOM, `fetch` and timers: see [JavaScript in sqlite-js](sqlite-js-javascript.html) for what is available. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 3578cb59c3..bf379a8a81 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -200,6 +200,6 @@ sqlite-js 1.4.0 includes fixes that App Dev For All sent to the upstream project ## Credits -This page adapts the README of [sqlite-js](https://github.com/sqliteai/sqlite-js), which is licensed under the [Apache License, Version 2.0](https://www.apache.org/licenses/LICENSE-2.0). It has been rewritten and changed for Code on the Go. The software is provided "as is", without warranties or conditions of any kind. +This page adapts the README of sqlite-js (the project sqliteai/sqlite-js on GitHub), which is licensed under the Apache License, Version 2.0. It has been rewritten and changed for Code on the Go. The software is provided "as is", without warranties or conditions of any kind. The JavaScript engine, quickjs-ng, is built into sqlite-js and is licensed under the MIT License. The license texts are installed with the package, in `$PREFIX/share/doc/sqlite-js/`: `LICENSE` for sqlite-js and `LICENSE.quickjs` for the engine. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index 88d4b1455c..d4fb549bdc 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -92,13 +92,13 @@ SELECT big_next(CAST(9223372036854775807 AS TEXT)); The global object `db` runs SQL on the same database. `db.exec(sql, ...parameters)` takes a statement, with `?` for each parameter, and returns a result set. The statement runs when you call `next()` or `toArray()`. One exception: some `PRAGMA` statements, such as `PRAGMA foreign_keys = ON`, take effect already in `db.exec`. Which ones do depends on the `PRAGMA`. `PRAGMA foreign_keys` has no effect inside a transaction or savepoint, so set it before you run `BEGIN` or `SAVEPOINT`. -| Result set | What it does | -|------------|--------------| -| `next()` | Runs the statement to its next row. It returns `true`, or `false` when there are no more rows. If the statement fails, it throws an error. | -| `get(i)` | Returns column `i` of the current row, counting from 0. | -| `name(i)` | Returns the name of column `i`. | -| `columnCount` | The number of columns. | -| `toArray()` | Returns all the remaining rows, each as an array. If the statement fails partway, it throws an error. | +A result set has these methods and property: + +- `next()` runs the statement to its next row. It returns `true`, or `false` when there are no more rows. If the statement fails, it throws an error. +- `get(i)` returns column `i` of the current row, counting from 0. +- `name(i)` returns the name of column `i`. +- `columnCount` is the number of columns. +- `toArray()` returns all the remaining rows, each as an array. If the statement fails partway, it throws an error. ```sql SELECT js_eval('(function () { diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 2362fb34ab..537cbef699 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -78,4 +78,4 @@ A function you create lasts until you close `sqlite3`. To keep your functions in - [JavaScript in sqlite-js](sqlite-js-javascript.html): how SQL values and JavaScript values convert, what the engine supports, and the limits. - [JavaScript basics](javascript-intro.html): a short introduction for new JavaScript users. -sqlite-js is developed at github.com/sqliteai/sqlite-js under the Apache License 2.0. Version 1.4.0 and later include fixes from App Dev For All, described in the reference. +sqlite-js is developed on GitHub, in the project sqliteai/sqlite-js, under the Apache License 2.0. Version 1.4.0 and later include fixes from App Dev For All, described in the reference. From 81f6ed332fec9d932aadcaf4a1ed64f9e8f34478 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Fri, 13 Nov 2026 03:04:42 -0800 Subject: [PATCH 19/24] ADFA-7083: Require helpmanual-jstopic.peb before adding the pages CodeRabbit: the app renders the new pages' jstopics files, which the help viewer loads to move between pages, with helpmanual-jstopic.peb, and answers 404 without it. apply.py checked only helpmanual-topic.peb, so it would add pages to a database where moving to them fails. It now refuses such a database. Checked: a copy of the current database without the template is refused and left unchanged; the current and the 3.0.0 databases still take the pages, and a second run changes nothing. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/apply.py | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/docdb/ADFA-7083/apply.py b/docs/docdb/ADFA-7083/apply.py index dda88a61c8..1357acb1e0 100644 --- a/docs/docdb/ADFA-7083/apply.py +++ b/docs/docdb/ADFA-7083/apply.py @@ -110,10 +110,12 @@ def main(): db = sqlite3.connect(db_path, isolation_level=None) db.execute("PRAGMA foreign_keys = ON") template = db.execute("SELECT id FROM Templates WHERE name = 'helpmanual-topic.peb'").fetchone() + # the app renders the new pages' jstopics files with it: without it, moving to them fails + jstopic_template = db.execute("SELECT id FROM Templates WHERE name = 'helpmanual-jstopic.peb'").fetchone() html_type = db.execute("SELECT id FROM ContentTypes WHERE value = 'text/html'").fetchone() lang = db.execute("SELECT id FROM Languages WHERE lower(value) = 'en-us'").fetchone() dictionary = db.execute("SELECT data FROM CompressionDictionary WHERE id = 1").fetchone() - if not (template and html_type and lang and dictionary): + if not (template and jstopic_template and html_type and lang and dictionary): sys.exit("this does not look like a documentation.db: a template, content type, language or " "dictionary is missing") # The app's own query and order (DatabaseVersionResolver): the newest row's version counts. From 139870bad5d52ac6e59975468bed4555614668dc Mon Sep 17 00:00:00 2001 From: David Schachter Date: Fri, 13 Nov 2026 10:00:30 -0800 Subject: [PATCH 20/24] ADFA-7083: Describe sqlite-js 1.4.2 Upstream released 1.4.2 with App Dev For All's stack, garbage collection and interrupted result set fixes (sqliteai/sqlite-js #22 to #24) and the maintainer's fix for issue #25, and the terminal package moves to it with no patches. The pages now say: - top: they describe 1.4.2 and later. - functions: js_version() gives 1.4.2; js_set_max_stack_size allows at most 1 MB when the stack has no limit (ulimit -s unlimited); "What changed in sqlite-js 1.4" lists 1.4.2's five differences from 1.4.1. - javascript: a db.exec result set lasts only until the call into JavaScript that created it returns, and reading it later throws "The result set was closed when the call into JavaScript that created it returned", with a tested example; Variables says a variable cannot keep a result set open. test_examples.py: 66 of 66 on a host build of plain 1.4.2 (28 examples, 19 run twice, 19 links). On 1.4.1 the js_version and kept result set examples fail, as they should. Probed on 1.4.2: the PRAGMA claims hold, an INSERT without next() still adds nothing, a result set read to its end still gives next() false, and under ulimit -s unlimited the stack error names 1048576. CLAIMS.md records each check; the window function crash fix rests on 1.4.2's code and changelog, since the saved reproducer did not crash the 1.4.1 host build either. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 11 ++++++----- .../docdb/ADFA-7083/pages/sqlite-js-functions.md | 12 ++++++++++-- .../ADFA-7083/pages/sqlite-js-javascript.md | 16 ++++++++++++++++ docs/docdb/ADFA-7083/pages/sqlite-js-top.md | 2 +- 4 files changed, 33 insertions(+), 8 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 10f6cbff23..63c4a9eaad 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,13 +1,13 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (64 of 64 on 1.4.1 with the package's two patches, stack and garbage collection, which change none of the pages' claims, and on 1.4.1 without them, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`, with the package's patches 0001 and 0002 (sqliteai/sqlite-js #22 and #23). `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (66 of 66 on plain 1.4.2, the package from the move to 1.4.2, 2026-11-13: 28 SQL examples, 19 of them run a second time, and 19 links; on 1.4.1 the `js_version` example and the kept result set example fail, as they should. Before that, 64 of 64 on 1.4.1 with the package's two patches, stack and garbage collection, which change none of the pages' claims, and on 1.4.1 without them, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`, with the package's patches 0001 and 0002 (sqliteai/sqlite-js #22 and #23). `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.4.1 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) builds the 1.4.1 tag with no revision, so its version is `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.1 gt 1.3.7-p20261112-0` is true; the examples pass on 1.4.1; host: `dpkg -s` prints a `Version:` line. TODO: the 1.4.1 package on stage, and `dpkg -s sqlite-js` on the phone | +| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) moves to the 1.4.2 tag with no patches and no revision (2026-11-13), so its version is `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.1 gt 1.3.7-p20261112-0` is true; the examples pass on 1.4.1; host: `dpkg -s` prints a `Version:` line. TODO: the 1.4.1 package on stage, and `dpkg -s sqlite-js` on the phone | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | @@ -32,10 +32,11 @@ | `js_init_table()` columns and "whenever that table exists, each function you create is stored" | run on phone: with the table left by an earlier session and no `js_init_table()` call, a new function was stored (`js_functions` listed both), and `js_init_table(1)` in a new session restored both | | `js_load_text`/`js_load_blob`: empty file gives empty value; error text `Unable to open the file` | run on phone: empty file gives `''` / `X''`; a missing file gives `Unable to open the file` (the shell prefixes `Error: stepping, `) | | `js_set_memory_limit`: default 64 MB, 0 no limit, a whole number 0 or more, error text, returns NULL; a limit below the memory variables still hold leaves no room for any JavaScript until raised | run (package AC9), code; probe: `'64'`, `-1` and `1.5` each give the error; probe on 1.3.7-5, 1.3.7-p20261112 and 1.4.1: with 40 MB held in a global and the limit at 4 MB, `js_eval('1 + 1')` and `js_eval('2 + 2')` fail with `out of memory`, and after raising it to 64 MB `js_eval('3 + 3')` gives 6 | -| `js_set_max_stack_size` (1.4.0; 1.3.7-2 to 1.3.7-5 as patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes | code (1.4.1; patch 0007 before); probe on the 1.4.1 host build (2026-11-12): the same inputs give the error, naming 8131952 as the largest size, `262144`, `1048576` and `0` are accepted, and endless recursion at 4 KB below the largest fails with `Maximum call stack size exceeded`; probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | +| `js_set_max_stack_size` (1.4.0; 1.3.7-2 to 1.3.7-5 as patch 0007): default 1 MB; only a whole number the stack allows, else an error naming the largest size; 0 turns the check off and deep recursion crashes; with no stack limit, at most 1 MB | code (1.4.1; patch 0007 before; 1.4.2's 31ed714 counts an unlimited `RLIMIT_STACK` as unmeasurable); probe on the 1.4.2 host build (glibc, 2026-11-13): under `ulimit -s unlimited` the error names 1048576 as the largest size, and with 8 MB it names 8109952. TODO: the unlimited case on the phone (Bionic); probe on the 1.4.1 host build (2026-11-12): the same inputs give the error, naming 8131952 as the largest size, `262144`, `1048576` and `0` are accepted, and endless recursion at 4 KB below the largest fails with `Maximum call stack size exceeded`; probe on the 1.3.7-2 host build: `'abc'`, `-1`, `1.5`, `NULL`, `1073741824` and `16777216` give the error, `262144` and `1048576` are accepted, endless recursion fails cleanly just below the largest size; package run-tests.sh AC13 (87 of 87). 1.3.7-1 instead crashed on text, 0, or sizes above the stack. Phone: 1.3.7-2 passed 87 of 87; 1.3.7-3 and 1.3.7-4 passed 91 of 91 on aarch64 and arm | | `js_version(1)` is quickjs-ng | run (package AC2) | -| `js_version()` gives `1.4.1` | run (the example's `-- 1.4.1`); `SQLITE_JS_VERSION` is "1.4.1" in the 1.4.1 tag's `src/sqlitejs.h` | +| `js_version()` gives `1.4.2` | run (the example's `-- 1.4.2`); `SQLITE_JS_VERSION` is "1.4.2" in the 1.4.2 tag's `src/sqlitejs.h` | | sqlite-js 1.4.0 includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: 1.4.0 contains c8ae85a (2026-11-12), which merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19, and its CHANGELOG lists the same changes; 1.4.1's C code is c8ae85a's; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build, the same C code | +| sqlite-js 1.4.2 has more fixes, several sent or reported by App Dev For All; the five differences from 1.4.1 listed | code: upstream's 1.4.2 tag (74557b2) contains App Dev For All's sqliteai/sqlite-js PRs #22 (stack, with the maintainer's bd3d6c4 and 31ed714), #23 (garbage collection) and #24 (result sets an interrupt leaves open), whose patch ids match the package's former patches 0001 to 0004, and the maintainer's #26 for App Dev For All's issue #25 (window function crash) with the result set lifetime change; 1.4.2's CHANGELOG lists the same. run: the kept result set example (fails on 1.4.1, where `next()` returned `true` and `sqlite3` reported `unable to close due to unfinalized statements`). The window function crash was not reproduced here: the saved reproducer ran 20,000 turns without crashing on the 1.4.1 host build too, so that fix rests on the code and the CHANGELOG | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | @@ -51,7 +52,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec` | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec`; a result set lasts until the call into JavaScript that created it returns, after which `next()`, `toArray()`, `get()` and `name()` throw the quoted message and one read to its end still reports no more rows (1.4.2) | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction. Probe on the 1.4.2 host build (2026-11-13): the same PRAGMA results; a result set kept in a global throws the quoted message from `next()`, `toArray()`, `get()` and `name()` in a later `js_eval` (`columnCount` still answers); one read to its end by `toArray()` gives `next()` false; an INSERT without `next()` still adds nothing; the kept result set example | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index bf379a8a81..38e8486c68 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -167,7 +167,7 @@ Sets the most memory, in bytes, that JavaScript may use in this session. The def js_set_max_stack_size(bytes) ``` -Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Give a whole number of bytes, such as `1048576`, no larger than the stack of `sqlite3` allows. Anything else is an error, and the error message names the largest size allowed. Do not use `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. +Sets the size of the JavaScript stack, which limits how deeply functions can call themselves. The default is 1 MB. Give a whole number of bytes, such as `1048576`, no larger than the stack of `sqlite3` allows. Anything else is an error, and the error message names the largest size allowed. If the stack has no limit, as after `ulimit -s unlimited`, sqlite-js cannot measure it and allows at most 1 MB. Do not use `0`: it turns the check off, and code that calls itself without end then crashes `sqlite3`. ## Version @@ -182,7 +182,7 @@ js_version(1) ```sql SELECT js_version(); --- 1.4.1 +-- 1.4.2 ``` ## What changed in sqlite-js 1.4 @@ -198,6 +198,14 @@ sqlite-js 1.4.0 includes fixes that App Dev For All sent to the upstream project - `js_set_max_stack_size` accepts only a size the stack can hold, or 0, which turns the check off. With a size set, code that calls itself without end fails instead of crashing `sqlite3`. - An error while `db.exec` runs a statement, such as a failed constraint, is thrown instead of being ignored. +sqlite-js 1.4.2 has more fixes, several of them sent or reported by App Dev For All. Compared with 1.4.1: + +- A result set from `db.exec` lasts only until the call into JavaScript that created it returns. Reading it in a later call throws an error, where before it went on with its rows. +- Ctrl-C no longer leaves `sqlite3` failing every later query with `interrupted` when JavaScript kept a result set. +- Code that calls itself without end fails instead of crashing `sqlite3` in more cases. If the stack has no limit, `js_set_max_stack_size` allows at most 1 MB. +- JavaScript that runs out of memory while much of its memory is garbage no longer fails again on every try. +- A query that computes a JavaScript window function no longer crashes `sqlite3` when JavaScript drops its result set partway. + ## Credits This page adapts the README of sqlite-js (the project sqliteai/sqlite-js on GitHub), which is licensed under the Apache License, Version 2.0. It has been rewritten and changed for Code on the Go. The software is provided "as is", without warranties or conditions of any kind. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index d4fb549bdc..86e4a307e1 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -140,11 +140,27 @@ SELECT js_eval('(function () { -- not added: UNIQUE constraint failed: colors.name ``` +A result set lasts only until the call into JavaScript that created it returns: the `js_eval`, or the call of your function. Read what you need before then. In a later call, `next()`, `toArray()`, `get()` and `name()` throw `The result set was closed when the call into JavaScript that created it returned`. A result set that was read to its end still reports that it has no more rows. + +```sql +SELECT js_eval('globalThis.kept = db.exec("SELECT 1 UNION ALL SELECT 2"); kept.next(); kept.get(0)'); +SELECT js_eval('(function () { + try { + return kept.next(); + } catch (e) { + return e.message; + } +})()'); +-- 1 +-- The result set was closed when the call into JavaScript that created it returned +``` + ## Variables and state - Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. Aggregate and window functions cannot see them: define what they need in their `init` code. - A name declared with `const`, `let` or `class` at the top level of `js_eval` code also lasts for the session, so running the same code again fails with `redeclaration of`. Put the code in braces, `{ ... }`, to keep its names to one run. `js_eval` still returns the value of the last expression. - Each running aggregate or window function has variables of its own, which its `init` code sets up. That is why two groups never mix their state. +- A variable can keep a result set from `db.exec`, but not keep it open: the result set closes when the call that created it returns, as described in Run SQL from JavaScript. ## Limits diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md index 537cbef699..389c67bad7 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-top.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-top.md @@ -29,7 +29,7 @@ apt install sqlite sqlite-js The `sqlite` package provides the `sqlite3` command that sqlite-js works with. -These pages describe sqlite-js 1.4.1 and later versions. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. +These pages describe sqlite-js 1.4.2 and later versions. To see which version you have, enter `dpkg -s sqlite-js` and read its `Version` line. `apt update` and then `apt install sqlite-js` bring an older one up to date. ## Load sqlite-js From d2804d1e8e7df5120cf28b0adf499cefe885016e Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 15 Nov 2026 20:10:55 -0800 Subject: [PATCH 21/24] ADFA-7083: Say how long a result set lasts in nested calls From the independent review of 139870bad (one medium finding, the rest low), each checked on the 1.4.2 host build: - F01 (medium): a result set made by a function that SQL in another call's db.exec runs lasts until that outer call returns, not until the function returns. The Run SQL section says so, and the Variables bullet and "What changed" item say "the outermost call". - F02: what still reports no more rows is a result set whose next() returned false, or whose toArray() ran. One whose rows were all read but whose next() never returned false throws in a later call. - F03: after Ctrl-C a kept result set throws "interrupted", not the "closed" message. - F04 to F06: CLAIMS.md records the window-function crash as reproduced (1.4.1 segfaults within 60 turns of the reviewer's probe; 1.4.2 passes), the 1.4.2 source and probes in its legend, and the 1.4.2 version check; README.md says 1.4.2 and later. test_examples.py: 66 of 66 on 1.4.2; on 1.4.1 the same 4 fail as before, as they should. build.py builds the pages. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 8 ++++---- docs/docdb/ADFA-7083/README.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 4 ++-- 4 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 63c4a9eaad..b8b9977a68 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -1,13 +1,13 @@ # Claims in the ADFA-7083 pages and how each was checked -`run` = executed by `test_examples.py` against a host build of the package (66 of 66 on plain 1.4.2, the package from the move to 1.4.2, 2026-11-13: 28 SQL examples, 19 of them run a second time, and 19 links; on 1.4.1 the `js_version` example and the kept result set example fail, as they should. Before that, 64 of 64 on 1.4.1 with the package's two patches, stack and garbage collection, which change none of the pages' claims, and on 1.4.1 without them, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`, with the package's patches 0001 and 0002 (sqliteai/sqlite-js #22 and #23). `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12). `TODO` = still open. +`run` = executed by `test_examples.py` against a host build of the package (66 of 66 on plain 1.4.2, the package from the move to 1.4.2, 2026-11-13: 28 SQL examples, 19 of them run a second time, and 19 links; on 1.4.1 the `js_version` example and the kept result set example fail, as they should. Before that, 64 of 64 on 1.4.1 with the package's two patches, stack and garbage collection, which change none of the pages' claims, and on 1.4.1 without them, 2026-11-12: 27 SQL examples, 18 of them run a second time in the same session, and 19 links; earlier, 1.3.7-p20261112, upstream main at c8ae85a, and 1.3.7-1 to 1.3.7-5, 1.3.7 with patches 0001 to 0008). `code` = read in the source: the patched 1.3.7 up to 1.3.7-5, then upstream main at c8ae85a, then 1.4.1, whose `src/` and `libs/` differ from c8ae85a only in `SQLITE_JS_VERSION`, with the package's patches 0001 and 0002 (sqliteai/sqlite-js #22 and #23), then 1.4.2 (tag 74557b2), which the package builds with no patches. `probe` = run by hand against the host build, in the accuracy review (2026-11-11), the review of PR 2241 (2026-11-12) or the move to 1.4.1 (2026-11-12), or the move to 1.4.2 and its review (2026-11-13 to 2026-11-15). `TODO` = still open. ## sqlite-js-top | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) moves to the 1.4.2 tag with no patches and no revision (2026-11-13), so its version is `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.1 gt 1.3.7-p20261112-0` is true; the examples pass on 1.4.1; host: `dpkg -s` prints a `Version:` line. TODO: the 1.4.1 package on stage, and `dpkg -s sqlite-js` on the phone | +| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) moves to the 1.4.2 tag with no patches and no revision (2026-11-13), so its version is `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.2 gt 1.4.1` is true; the examples pass on 1.4.2; host: `dpkg -s` prints a `Version:` line; phone (2026-11-13): sqlite-js installed from stage, and `dpkg-query -W` reports `1.4.2` | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | @@ -36,7 +36,7 @@ | `js_version(1)` is quickjs-ng | run (package AC2) | | `js_version()` gives `1.4.2` | run (the example's `-- 1.4.2`); `SQLITE_JS_VERSION` is "1.4.2" in the 1.4.2 tag's `src/sqlitejs.h` | | sqlite-js 1.4.0 includes fixes from App Dev For All; the eight differences from 1.3.7 listed | code: 1.4.0 contains c8ae85a (2026-11-12), which merges PRs #11 to #18, the changes of patches 0001 to 0008, and #19, and its CHANGELOG lists the same changes; 1.4.1's C code is c8ae85a's; package run-tests.sh: 98 of 98 on the 1.3.7-p20261112 host build, the same C code | -| sqlite-js 1.4.2 has more fixes, several sent or reported by App Dev For All; the five differences from 1.4.1 listed | code: upstream's 1.4.2 tag (74557b2) contains App Dev For All's sqliteai/sqlite-js PRs #22 (stack, with the maintainer's bd3d6c4 and 31ed714), #23 (garbage collection) and #24 (result sets an interrupt leaves open), whose patch ids match the package's former patches 0001 to 0004, and the maintainer's #26 for App Dev For All's issue #25 (window function crash) with the result set lifetime change; 1.4.2's CHANGELOG lists the same. run: the kept result set example (fails on 1.4.1, where `next()` returned `true` and `sqlite3` reported `unable to close due to unfinalized statements`). The window function crash was not reproduced here: the saved reproducer ran 20,000 turns without crashing on the 1.4.1 host build too, so that fix rests on the code and the CHANGELOG | +| sqlite-js 1.4.2 has more fixes, several sent or reported by App Dev For All; the five differences from 1.4.1 listed | code: upstream's 1.4.2 tag (74557b2) contains App Dev For All's sqliteai/sqlite-js PRs #22 (stack, with the maintainer's bd3d6c4 and 31ed714), #23 (garbage collection) and #24 (result sets an interrupt leaves open), whose patch ids match the package's former patches 0001 to 0004, and the maintainer's #26 for App Dev For All's issue #25 (window function crash) with the result set lifetime change; 1.4.2's CHANGELOG lists the same. run: the kept result set example (fails on 1.4.1, where `next()` returned `true` and `sqlite3` reported `unable to close due to unfinalized statements`). probe (review of 139870bad, 2026-11-15): a window function whose `final` allocates, read through `db.exec(...).next()` and dropped in a loop, crashes `sqlite3` on the 1.4.1 host build (segmentation fault within 60 turns) and passes on 1.4.2; with `toArray()` in place of `next()`, 1.4.1 passes too | | quickjs-ng is MIT-licensed; `LICENSE` and `LICENSE.quickjs` are in `$PREFIX/share/doc/sqlite-js/` | code: package build (`LICENSE.quickjs` collected from the sources); file list of the 1.3.7-1 aarch64 `.deb` checked | | Apache-2.0 attribution wording | TODO: legal wording check; upstream names no copyright holder | @@ -52,7 +52,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec`; a result set lasts until the call into JavaScript that created it returns, after which `next()`, `toArray()`, `get()` and `name()` throw the quoted message and one read to its end still reports no more rows (1.4.2) | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction. Probe on the 1.4.2 host build (2026-11-13): the same PRAGMA results; a result set kept in a global throws the quoted message from `next()`, `toArray()`, `get()` and `name()` in a later `js_eval` (`columnCount` still answers); one read to its end by `toArray()` gives `next()` false; an INSERT without `next()` still adds nothing; the kept result set example | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec`; a result set lasts until the call into JavaScript that created it returns, or the outer call when that call was made by SQL another call ran with `db.exec`, after which `next()`, `toArray()`, `get()` and `name()` throw the quoted message, or `interrupted` after Ctrl-C, and one whose `next()` returned `false` or whose `toArray()` ran still reports no more rows (1.4.2) | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction. Probe on the 1.4.2 host build (2026-11-13): the same PRAGMA results; a result set kept in a global throws the quoted message from `next()`, `toArray()`, `get()` and `name()` in a later `js_eval` (`columnCount` still answers); one read to its end by `toArray()` gives `next()` false; an INSERT without `next()` still adds nothing; the kept result set example; probe on the 1.4.2 host build (review of 139870bad, 2026-11-15): a scalar function called by SQL in a `js_eval`'s `db.exec` keeps the result set it made readable until the `js_eval` returns; a result set whose rows were all read but whose `next()` never returned `false` throws in a later call; after Ctrl-C in an interactive session a kept result set throws `interrupted` | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/README.md b/docs/docdb/ADFA-7083/README.md index 5b8dcc1a0d..30ea6692be 100644 --- a/docs/docdb/ADFA-7083/README.md +++ b/docs/docdb/ADFA-7083/README.md @@ -9,7 +9,7 @@ Four Help+Manual style pages for `documentation.db`, in the `i/` set, reached fr | JavaScript in sqlite-js | `i/sqlite-js-javascript.html` | | JavaScript basics | `i/javascript-intro.html` | -They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.4.1 and later: upstream's 1.4.0 release merged our fixes. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. +They document the terminal package ([ADFA-7081](https://appdevforall.atlassian.net/browse/ADFA-7081)), sqlite-js 1.4.2 and later: upstream's 1.4.0 and 1.4.2 releases merged our fixes. Apps built with Code on the Go ([ADFA-7082](https://appdevforall.atlassian.net/browse/ADFA-7082)) are not covered yet. - `pages/*.md`: the source, a small Markdown subset (see `build.py`). - `build.py OUT_DIR`: turns each page into the JSON facts that `helpmanual-topic.peb` renders. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 38e8486c68..7a37c82cca 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -200,7 +200,7 @@ sqlite-js 1.4.0 includes fixes that App Dev For All sent to the upstream project sqlite-js 1.4.2 has more fixes, several of them sent or reported by App Dev For All. Compared with 1.4.1: -- A result set from `db.exec` lasts only until the call into JavaScript that created it returns. Reading it in a later call throws an error, where before it went on with its rows. +- A result set from `db.exec` lasts only until the outermost call into JavaScript returns. Reading it in a later call throws an error, where before it went on with its rows. - Ctrl-C no longer leaves `sqlite3` failing every later query with `interrupted` when JavaScript kept a result set. - Code that calls itself without end fails instead of crashing `sqlite3` in more cases. If the stack has no limit, `js_set_max_stack_size` allows at most 1 MB. - JavaScript that runs out of memory while much of its memory is garbage no longer fails again on every try. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index 86e4a307e1..442ade498a 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -140,7 +140,7 @@ SELECT js_eval('(function () { -- not added: UNIQUE constraint failed: colors.name ``` -A result set lasts only until the call into JavaScript that created it returns: the `js_eval`, or the call of your function. Read what you need before then. In a later call, `next()`, `toArray()`, `get()` and `name()` throw `The result set was closed when the call into JavaScript that created it returned`. A result set that was read to its end still reports that it has no more rows. +A result set lasts only until the call into JavaScript that created it returns: the `js_eval`, or the call of your function from SQL. When that call was made inside another one, by SQL that the other call ran with `db.exec`, the result set lasts until the outer call returns. Read what you need before then. In a later call, `next()`, `toArray()`, `get()` and `name()` throw `The result set was closed when the call into JavaScript that created it returned`, or `interrupted` after Ctrl-C stopped the call. A result set whose `next()` returned `false`, or whose `toArray()` ran, still reports that it has no more rows. ```sql SELECT js_eval('globalThis.kept = db.exec("SELECT 1 UNION ALL SELECT 2"); kept.next(); kept.get(0)'); @@ -160,7 +160,7 @@ SELECT js_eval('(function () { - Variables that you create with `js_eval`, such as `globalThis.rate = 0.2`, are shared by `js_eval`, your scalar functions and your collations. Aggregate and window functions cannot see them: define what they need in their `init` code. - A name declared with `const`, `let` or `class` at the top level of `js_eval` code also lasts for the session, so running the same code again fails with `redeclaration of`. Put the code in braces, `{ ... }`, to keep its names to one run. `js_eval` still returns the value of the last expression. - Each running aggregate or window function has variables of its own, which its `init` code sets up. That is why two groups never mix their state. -- A variable can keep a result set from `db.exec`, but not keep it open: the result set closes when the call that created it returns, as described in Run SQL from JavaScript. +- A variable can keep a result set from `db.exec`, but not keep it open: the result set closes when the outermost call into JavaScript returns, as described in Run SQL from JavaScript. ## Limits From b92378a2e0b25556adc9634ab187f7620355a841 Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 15 Nov 2026 21:02:17 -0800 Subject: [PATCH 22/24] ADFA-7083: Record the 1.4.2-1 package in CLAIMS.md The package is now 1.4.2-1: revision 1 ships its new tests and README. dpkg sorts 1.4.2-1 above 1.4.2, so the pages' "1.4.2 and later" still holds; on the phone, dpkg-query reports 1.4.2-1 after installing from stage. No page changes. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index b8b9977a68..90706401b5 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -7,7 +7,7 @@ | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) moves to the 1.4.2 tag with no patches and no revision (2026-11-13), so its version is `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.2 gt 1.4.1` is true; the examples pass on 1.4.2; host: `dpkg -s` prints a `Version:` line; phone (2026-11-13): sqlite-js installed from stage, and `dpkg-query -W` reports `1.4.2` | +| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) builds the 1.4.2 tag with no patches; revision 1 (2026-11-15), for its new tests and README, makes its version `1.4.2-1`, which `dpkg --compare-versions` sorts above `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.2 gt 1.4.1` is true; the examples pass on 1.4.2; host: `dpkg -s` prints a `Version:` line; phone: sqlite-js installed from stage, and `dpkg-query -W` reports `1.4.2` (2026-11-13), then `1.4.2-1` (2026-11-16) | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion | From 152c348930c89416f5f837fa76004a7f0c4dccdf Mon Sep 17 00:00:00 2001 From: David Schachter Date: Sun, 15 Nov 2026 21:23:14 -0800 Subject: [PATCH 23/24] ADFA-7083: Document result sets that end after an error or a full read A result set read to its end still reports no more rows in a later call. A result set whose next() or toArray() threw also reports no more rows (next() false, toArray() []) with no error, so catch errors where they happen. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-functions.md | 2 +- docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 90706401b5..8e46c34f12 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -52,7 +52,7 @@ | `console.log` and `print` write to `sqlite3` standard output | run on phone: `console.log` and `print` both write to standard output | | SQL-to-JS and JS-to-SQL conversion tables, including `NaN` -> NULL, other objects -> NULL, BigInt wraps outside 64 bits | run (package AC6, and the BigInt wraparound example), code; probe: BLOB arrives as `[object ArrayBuffer]`; `NaN`, arrays, `Date`, functions give NULL; `Infinity` and `2**60` give REAL; `2n**70n` gives 0 | | 3.0 comes back as 3; INTEGER beyond 2^53 - 1 loses digits; BigInt workaround | run | -| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec`; a result set lasts until the call into JavaScript that created it returns, or the outer call when that call was made by SQL another call ran with `db.exec`, after which `next()`, `toArray()`, `get()` and `name()` throw the quoted message, or `interrupted` after Ctrl-C, and one whose `next()` returned `false` or whose `toArray()` ran still reports no more rows (1.4.2) | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction. Probe on the 1.4.2 host build (2026-11-13): the same PRAGMA results; a result set kept in a global throws the quoted message from `next()`, `toArray()`, `get()` and `name()` in a later `js_eval` (`columnCount` still answers); one read to its end by `toArray()` gives `next()` false; an INSERT without `next()` still adds nothing; the kept result set example; probe on the 1.4.2 host build (review of 139870bad, 2026-11-15): a scalar function called by SQL in a `js_eval`'s `db.exec` keeps the result set it made readable until the `js_eval` returns; a result set whose rows were all read but whose `next()` never returned `false` throws in a later call; after Ctrl-C in an interactive session a kept result set throws `interrupted` | +| `db.exec` result set: `next`, `get`, `name`, `columnCount`, `toArray` (remaining rows); the statement runs only at `next()`/`toArray()`, except some PRAGMAs, which take effect when `db.exec` prepares them; `foreign_keys` has no effect inside a transaction or savepoint; a run-time error makes `next()` and `toArray()` throw with SQLite's message (1.4.0; 1.3.7-3 to 1.3.7-5 as patch 0008); SQL errors are thrown by `db.exec`; a result set lasts until the call into JavaScript that created it returns, or the outer call when that call was made by SQL another call ran with `db.exec`, after which `next()`, `toArray()`, `get()` and `name()` throw the quoted message, or `interrupted` after Ctrl-C, and one whose `next()` returned `false` or whose `toArray()` ran still reports no more rows (1.4.2) | run (the notes example: an INSERT without `next()` adds nothing; the colors example: a failed UNIQUE INSERT is caught with SQLite's message, and it fails on 1.3.7-2, which returns `added`); probe on the 1.3.7-4 host build: a failed UNIQUE INSERT through `next()` fails the query with `UNIQUE constraint failed: u.x`, caught as `InternalError`; `toArray()` over a 3-row query whose second row overflows (`abs(-9223372036854775808)`) fails with `integer overflow`; `next()` is `false` at the end of the rows and after a successful INSERT; `db.exec("SELEC 1")` throws `near "SELEC": syntax error`. The same on the 1.4.1 host build. Before 0008, run-time errors were dropped: `next()` returned `false` and `toArray()` the rows before the error. Code: `js_rowset_next`/`js_rowset_to_array` throw unless `sqlite3_step` returns `SQLITE_ROW` or `SQLITE_DONE`; probe on 1.4.1 (2026-11-13, CodeRabbit): without `next()`, `db.exec("PRAGMA busy_timeout=5000")`, `foreign_keys=ON` and `cache_size=-4321` take effect, and `user_version=7` and `application_id=9` do not; with the C API on SQLite 3.49.1, `PRAGMA journal_mode=OFF` prepared but not stepped leaves the mode `delete`; `PRAGMA foreign_keys = ON` through `db.exec` leaves it 0 after `BEGIN` or `SAVEPOINT`, and sets it to 1 outside a transaction. Probe on the 1.4.2 host build (2026-11-13): the same PRAGMA results; a result set kept in a global throws the quoted message from `next()`, `toArray()`, `get()` and `name()` in a later `js_eval` (`columnCount` still answers); one read to its end by `toArray()` gives `next()` false; an INSERT without `next()` still adds nothing; the kept result set example; probe on the 1.4.2 host build (review of 139870bad, 2026-11-15): a scalar function called by SQL in a `js_eval`'s `db.exec` keeps the result set it made readable until the `js_eval` returns; a result set whose rows were all read but whose `next()` never returned `false` throws in a later call; after Ctrl-C in an interactive session a kept result set throws `interrupted`; probe on the 1.4.2 host build (2026-11-16, CodeRabbit): a result set read to its end by `next()` false or by `toArray()` gives `next()` false and `toArray()` `[]` in a later call, with no error; a failed UNIQUE INSERT (`UNIQUE constraint failed: u.x`) or an `abs(-9223372036854775808)` overflow caught from `next()` or `toArray()` leaves `next()` false and `toArray()` `[]` afterward, in the same call and in a later one, with no error | | Parameters convert like results | run (package AC6) | | Variable sharing: `js_eval`, scalar functions, collations share; each aggregate has its own; aggregate and window functions cannot see `js_eval` variables, and can define what they need in `init` | run (AC3); probe on 1.3.7-4 and 1.4.1: after `globalThis.rate = 2`, an aggregate's and a window function's `step` fail with `rate is not defined`; `const rate = 2` in `init` gives 6 for rows 1 and 2 | | Top-level `const`, `let` and `class` names last for the session, so running the code again fails with `redeclaration of`; braces keep them to one run and still return the last expression | run: `test_examples.py` runs each example that creates nothing twice; before the braces it failed with `redeclaration of 'name'`, `'total'`, `'half'`, `'result'` and `'Point'` | diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md index 7a37c82cca..39e3073bc9 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-functions.md @@ -200,7 +200,7 @@ sqlite-js 1.4.0 includes fixes that App Dev For All sent to the upstream project sqlite-js 1.4.2 has more fixes, several of them sent or reported by App Dev For All. Compared with 1.4.1: -- A result set from `db.exec` lasts only until the outermost call into JavaScript returns. Reading it in a later call throws an error, where before it went on with its rows. +- A result set from `db.exec` lasts only until the outermost call into JavaScript returns. Reading it in a later call throws an error, where before it went on with its rows. A result set already read to its end still reports that it has no more rows. - Ctrl-C no longer leaves `sqlite3` failing every later query with `interrupted` when JavaScript kept a result set. - Code that calls itself without end fails instead of crashing `sqlite3` in more cases. If the stack has no limit, `js_set_max_stack_size` allows at most 1 MB. - JavaScript that runs out of memory while much of its memory is garbage no longer fails again on every try. diff --git a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md index 442ade498a..05c3a261d4 100644 --- a/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md +++ b/docs/docdb/ADFA-7083/pages/sqlite-js-javascript.md @@ -140,7 +140,7 @@ SELECT js_eval('(function () { -- not added: UNIQUE constraint failed: colors.name ``` -A result set lasts only until the call into JavaScript that created it returns: the `js_eval`, or the call of your function from SQL. When that call was made inside another one, by SQL that the other call ran with `db.exec`, the result set lasts until the outer call returns. Read what you need before then. In a later call, `next()`, `toArray()`, `get()` and `name()` throw `The result set was closed when the call into JavaScript that created it returned`, or `interrupted` after Ctrl-C stopped the call. A result set whose `next()` returned `false`, or whose `toArray()` ran, still reports that it has no more rows. +A result set lasts only until the call into JavaScript that created it returns: the `js_eval`, or the call of your function from SQL. When that call was made inside another one, by SQL that the other call ran with `db.exec`, the result set lasts until the outer call returns. Read what you need before then. In a later call, `next()`, `toArray()`, `get()` and `name()` throw `The result set was closed when the call into JavaScript that created it returned`, or `interrupted` after Ctrl-C stopped the call. A result set whose `next()` returned `false`, or whose `toArray()` ran, still reports that it has no more rows. A result set whose `next()` or `toArray()` threw an error, such as a failed constraint, also reports no more rows afterward: `next()` returns `false` and `toArray()` returns `[]`, with no error. Catch the error where it happens, and do not read that as success. ```sql SELECT js_eval('globalThis.kept = db.exec("SELECT 1 UNION ALL SELECT 2"); kept.next(); kept.get(0)'); From a414667342a9b18bee9768a4b36d3e180be3eeee Mon Sep 17 00:00:00 2001 From: David Schachter Date: Mon, 16 Nov 2026 10:31:52 -0800 Subject: [PATCH 24/24] ADFA-7083: Record the 1.4.2-2 package in CLAIMS.md terminal-packages #6 revision 2 (the review's test fixes) is on stage, and the phone reports 1.4.2-2 after the upgrade from 1.4.2-1. The pages say 1.4.2 and later, so they are unchanged. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vh2DAXGyHbMzjSsi2zbrkw --- docs/docdb/ADFA-7083/CLAIMS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/docdb/ADFA-7083/CLAIMS.md b/docs/docdb/ADFA-7083/CLAIMS.md index 8e46c34f12..fbd8878acc 100644 --- a/docs/docdb/ADFA-7083/CLAIMS.md +++ b/docs/docdb/ADFA-7083/CLAIMS.md @@ -7,7 +7,7 @@ | Claim | Check | |-------|-------| | `apt install sqlite sqlite-js` installs both; the `sqlite` package provides `sqlite3` | phone: `sqlite3` is provided by package `sqlite` (dpkg -S), 3.51.2. `apt install sqlite sqlite-js` from the stage repo (https://stage.appdevforall.org/apt/termux-main/, same signing key as production) installed sqlite-js 1.3.7-1 on aarch64, which loaded and passed run-tests.sh 77 of 77. 1.3.7-3 and 1.3.7-4 from stage passed 91 of 91 on aarch64, and the arm packages passed 91 of 91 as a 32-bit process on the same phone, which runs 32-bit code. Production gets it when the package is published there | -| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) builds the 1.4.2 tag with no patches; revision 1 (2026-11-15), for its new tests and README, makes its version `1.4.2-1`, which `dpkg --compare-versions` sorts above `1.4.2`; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.2 gt 1.4.1` is true; the examples pass on 1.4.2; host: `dpkg -s` prints a `Version:` line; phone: sqlite-js installed from stage, and `dpkg-query -W` reports `1.4.2` (2026-11-13), then `1.4.2-1` (2026-11-16) | +| These pages describe 1.4.2 and later; `dpkg -s sqlite-js` shows the installed version; `apt install` brings an older one up to date | code: the package (terminal-packages #6) builds the 1.4.2 tag with no patches; revision 1 (2026-11-15), for its new tests and README, makes its version `1.4.2-1`, which `dpkg --compare-versions` sorts above `1.4.2`, and revision 2 (2026-11-16), for the review's test fixes, makes it `1.4.2-2`, its library built from the same source; before, it built the 1.4.1 tag with no revision, so its version was `1.4.1` (`termux_step_start_build.sh` adds `-REVISION` only for a revision other than 0 or a version with a hyphen); 1.3.7-3 to 1.3.7-5 lack upstream's ef06558, so a lowered memory limit behaves differently there (package AC9 fails on 1.3.7-5); `dpkg --compare-versions 1.4.2 gt 1.4.1` is true; the examples pass on 1.4.2; host: `dpkg -s` prints a `Version:` line; phone: sqlite-js installed from stage, and `dpkg-query -W` reports `1.4.2` (2026-11-13), then `1.4.2-1` and `1.4.2-2` (2026-11-16) | | `.shell echo $PREFIX/lib/sqlite-js` in `sqlite3` prints the path | probe on host `sqlite3` 3.46.1: `.shell echo $HOME` prints the expanded path. TODO: on the phone | | `.load sqlite-js` finds `$PREFIX/lib/sqlite-js.so`; `.load` does not expand `$PREFIX` | run on phone: `.load sqlite-js`, `.load sqlite-js.so` and the full path without `.so` work; `.load $PREFIX/lib/sqlite-js` fails with `dlopen failed: library "$PREFIX/lib/sqlite-js.so" not found` | | `js_create_scalar` prints 0; slugify and titles examples | run. The opening line no longer claims JavaScript is quicker than Java or Kotlin, which was opinion |