From 49f22f6a31332656b545f3dcf90f9a1b4e9a7420 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 10:33:36 +0200 Subject: [PATCH 01/16] [maven-release-plugin] prepare for next development iteration --- pom.xml | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/pom.xml b/pom.xml index 83dcd8d4c..f98dec3e4 100644 --- a/pom.xml +++ b/pom.xml @@ -3,7 +3,7 @@ com.atomgraph linkeddatahub - 6.0.0 + 6.0.1-SNAPSHOT ${packaging.type} AtomGraph LinkedDataHub @@ -46,7 +46,7 @@ https://github.com/AtomGraph/LinkedDataHub scm:git:git://github.com/AtomGraph/LinkedDataHub.git scm:git:git@github.com:AtomGraph/LinkedDataHub.git - linkeddatahub-6.0.0 + linkeddatahub-5.5.4 From a56deca9604aae2b5d263757a05f5d08653c18da Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 10:33:39 +0200 Subject: [PATCH 02/16] Set the CLI version to 6.0.1-SNAPSHOT --- cli/pom.xml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/cli/pom.xml b/cli/pom.xml index ac61d1e64..4b7b1a567 100644 --- a/cli/pom.xml +++ b/cli/pom.xml @@ -4,7 +4,7 @@ com.atomgraph linkeddatahub-cli - 6.0.0 + 6.0.1-SNAPSHOT jar LinkedDataHub CLI From 94494701e8f2ce8fac5367361b07a4dae10d69e3 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 12:29:38 +0200 Subject: [PATCH 03/16] Proxied XHTML content resolves path-absolute references against the document they came from A reference like /uploads/{sha1} carries no authority, so the browser fills in the origin it is displayed on - the proxying dataspace - and the authored media 404s there. The proxy-mode rewrite in ldh:XHTMLContent excluded exactly those references while resolving the relative ones beside them, so a docs page proxied into another dataspace rendered its prose and its links correctly and its images broken. Only fragments stay unresolved now, because they address the rendering rather than the source. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 4 ++++ .../atomgraph/linkeddatahub/xsl/imports/default.xsl | 12 ++++++++---- 2 files changed, 12 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e9b51971e..026ac6d6b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,7 @@ +## [Unreleased] +### Fixed +- Proxied XHTML content left path-absolute `@href`/`@src` references unresolved, so authored media (`/uploads/{sha1}`) resolved against the proxying dataspace's origin and 404'd. The proxy-mode rewrite in `ldh:XHTMLContent` now resolves them against the content's base URI alongside the relative ones; only fragments stay untouched, because they address the rendering rather than the source + ## [6.0.0] - 2026-09-29 LinkedDataHub has a new interface. The app shell, content blocks, action bar, breadcrumbs, mode lists, type badges, property lists, tables, pager, modals and forms are drawn against a design system vendored into the platform — its tokens, components and typefaces ship with LDH, and `ldh.css` is the single app layer loaded over them, the one file a dataspace stylesheet has to reckon with. IXSL templates drive the dropdowns and modals, the `msi` font draws the icons and the RDFa editor edits `rdf:XMLLiteral`, so jQuery, `bootstrap.js`, WYMEditor and the sprite sheet are gone. diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/imports/default.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/imports/default.xsl index 0e59e6ff9..673a8d2a8 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/imports/default.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/imports/default.xsl @@ -1471,13 +1471,17 @@ exclude-result-prefixes="#all" - - + + - - + + From a4b5a550612aaa45656eba63757a103d0e46ba14 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 13:39:04 +0200 Subject: [PATCH 04/16] The documentation links point at docs.linkeddatahub.com The docs under atomgraph.github.io/LinkedDataHub/ are a stale static mirror; the live documentation is the docs.linkeddatahub.com dataspace, whose paths are identical, so the rewrite is a straight prefix swap. The Maven project URL points at linkeddatahub.com. Co-Authored-By: Claude Opus 5 (1M context) --- README.md | 14 +++++++------- pom.xml | 2 +- src/main/webapp/WEB-INF/web.xml | 4 ++-- .../atomgraph/linkeddatahub/xsl/client/modal.xsl | 6 +++--- .../com/atomgraph/linkeddatahub/xsl/layout.xsl | 2 +- 5 files changed, 14 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 3bd374828..228587c12 100644 --- a/README.md +++ b/README.md @@ -16,7 +16,7 @@ What makes LinkedDataHub unique is its completely _data-driven architecture_: ap XHTML documents can be edited in-place using a built-in RDFa-aware rich text editor — annotations link selected text directly to Knowledge Graph terms, embedding machine-readable RDF statements in the markup without leaving the page. -**Follow the [Get started](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/get-started/) guide to LinkedDataHub.** The setup and basic configuration sections are provided below and should get you running. +**Follow the [Get started](https://docs.linkeddatahub.com/get-started/) guide to LinkedDataHub.** The setup and basic configuration sections are provided below and should get you running. **LinkedDataHub is also available as a free AWS Marketplace product!** AWS Marketplace It takes a few clicks and filling out a form to install the product into your own AWS account. No manual setup or configuration necessary! @@ -247,7 +247,7 @@ _:warning: Do not use blank nodes to identify applications or services. We recom
Port number of the mail server
-The options are described in more detail in the [configuration documentation](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/reference/configuration/). +The options are described in more detail in the [configuration documentation](https://docs.linkeddatahub.com/reference/configuration/). ## Reset @@ -260,13 +260,13 @@ The options are described in more detail in the [configuration documentation](ht _:warning: This will **remove the persisted data and files** as well as Docker volumes._ -## [Documentation](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/) +## [Documentation](https://docs.linkeddatahub.com/) -* [Get started](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/get-started/) -* [Reference](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/reference/) -* [User guide](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/user-guide/) +* [Get started](https://docs.linkeddatahub.com/get-started/) +* [Reference](https://docs.linkeddatahub.com/reference/) +* [User guide](https://docs.linkeddatahub.com/user-guide/) -## [Command line interface](https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/reference/command-line-interface/) +## [Command line interface](https://docs.linkeddatahub.com/reference/command-line-interface/) `ldh` wraps the HTTP API into a single executable with convenient parameters. It can be used for testing, automation, scheduled execution and such. It is usually much quicker to perform actions using the CLI rather than the user interface, as well as easier to reproduce. diff --git a/pom.xml b/pom.xml index f98dec3e4..f6a39f4fe 100644 --- a/pom.xml +++ b/pom.xml @@ -8,7 +8,7 @@ AtomGraph LinkedDataHub The low-code Knowledge Graph application platform - https://atomgraph.github.io/LinkedDataHub/ + https://linkeddatahub.com/ 2014 diff --git a/src/main/webapp/WEB-INF/web.xml b/src/main/webapp/WEB-INF/web.xml index 67fe1ada8..252bc4e74 100644 --- a/src/main/webapp/WEB-INF/web.xml +++ b/src/main/webapp/WEB-INF/web.xml @@ -259,7 +259,7 @@ DELETE WHERE A PKCS12 client certificate for your WebID is attached to this message. Download the file, install it into your web browser and restart the browser afterwards. For use with the LinkedDataHub CLI and/or curl, convert the PKCS12 file to PEM using openssl: https://docs.openssl.org/master/man1/openssl-pkcs12/ -Get started with your LinkedDataHub application by following this guide: https://atomgraph.github.io/LinkedDataHub/linkeddatahub/docs/get-started/ +Get started with your LinkedDataHub application by following this guide: https://docs.linkeddatahub.com/get-started/ Application's base URI: %s Your WebID profile: %s @@ -273,7 +273,7 @@ support@atomgraph.com]]> https://w3id.org/atomgraph/linkeddatahub/config#oAuthSignUpEMailText
- - + +
@@ -198,7 +198,7 @@ LIMIT 10 - + . diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/layout.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/layout.xsl index a167b7010..6658451e6 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/layout.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/layout.xsl @@ -944,7 +944,7 @@ WHERE

- +
From b87fc9be5ffa04610377dfb825c12693d735d2d0 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 14:03:37 +0200 Subject: [PATCH 05/16] A 429 retry re-sends the request its step made, not the first request of the chain: ldh:retry-request re-fired $context('request') whatever response key it was retrying, so a chart whose results POST was rate-limited got the RDF/XML of its query's document back as the results, and drew "Table has no columns" with one row per resource in that document. The request is looked up under the key paired with the response (chart-results-request for chart-results-response, metadata-request for metadata-response, ...); a step that threads no paired key, as ldh:view-results-thunk reuses 'request', still falls back to it. Co-Authored-By: Claude Fable 5.1 --- .../com/atomgraph/linkeddatahub/xsl/client/functions.xsl | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl index c87bd5ad5..99170968a 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl @@ -662,7 +662,12 @@ exclude-result-prefixes="#all" - + + + --- .../com/atomgraph/linkeddatahub/xsl/client/functions.xsl | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl index 99170968a..ad9198462 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/functions.xsl @@ -321,9 +321,10 @@ exclude-result-prefixes="#all" - - + + From 9f9e1ea9d67f697636cc639ad6355d5492b34ff2 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 16:26:46 +0200 Subject: [PATCH 07/16] The chart block is asserted to draw its own results after its results request is refused with 429 once, and to draw in the design tokens where the browser does not enumerate custom properties: rate-limit.spec checks that the retry re-sends the query POST and the drawing is labelled by kind, tokens.spec hides custom properties from getComputedStyle's enumeration the way Chromium 131 does and checks the series is the --ldh-blue-500 colour. Against a client SEF without b87fc9be5 and 0ffe1a46f each fails on its own defect ("Table has no columns", "Cannot read properties of null (reading 'color')"), as owner and anonymous; with them the chart specs pass 30/30 over three repeats. Co-Authored-By: Claude Fable 5.1 --- .../document/blocks/chart/rate-limit.spec.mjs | 60 ++++++++++++++++++ .../document/blocks/chart/tokens.spec.mjs | 63 +++++++++++++++++++ 2 files changed, 123 insertions(+) create mode 100644 tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs create mode 100644 tests/ui/specs/document/blocks/chart/tokens.spec.mjs diff --git a/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs b/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs new file mode 100644 index 000000000..97af321dc --- /dev/null +++ b/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs @@ -0,0 +1,60 @@ +// The chart block, when its results request is rate-limited. +// +// A page of content blocks fires more requests at once than nginx's `linked_data` zone lets through +// (15 r/s, burst 30), so on a busy page some of them come back 429 and the client retries them after +// Retry-After. The retry re-sent `$context('request')` whatever step it was retrying - and for a +// chart that is the GET of the query's DOCUMENT, the first request of its chain, not the POST of +// the query. The chart was handed the document's RDF/XML as its "results", mapped `?kind` and +// `?items` as property URIs onto it, and drew "Table has no columns." over one row per resource in +// the document. Which chart it hit depended on which request the limiter refused, so on the +// Northwind demo it was a different chart on every reload, and reloading "fixed" it. +// +// The limiter is not something a spec can trigger on demand, so the 429 is fulfilled by a route - +// once, for the results POST alone - and the assertion is on both ends of the retry: the request +// that went out again, and the picture it drew. +import { test, expect } from '../../../../lib/console.mjs'; +import { goto } from '../../../../lib/settle.mjs'; +import { fixtures, kinds } from '../../../../lib/fixtures.mjs'; +import { canvas, chartBlock, drawing } from '../../../../lib/chart.mjs'; + +// The results request: the fixture chart's query, POSTed to the endpoint. Its body is the query +// text, and the aggregate is what tells it from the view's query over the same items. +const isResults = request => request.method() === 'POST' + && /COUNT\(\?item\) AS \?items/.test(request.postData() ?? ''); + +test('a results request refused with 429 is retried, and the chart draws its own results', async ({ page, allowNoise }) => { + allowNoise.push({ pattern: /^HTTP 429: /, reason: 'this spec refuses the results request once on purpose' }); + allowNoise.push({ + pattern: /console\.error: Failed to load resource.*429/i, + reason: 'the browser logs the injected 429 the route fulfils', + }); + + const results = []; + await page.route('**/*', route => { + const request = route.request(); + if (!isResults(request)) return route.fallback(); + + results.push(request); + // No Retry-After, as nginx sends none: the client falls back to its default wait. + return results.length === 1 + ? route.fulfill({ status: 429, contentType: 'text/html', body: '429 Too Many Requests' }) + : route.fallback(); + }); + + await goto(page, fixtures.container); + + const block = chartBlock(page); + await expect(drawing(block), 'the chart draws after the retry').toBeVisible({ timeout: 30_000 }); + // What went out again is the query, not the document it is stored in. Before the fix the count + // stayed at 1: the retry was a GET, which this route never sees. + expect(results, 'the refused results POST is the request that is retried').toHaveLength(2); + expect(results[1].postData()).toBe(results[0].postData()); + + // And the drawing is of the query's rows: a label per kind. A chart drawn from the document's + // RDF/XML has no column the category maps onto, so it has no kind labels either. + const labels = await canvas(block).locator('svg text').allTextContents(); + for (const kind of kinds) { + expect(labels, `the ${kind} bar is labelled`).toContain(kind); + } + await expect(canvas(block), 'the chart matched nothing').not.toContainText('Table has no columns'); +}); diff --git a/tests/ui/specs/document/blocks/chart/tokens.spec.mjs b/tests/ui/specs/document/blocks/chart/tokens.spec.mjs new file mode 100644 index 000000000..a950b9fa8 --- /dev/null +++ b/tests/ui/specs/document/blocks/chart/tokens.spec.mjs @@ -0,0 +1,63 @@ +// The chart block, drawn in the design system's colours. +// +// Google Charts takes concrete colour strings, not `var()` references, so ac:draw-chart resolves the +// design tokens with ldh:css-token() at draw time - the series palette, the axis text and titles, +// the gridlines and the baseline. It used to look each token up in the map ixsl:style() returns, +// which Saxon-JS builds by ENUMERATING getComputedStyle, and Chromium 131 - still what Playwright +// bundled into its older releases - enumerates no custom properties at all. Every token came back +// empty, and Google Charts failed on the empty colours before drawing anything: "Cannot read +// properties of null (reading 'color')", on every chart on the page. +// +// This suite's own Chromium does enumerate them, so the spec takes the enumeration away the way +// Chromium 131 did - the properties are still there by name, just not listed - and asserts that +// the bars are the token's colour rather than merely that something drew. +import { test, expect } from '../../../../lib/console.mjs'; +import { goto } from '../../../../lib/settle.mjs'; +import { fixtures } from '../../../../lib/fixtures.mjs'; +import { canvas, chartBlock, drawing } from '../../../../lib/chart.mjs'; + +// The first colour of the palette ac:draw-chart hands the chart, so the one the single series is +// drawn in. +const SERIES_TOKEN = '--ldh-blue-500'; + +test('draws in the design tokens where the browser does not enumerate custom properties', async ({ page }) => { + await page.addInitScript(() => { + const computed = window.getComputedStyle; + window.getComputedStyle = function (...args) { + const style = computed.apply(this, args); + const listed = Array.from(style).filter(name => !name.startsWith('--')); + return new Proxy(style, { + get(target, property) { + if (property === 'length') return listed.length; + if (property === 'item') return index => listed[index] ?? ''; + if (property === Symbol.iterator) return listed[Symbol.iterator].bind(listed); + if (typeof property === 'string' && /^\d+$/.test(property)) return listed[property]; + // Bound to the declaration itself: its methods throw "Illegal invocation" on a proxy. + const value = Reflect.get(target, property, target); + return typeof value === 'function' ? value.bind(target) : value; + }, + }); + }; + }); + + await goto(page, fixtures.container); + + const block = chartBlock(page); + await expect(drawing(block), 'the chart draws').toBeVisible({ timeout: 30_000 }); + + // Both sides normalised by the browser, since the token may be declared in any colour syntax and + // Google Charts writes the fill as hex. + const { token, fills } = await canvas(block).evaluate((node, name) => { + const probe = document.createElement('span'); + document.body.append(probe); + const normalise = colour => { probe.style.color = ''; probe.style.color = colour; return getComputedStyle(probe).color; }; + const token = getComputedStyle(document.documentElement).getPropertyValue(name).trim(); + const fills = [...node.querySelectorAll('svg rect[fill]')].map(rect => normalise(rect.getAttribute('fill'))); + const result = { token: token && normalise(token), fills }; + probe.remove(); + return result; + }, SERIES_TOKEN); + + expect(token, `${SERIES_TOKEN} resolves on this page`).toBeTruthy(); + expect(fills, `the series is drawn in ${SERIES_TOKEN}`).toContain(token); +}); From 7a774fcb6e2032e3ff5f856698d8f962238f449a Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 16:55:19 +0200 Subject: [PATCH 08/16] A load suite, tests/load, bursts the running stack with renders and asserts it still answers: every server-side HTML render calls back into the platform through the proxy for its /sparql and /ns labels, each callback needs a request thread of its own and holds a pooled connection while it waits, so once renders outnumber the connector's threads they wait on each other until reads time out. linkeddatahub.com wedged that way on 2026-09-29 under a scanner probing non-existent paths, every 403 and 404 page being a full render. The stack for it shrinks the connector to 16 threads (HTTP_MAX_THREADS, a new entrypoint setting applied by platform/server.xsl after letsencrypt-tomcat.xsl has written server.xml, which sizes only the HTTPS connector) and the client pool to 4 per route, production's ratio at a size a runner can saturate, and pushes the socket timeout to ten minutes so a timeout cannot pass for a recovery. It runs apart from the HTTP suite, since a failed burst leaves the platform wedged for whatever follows. On 6.0.0 it fails; a pool-wait timeout alone shortens the outage to 113 s and it still fails. Co-Authored-By: Claude Fable 5.1 --- .github/workflows/load-tests.yml | 55 +++++++++++++++ .gitignore | 1 + Dockerfile | 4 ++ make/config.mk | 2 +- make/local.mk | 7 ++ platform/entrypoint.sh | 8 +++ platform/server.xsl | 33 +++++++++ tests/load/docker-compose.load-tests.yml | 22 ++++++ tests/load/render-burst-no-deadlock.sh | 66 ++++++++++++++++++ tests/load/run.sh | 88 ++++++++++++++++++++++++ 10 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 .github/workflows/load-tests.yml create mode 100644 platform/server.xsl create mode 100644 tests/load/docker-compose.load-tests.yml create mode 100755 tests/load/render-burst-no-deadlock.sh create mode 100755 tests/load/run.sh diff --git a/.github/workflows/load-tests.yml b/.github/workflows/load-tests.yml new file mode 100644 index 000000000..92756ffe6 --- /dev/null +++ b/.github/workflows/load-tests.yml @@ -0,0 +1,55 @@ +name: Load-tests + +on: push + +jobs: + load-tests: + name: Build Docker image and run the load test suite against a small stack + runs-on: ubuntu-latest + steps: + - name: Checkout code + uses: actions/checkout@v7 + - name: Add bin/ and its subdirectories to PATH + run: | + find "$GITHUB_WORKSPACE/bin" -type d >> "$GITHUB_PATH" + - name: Generating server certificate + run: | + server-cert-gen.sh .env nginx ssl + working-directory: tests/http + - name: Writing secrets to files + run: | + mkdir -p ./secrets + printf "%s" "${{ secrets.HTTP_TEST_OWNER_CERT_PASSWORD }}" > ./secrets/owner_cert_password.txt + printf "%s" "${{ secrets.HTTP_TEST_SECRETARY_CERT_PASSWORD }}" > ./secrets/secretary_cert_password.txt + printf "%s" "${{ secrets.HTTP_TEST_SECRETARY_CERT_PASSWORD }}" > ./secrets/client_truststore_password.txt + shell: bash + - name: Build Docker image & Run Docker containers + # the HTTP suite's fixtures, with the connector and client pool shrunk by the load override + run: docker compose -f docker-compose.yml -f ./tests/http/docker-compose.http-tests.yml -f ./tests/load/docker-compose.load-tests.yml --env-file ./tests/http/.env up --build -d + - name: Wait for the server to start... + run: while ! (status=$(curl -k -s -w "%{http_code}\n" https://localhost:4443 -o /dev/null) && echo "$status" && echo "$status" | grep "403") ; do sleep 1 ; done # wait for the webapp to start (returns 403 by default) + - name: Run load test scripts + run: ./run.sh + shell: bash + working-directory: tests/load + - name: Generate test summary + if: always() + run: python3 scripts/generate_test_summary.py tests/load/out tests/load/out/report.md + - name: Write job summary + if: always() && hashFiles('tests/load/out/report.md') != '' + run: cat tests/load/out/report.md >> "$GITHUB_STEP_SUMMARY" + - name: Collect container logs on failure + if: failure() + run: | + mkdir -p tests/load/out/containers + timeout 300 docker compose --env-file ./tests/http/.env logs --no-color > tests/load/out/containers/compose.log 2>&1 || true + timeout 300 docker compose --env-file ./tests/http/.env exec -T linkeddatahub sh -c 'for f in /usr/local/tomcat/logs/*; do echo "=== $f ==="; cat "$f"; done' > tests/load/out/containers/tomcat.log 2>&1 || true + - name: Upload test results and container logs + if: always() && hashFiles('tests/load/out/**') != '' + uses: actions/upload-artifact@v4 + with: + name: load-test-results + path: tests/load/out/ + if-no-files-found: ignore + - name: Stop Docker containers and remove volumes + run: docker compose --env-file ./tests/http/.env down -v diff --git a/.gitignore b/.gitignore index 3259436ff..ee91a6fa0 100644 --- a/.gitignore +++ b/.gitignore @@ -16,6 +16,7 @@ /tests/http/datasets /tests/http/uploads /tests/http/out +/tests/load/out /fuseki .claude/scheduled_tasks.lock /cli/target diff --git a/Dockerfile b/Dockerfile index 0c9ed5ad6..555d66fd4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -227,6 +227,10 @@ COPY platform/context.xsl /var/linkeddatahub/xsl/context.xsl COPY platform/web.xsl /var/linkeddatahub/xsl/web.xsl +# add XSLT stylesheet that makes changes to server.xml, after letsencrypt-tomcat.xsl has generated it + +COPY platform/server.xsl /var/linkeddatahub/xsl/server.xsl + # copy entrypoint COPY platform/entrypoint.sh entrypoint.sh diff --git a/make/config.mk b/make/config.mk index 1583dc9d5..3875143b9 100644 --- a/make/config.mk +++ b/make/config.mk @@ -4,7 +4,7 @@ # has no app to install and compiles its client stylesheet from source rather than from a # published image. Everything it adds lives in make/local.mk. -LOCAL_TARGETS := sef release cli cli-version tests ui-tests ui-tests-install +LOCAL_TARGETS := sef release cli cli-version tests ui-tests ui-tests-install load-tests # only the deployment configuration is RDF worth parsing here; the rest of the tree is source VALIDATE_PATHS := config datasets diff --git a/make/local.mk b/make/local.mk index d03228d46..54491bfab 100644 --- a/make/local.mk +++ b/make/local.mk @@ -46,3 +46,10 @@ ui-tests-install: # been published with `make sef` first - the preflight says so if it has not. ui-tests: cli cd tests/ui && PATH="$(CURDIR)/cli/bin:$$PATH" npx playwright test + +# Burst the running stack and assert it still answers. Needs the stack brought up with +# tests/load/docker-compose.load-tests.yml on top of the HTTP suite's, which shrinks the connector +# and the client pool so a burst the runner can produce is enough. Kept apart from `tests`: a failing +# run leaves the platform wedged, and every test after it would fail for the wrong reason. +load-tests: + cd tests/load && ./run.sh diff --git a/platform/entrypoint.sh b/platform/entrypoint.sh index 7b2af113b..ba224b5e0 100755 --- a/platform/entrypoint.sh +++ b/platform/entrypoint.sh @@ -92,6 +92,14 @@ transform="xsltproc \ eval "$transform" +# the proxied HTTP connector's thread count: letsencrypt-tomcat.xsl sizes only the HTTPS connector, so +# this one keeps Tomcat's default of 200 unless set. Beyond capacity, the count bounds how many renders +# can wait on each other, since every server-side render calls back into this connector through the +# proxy; tests/load runs with a small value to reproduce that. +if [ -n "$HTTP_MAX_THREADS" ]; then + xsltproc --output conf/server.xml --stringparam Connector.maxThreads.http "$HTTP_MAX_THREADS" /var/linkeddatahub/xsl/server.xsl conf/server.xml +fi + ### PLATFORM ### # check mandatory environmental variables (which are used in conf/ROOT.xml) diff --git a/platform/server.xsl b/platform/server.xsl new file mode 100644 index 000000000..1a3000c8a --- /dev/null +++ b/platform/server.xsl @@ -0,0 +1,33 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/load/docker-compose.load-tests.yml b/tests/load/docker-compose.load-tests.yml new file mode 100644 index 000000000..3833f0973 --- /dev/null +++ b/tests/load/docker-compose.load-tests.yml @@ -0,0 +1,22 @@ +# On top of docker-compose.yml and tests/http/docker-compose.http-tests.yml: +# +# docker compose -f docker-compose.yml -f tests/http/docker-compose.http-tests.yml \ +# -f tests/load/docker-compose.load-tests.yml --env-file tests/http/.env up --build -d +# +# A stack sized so that a burst the test runner can produce reproduces what only real traffic +# reaches on a production connector of 200 threads. The ratio is what matters and it mirrors +# production: more request threads than outbound connections per route, so a burst of renders +# first exhausts the client pool and then the connector, in that order. +services: + linkeddatahub: + environment: + - HTTP_MAX_THREADS=16 # Tomcat's connector behind the proxy; default 200 + - MAX_CONN_PER_ROUTE=4 # the platform's outbound client pool, per host; production had 40 + - MAX_TOTAL_CONN=8 + # far past any bound the tests use: a wedge that resolves only because the platform's reads + # time out is still the wedge, and the dev default of two minutes would hide it behind the probe + - CLIENT_SOCKET_TIMEOUT=600000 + fuseki: + # the HTTP suite publishes the triplestore for its fixtures; this suite only talks to the front + # door, and a developer's own stack may already hold the port + ports: !override [] diff --git a/tests/load/render-burst-no-deadlock.sh b/tests/load/render-burst-no-deadlock.sh new file mode 100755 index 000000000..d7206fb2b --- /dev/null +++ b/tests/load/render-burst-no-deadlock.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +set -euo pipefail + +# Test: a burst of renders does not deadlock the platform against itself. +# +# Every server-side HTML render calls back into the platform over HTTP: layout.xsl POSTs to the +# dataspace's /sparql and /ns for the labels it puts on the page (ldh:send-request), and in a +# deployment those calls go out through the proxy - nginx, Varnish, and back into this same Tomcat. +# Each callback needs a free request thread to be answered and holds one of the client pool's +# connections while it waits. So once renders outnumber the connector's threads, every thread is a +# render waiting for a callback that no thread is free to serve, and the rest queue on the pool +# behind them: nothing completes until reads time out, and with a steady trickle of new requests +# not even then. linkeddatahub.com wedged exactly so on 2026-09-29 - a scanner probed paths that +# do not exist across the dataspace origins, every 403 and 404 page is a full render, and the 200 +# threads filled within a minute. The pages were error pages, the callbacks were not. +# +# The stack runs with a 16-thread connector and a 4-per-route client pool +# (docker-compose.load-tests.yml), production's ratio at a size the runner can saturate: a burst of +# three times the connector is enough. The requests are anonymous GETs of documents that do not +# exist, which is what the scanner sent. The bound is the assertion: once the burst has been +# answered or given up on, a plain request must get an HTTP status within 30 s. curl's 000 and the +# proxy's 502/503/504 are the deadlock, whatever the platform would eventually have answered. +# +# Measured on 6.0.0 with this stack: without a fix the platform answers nothing for as long as its +# reads take to time out (the override sets that to ten minutes). A pool-wait timeout +# (CONNECTION_REQUEST_TIMEOUT=10000) is a mitigation, not a fix - the renders queued on the pool +# fail in ten-second waves and the platform answered again 113 s after the burst began, which this +# bound is right to reject. It passes once a render no longer needs a request thread of its own +# to be answered: the callbacks answered in-process, or through a client of their own that cannot +# take the connector down with it. + +burst=$(( HTTP_MAX_THREADS * 3 )) +bound=30 + +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT + +# the burst: each an error page, each a render with its callbacks. Given a long leash on purpose - +# the test is about the platform afterwards, and a burst that is still hanging when the leash runs +# out is reported below rather than asserted on + +for i in $(seq 1 "$burst"); do + curl -k -s -o /dev/null -w "%{http_code} %{time_total}\n" --max-time 90 \ + -H "Accept: text/html" \ + "${END_USER_BASE_URL}load-burst-${i}-$$/" > "$tmp/$i" 2>/dev/null & +done +wait + +cat "$tmp"/* | awk '{print $1}' | sort | uniq -c | while read -r n s; do echo "DEBUG: [burst] $n x HTTP $s"; done +echo "DEBUG: [burst] slowest: $(cat "$tmp"/* | awk '{print $2}' | sort -n | tail -1) s" + +# the platform afterwards: any status is an answer - anonymous access to the root is 403 here + +start=$(date +%s) +status=$(curl -k -s -o /dev/null -w "%{http_code}" --max-time "$bound" \ + -H "Accept: application/n-triples" \ + "$END_USER_BASE_URL" || true) +elapsed=$(( $(date +%s) - start )) + +echo "DEBUG: [after] Expected: an HTTP status within ${bound} s Got: $status after ${elapsed} s" +case "$status" in + 000|502|503|504) + echo "DEBUG: [after] the platform did not answer after the burst: its renders are waiting on each other's callbacks" >&2 + exit 1 + ;; +esac diff --git a/tests/load/run.sh b/tests/load/run.sh new file mode 100755 index 000000000..99ec6166b --- /dev/null +++ b/tests/load/run.sh @@ -0,0 +1,88 @@ +#!/usr/bin/env bash + +# Runs every test under tests/load against the stack on https://localhost:4443/, which must have been +# brought up with tests/load/docker-compose.load-tests.yml (see its header). No certificates: the +# suite drives the platform the way unauthenticated traffic does. Writes a CTRF report per suite to +# $TEST_RESULTS_DIR (default: out/), the same shape tests/http/run.sh writes, so +# scripts/generate_test_summary.py reads both. + +hash curl 2>/dev/null || { echo >&2 "curl not on \$PATH. Aborting."; exit 1; } + +export END_USER_BASE_URL="https://localhost:4443/" +export HTTP_MAX_THREADS="${HTTP_MAX_THREADS:-16}" # keep in step with docker-compose.load-tests.yml +export TEST_RESULTS_DIR="${TEST_RESULTS_DIR:-$PWD/out}" + +_ms_probe=$(date +%s%3N 2>/dev/null) +if [[ "$_ms_probe" == *N ]] || [ -z "$_ms_probe" ]; then + function now_ms() { python3 -c 'import time; print(int(time.time()*1000))'; } +else + function now_ms() { date +%s%3N; } +fi + +function json_escape() +{ + local s="$1" + s="${s//\\/\\\\}"; s="${s//\"/\\\"}"; s="${s//$'\n'/\\n}"; s="${s//$'\r'/\\r}"; s="${s//$'\t'/\\t}" + printf '%s' "$s" +} + +error_count=0 + +function run_tests() +{ + local suite_name="$1" + shift + local suite_start_ms suite_end_ms + suite_start_ms=$(now_ms) + mkdir -p "$TEST_RESULTS_DIR" + local results_file="$TEST_RESULTS_DIR/${suite_name}.ctrf.json" + : > "$results_file.tests" + local tests_total=0 tests_passed=0 tests_failed=0 + for script_pathname in "$@" + do + echo -n "$script_pathname" + local log_file t_start_ms t_end_ms duration_ms exit_code status message="" + log_file=$(mktemp) + t_start_ms=$(now_ms) + ( cd "$(dirname "$script_pathname")" || exit; bash -e "$(basename "$script_pathname")"; ) > "$log_file" 2>&1 + exit_code=$? + t_end_ms=$(now_ms) + duration_ms=$(( t_end_ms - t_start_ms )) + if [[ $exit_code == "0" ]]; then + echo " ok"; status="passed"; (( tests_passed += 1 )) + else + echo " failed"; status="failed"; (( tests_failed += 1 )); (( error_count += 1 )) + cat "$log_file" + message=$(tail -c 4096 "$log_file") + fi + (( tests_total += 1 )) + { + printf ' {"name":"%s","status":"%s","duration":%s,"suite":"%s"' "$(json_escape "${script_pathname#./}")" "$status" "$duration_ms" "$(json_escape "$suite_name")" + [ -n "$message" ] && printf ',"message":"%s"' "$(json_escape "$message")" + printf '}\n' + } >> "$results_file.tests" + rm -f "$log_file" + done + suite_end_ms=$(now_ms) + { + printf '{\n "results": {\n "tool": {"name": "load-tests-run.sh"},\n "summary": {\n' + printf ' "tests": %s,\n "passed": %s,\n "failed": %s,\n' "$tests_total" "$tests_passed" "$tests_failed" + printf ' "pending": 0,\n "skipped": 0,\n "other": 0,\n "suites": 1,\n' + printf ' "start": %s,\n "stop": %s\n },\n "tests": [\n' "$suite_start_ms" "$suite_end_ms" + awk 'NR>1 {printf ",\n"} {printf "%s", $0}' "$results_file.tests" + printf '\n ]\n }\n}\n' + } > "$results_file" + rm -f "$results_file.tests" + return $tests_failed +} + +# the platform must be answering before the burst, or the assertion measures the boot instead +until [ "$(curl -k -s -o /dev/null -w '%{http_code}' --max-time 5 "$END_USER_BASE_URL")" = 403 ]; do sleep 1; done + +run_tests "load" $(find . -maxdepth 1 -type f -name '*.sh' ! -name 'run.sh' | sort) + +echo "### Failed tests: $error_count" +if [ "$error_count" -gt 0 ]; then + echo "A failed burst leaves the platform wedged: restart it before running anything else against this stack." >&2 + exit 1 +fi From 9372879a84e0d67f8940133892cf095b1fc9ce8c Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 17:19:00 +0200 Subject: [PATCH 09/16] The 429 retry is asserted as an axis, not a chart spec: axes/rate-limit.spec refuses every block request on the fixture page once and checks that each is sent again as itself, that nothing is fetched more often than on an unrefused load, and that the chart, the view and the object block all render; chart/rate-limit.spec, which asserted it on the chart alone, is folded into it. The retry is ldh:retry-request's and serves every block's request chain, so the chart was only where the defect showed. Against a client SEF without b87fc9be5 the spec fails on the chart ("Table has no columns"), and with the chart assertion taken out it still fails on six view and object metadata requests (four /sparql, two /ns) that were never re-sent; with the fix it passes 6/6 over three repeats, as owner and anonymous. Co-Authored-By: Claude Fable 5.1 --- tests/ui/specs/axes/rate-limit.spec.mjs | 104 ++++++++++++++++++ .../document/blocks/chart/rate-limit.spec.mjs | 60 ---------- 2 files changed, 104 insertions(+), 60 deletions(-) create mode 100644 tests/ui/specs/axes/rate-limit.spec.mjs delete mode 100644 tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs diff --git a/tests/ui/specs/axes/rate-limit.spec.mjs b/tests/ui/specs/axes/rate-limit.spec.mjs new file mode 100644 index 000000000..399543edb --- /dev/null +++ b/tests/ui/specs/axes/rate-limit.spec.mjs @@ -0,0 +1,104 @@ +// A request the rate limiter refuses is retried as itself, on every block kind. +// +// A page of content blocks fires more requests at once than nginx's `linked_data` zone lets through +// (15 r/s, burst 30), so on a busy page some come back 429 and the client retries them after +// Retry-After - in ldh:retry-request, which every block's request chain goes through. It re-sent +// `$context('request')` whatever step it was retrying, and that is only the FIRST request of a +// chain: a view's metadata, an object's metadata, a chart's results were each retried as the GET +// of their block's document. The chart showed it, because it draws what it gets - handed the +// document's RDF/XML as its "results" it drew "Table has no columns." - and on the Northwind demo +// that was a different chart on every reload. The other steps swallowed it quieter. +// +// An axis, not a chart spec: the claim is the retry's, and the chart is only where it was seen. So +// the page is the fixture container with a block of every kind on it, and EVERY request its blocks +// make is refused once, the limiter at its worst. The limiter cannot be triggered on demand, so a +// route stands in for it. +// +// Three assertions, because each defect has a different shape. A refused request that never goes +// out again is a retry that went elsewhere. But blocks repeat each other's requests, so "it went +// out again" can be satisfied by another block making the same one - hence the baseline: a load +// with nothing refused, and nothing may be fetched more often under refusal than it is there, +// because the misdirected retries are exactly those extra fetches. And last, what the reader sees: +// every block rendered. +import { test, expect } from '../../lib/console.mjs'; +import { goto } from '../../lib/settle.mjs'; +import { fixtures, itemTitle } from '../../lib/fixtures.mjs'; +import { chartBlock, drawing } from '../../lib/chart.mjs'; +import { fixtureView } from '../../lib/view.mjs'; + +// The requests the blocks make: XHR to this stack. Not the SEF - refusing that refuses the whole +// client, and there is nothing left to retry anything - nor a third party's. +const fromBlocks = (request, origin) => ['xhr', 'fetch'].includes(request.resourceType()) + && new URL(request.url()).origin === origin + && !request.url().endsWith('.sef.json'); + +// A request by what it asks for, so a retry is recognised as the request it retries. +const keyOf = request => `${request.method()} ${request.url()}\n${request.postData() ?? ''}`; +// And by its shape, for comparing two loads: a view names its SPARQL variables with a UUID minted +// per render, so the same query differs between loads by those alone - never within a retry. +const shapeOf = key => key.replace(/[0-9a-f]{8}_[0-9a-f]{4}_[0-9a-f]{4}_[0-9a-f]{4}_[0-9a-f]{12}/g, 'UUID'); + +// Routing also switches the HTTP cache off, which is what makes two loads of one page comparable. +async function record(page, { refuse }) { + const origin = new URL(fixtures.container).origin; + const sent = new Set(); + const shapes = new Map(); + const refused = new Set(); + await page.route('**/*', route => { + const request = route.request(); + if (!fromBlocks(request, origin)) return route.fallback(); + + const key = keyOf(request); + if (refuse && !refused.has(key)) { + refused.add(key); + // No Retry-After, as nginx sends none: the client falls back to its default wait. + return route.fulfill({ status: 429, contentType: 'text/html', body: '429 Too Many Requests' }); + } + sent.add(key); + shapes.set(shapeOf(key), (shapes.get(shapeOf(key)) ?? 0) + 1); + return route.fallback(); + }); + return { sent, shapes, refused }; +} + +// Every block on the fixture page, drawn: the chart has a picture, the view its first item, the +// object block the resource it names. +async function rendered(page) { + await expect(drawing(chartBlock(page)), 'the chart draws').toBeVisible({ timeout: 30_000 }); + await expect(fixtureView(page), 'the view lists the items').toContainText(itemTitle(1), { timeout: 30_000 }); + await expect(page.locator(`div.block.ldh-block[about="${fixtures.object}"] .ldh-obj-value`).first(), + 'the object block renders its resource').not.toBeEmpty({ timeout: 30_000 }); +} + +test('every request refused with 429 is retried as itself, and every block still renders', async ({ page, allowNoise }) => { + test.setTimeout(120_000); + allowNoise.push({ pattern: /^HTTP 429: /, reason: 'this spec refuses every block request once on purpose' }); + allowNoise.push({ + pattern: /console\.error: Failed to load resource.*429/i, + reason: 'the browser logs each injected 429 the route fulfils', + }); + + const unrefused = await page.context().newPage(); + const baseline = await record(unrefused, { refuse: false }); + await goto(unrefused, fixtures.container); + await rendered(unrefused); + // Every request the page makes unprovoked, not just the ones the blocks render from. + await unrefused.waitForLoadState('networkidle'); + await unrefused.close(); + + const { sent, shapes, refused } = await record(page, { refuse: true }); + await goto(page, fixtures.container); + await rendered(page); + + expect(refused.size, 'the page made block requests to refuse').toBeGreaterThan(0); + // Polled: a block renders from its first response, and the view's count and facet requests are + // still waiting out their Retry-After when its items are already on the page. + await expect.poll(() => [...refused].filter(key => !sent.has(key)), { + message: 'every refused request is sent again', + timeout: 20_000, + }).toEqual([]); + await page.waitForLoadState('networkidle'); + const extra = [...shapes].filter(([shape, count]) => count > (baseline.shapes.get(shape) ?? 0)) + .map(([shape, count]) => `${count}x (baseline ${baseline.shapes.get(shape) ?? 0}x) ${shape}`); + expect(extra, 'nothing is fetched more often than an unrefused load fetches it').toEqual([]); +}); diff --git a/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs b/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs deleted file mode 100644 index 97af321dc..000000000 --- a/tests/ui/specs/document/blocks/chart/rate-limit.spec.mjs +++ /dev/null @@ -1,60 +0,0 @@ -// The chart block, when its results request is rate-limited. -// -// A page of content blocks fires more requests at once than nginx's `linked_data` zone lets through -// (15 r/s, burst 30), so on a busy page some of them come back 429 and the client retries them after -// Retry-After. The retry re-sent `$context('request')` whatever step it was retrying - and for a -// chart that is the GET of the query's DOCUMENT, the first request of its chain, not the POST of -// the query. The chart was handed the document's RDF/XML as its "results", mapped `?kind` and -// `?items` as property URIs onto it, and drew "Table has no columns." over one row per resource in -// the document. Which chart it hit depended on which request the limiter refused, so on the -// Northwind demo it was a different chart on every reload, and reloading "fixed" it. -// -// The limiter is not something a spec can trigger on demand, so the 429 is fulfilled by a route - -// once, for the results POST alone - and the assertion is on both ends of the retry: the request -// that went out again, and the picture it drew. -import { test, expect } from '../../../../lib/console.mjs'; -import { goto } from '../../../../lib/settle.mjs'; -import { fixtures, kinds } from '../../../../lib/fixtures.mjs'; -import { canvas, chartBlock, drawing } from '../../../../lib/chart.mjs'; - -// The results request: the fixture chart's query, POSTed to the endpoint. Its body is the query -// text, and the aggregate is what tells it from the view's query over the same items. -const isResults = request => request.method() === 'POST' - && /COUNT\(\?item\) AS \?items/.test(request.postData() ?? ''); - -test('a results request refused with 429 is retried, and the chart draws its own results', async ({ page, allowNoise }) => { - allowNoise.push({ pattern: /^HTTP 429: /, reason: 'this spec refuses the results request once on purpose' }); - allowNoise.push({ - pattern: /console\.error: Failed to load resource.*429/i, - reason: 'the browser logs the injected 429 the route fulfils', - }); - - const results = []; - await page.route('**/*', route => { - const request = route.request(); - if (!isResults(request)) return route.fallback(); - - results.push(request); - // No Retry-After, as nginx sends none: the client falls back to its default wait. - return results.length === 1 - ? route.fulfill({ status: 429, contentType: 'text/html', body: '429 Too Many Requests' }) - : route.fallback(); - }); - - await goto(page, fixtures.container); - - const block = chartBlock(page); - await expect(drawing(block), 'the chart draws after the retry').toBeVisible({ timeout: 30_000 }); - // What went out again is the query, not the document it is stored in. Before the fix the count - // stayed at 1: the retry was a GET, which this route never sees. - expect(results, 'the refused results POST is the request that is retried').toHaveLength(2); - expect(results[1].postData()).toBe(results[0].postData()); - - // And the drawing is of the query's rows: a label per kind. A chart drawn from the document's - // RDF/XML has no column the category maps onto, so it has no kind labels either. - const labels = await canvas(block).locator('svg text').allTextContents(); - for (const kind of kinds) { - expect(labels, `the ${kind} bar is labelled`).toContain(kind); - } - await expect(canvas(block), 'the chart matched nothing').not.toContainText('Table has no columns'); -}); From 5cc4b0c2d6a6667da09f8fdbfc8331b75b456137 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 18:12:01 +0200 Subject: [PATCH 10/16] A request to this instance's own URL is bounded on its own, and the client pool is sized to the connector, so a burst of renders can no longer deadlock the platform against itself. Every server-side HTML render calls back into its own dataspace through the proxy for the page's labels, and each callback is answered by one of the same Tomcat's request threads while the render holds another; once renders outnumbered the threads, every thread was a render waiting for a callback no thread was free to serve, and nothing completed until the client's read timeout, which is sized for a stalled backend. ClientUriRewriteFilter, the one place that recognizes such a request and sends it to the proxy, now gives it a connect and read timeout of its own (CLIENT_SELF_REQUEST_TIMEOUT, 5 s), and SendHTTPRequest hands the client's ProcessingException to the stylesheet as the SaxonApiException its xsl:try catches, so a lookup that gave up is a missing label rather than a failed render. The image's pool grows from 20/40 to 200/400 connections, the connector's thread count: a self-call holds a pooled connection while it waits for another thread, so a smaller pool queued them behind each other and a burst drained one bounded wait at a time (218 s to recover on the load stack, against 78 s sized to its 16 threads). The pool-wait gets a code default of 30 s, matching the image's, where it was unbounded outside the image. tests/load starts from settings of its own rather than the repository's, whose package imports are the developer's. Co-Authored-By: Claude Fable 5.1 --- .gitignore | 2 + CHANGELOG.md | 1 + Dockerfile | 12 +++- platform/entrypoint.sh | 5 ++ .../atomgraph/linkeddatahub/Application.java | 10 ++- .../client/filter/ClientUriRewriteFilter.java | 32 ++++++++- .../writer/function/SendHTTPRequest.java | 6 +- .../filter/ClientUriRewriteFilterTest.java | 68 ++++++++++++++++--- tests/load/docker-compose.load-tests.yml | 16 +++-- tests/load/render-burst-no-deadlock.sh | 23 ++++--- tests/load/settings/.gitkeep | 0 11 files changed, 143 insertions(+), 32 deletions(-) create mode 100644 tests/load/settings/.gitkeep diff --git a/.gitignore b/.gitignore index ee91a6fa0..ac8300205 100644 --- a/.gitignore +++ b/.gitignore @@ -17,6 +17,8 @@ /tests/http/uploads /tests/http/out /tests/load/out +/tests/load/settings/* +!/tests/load/settings/.gitkeep /fuseki .claude/scheduled_tasks.lock /cli/target diff --git a/CHANGELOG.md b/CHANGELOG.md index 026ac6d6b..52bf90167 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,6 @@ ## [Unreleased] ### Fixed +- A burst of renders could deadlock the platform against itself. Every server-side HTML render calls back into its own dataspace through the proxy for the page's labels (`ldh:send-request` to `/sparql` and `/ns`, and the stylesheet's `document()` loads), and each callback is answered by one of the same Tomcat's request threads while the render holds another. Once renders outnumbered the connector's threads - a scanner probing non-existent paths did it on linkeddatahub.com, every 403 and 404 page being a full render - every thread was a render waiting for a callback no thread was free to serve, and nothing completed until the client's read timeout, which is sized for a stalled backend. A request to this instance's own URL is now bounded on its own: `ClientUriRewriteFilter`, the one place that recognizes such a request and sends it to the proxy, gives it a connect and read timeout of `CLIENT_SELF_REQUEST_TIMEOUT` (default 5 s), after which the render gives its thread back. And the image's client pool is sized to the connector (`MAX_CONN_PER_ROUTE=200`, `MAX_TOTAL_CONN=400`, from 20 and 40): a self-call holds a pooled connection while it waits for another thread, so a pool smaller than the connector queued them behind each other and a burst drained one bounded wait at a time. The pool-wait (`CONNECTION_REQUEST_TIMEOUT`) also has a code default of 30 s now, matching the image's, where it used to be unbounded outside the image. `tests/load/render-burst-no-deadlock.sh` reproduces the wedge on a 16-thread stack and guards the fix. - Proxied XHTML content left path-absolute `@href`/`@src` references unresolved, so authored media (`/uploads/{sha1}`) resolved against the proxying dataspace's origin and 404'd. The proxy-mode rewrite in `ldh:XHTMLContent` now resolves them against the content's base URI alongside the relative ones; only fragments stay untouched, because they address the rendering rather than the source ## [6.0.0] - 2026-09-29 diff --git a/Dockerfile b/Dockerfile index 555d66fd4..a5ad77f18 100644 --- a/Dockerfile +++ b/Dockerfile @@ -181,9 +181,12 @@ ENV OIDC_REFRESH_TOKENS=/var/linkeddatahub/oidc/refresh_tokens.properties ENV MAX_CONTENT_LENGTH=2097152 -ENV MAX_CONN_PER_ROUTE=20 +# sized to the connector's 200 threads: a request thread that calls back into this instance holds one pooled +# connection while it waits for another thread to answer, so a pool smaller than the connector queues +# those calls behind each other, one bounded wait at a time, and a burst of renders drains one slot at a time +ENV MAX_CONN_PER_ROUTE=200 -ENV MAX_TOTAL_CONN=40 +ENV MAX_TOTAL_CONN=400 ENV MAX_REQUEST_RETRIES=3 @@ -193,6 +196,11 @@ ENV CLIENT_SOCKET_TIMEOUT=120000 ENV CLIENT_CONNECT_TIMEOUT=10000 +# for requests to this instance's own URLs, answered by its own request threads: a label lookup takes milliseconds +# through the proxy, and this is also the longest a burst of renders can hold every thread waiting on each other +# (a render makes up to three in sequence) +ENV CLIENT_SELF_REQUEST_TIMEOUT=5000 + ENV CLIENT_CONNECTION_TIME_TO_LIVE=300000 ENV CLIENT_VALIDATE_AFTER_INACTIVITY=10000 diff --git a/platform/entrypoint.sh b/platform/entrypoint.sh index ba224b5e0..8cdd93c3d 100755 --- a/platform/entrypoint.sh +++ b/platform/entrypoint.sh @@ -1233,6 +1233,11 @@ if [ -n "$CLIENT_CONNECT_TIMEOUT" ]; then export CATALINA_OPTS="$CATALINA_OPTS -Dcom.atomgraph.linkeddatahub.connectTimeout=$CLIENT_CONNECT_TIMEOUT" fi +# connect and read timeout for the platform's requests to its own URLs, which its own request threads answer +if [ -n "$CLIENT_SELF_REQUEST_TIMEOUT" ]; then + export CATALINA_OPTS="$CATALINA_OPTS -Dcom.atomgraph.linkeddatahub.selfRequestTimeout=$CLIENT_SELF_REQUEST_TIMEOUT" +fi + if [ -n "$CLIENT_CONNECTION_TIME_TO_LIVE" ]; then export CATALINA_OPTS="$CATALINA_OPTS -Dcom.atomgraph.linkeddatahub.connectionTimeToLive=$CLIENT_CONNECTION_TIME_TO_LIVE" fi diff --git a/src/main/java/com/atomgraph/linkeddatahub/Application.java b/src/main/java/com/atomgraph/linkeddatahub/Application.java index a7780e746..d5f7f5266 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/Application.java +++ b/src/main/java/com/atomgraph/linkeddatahub/Application.java @@ -264,6 +264,8 @@ public class Application extends ResourceConfig /** Webapp path of the client stylesheet built at package time. Its digest fingerprints the platform build, so a composed stylesheet is invalidated by an upgrade */ public static final String CLIENT_SEF_PATH = "/static/com/atomgraph/linkeddatahub/xsl/client.xsl.sef.json"; + /** Milliseconds a request waits for a connection from the client pool when nothing configures it; the image sets the same through CONNECTION_REQUEST_TIMEOUT */ + public static final int DEFAULT_CONNECTION_REQUEST_TIMEOUT = 30000; /** Path of the client stylesheet source in the webapp, composed with package stylesheets per import set */ public static final String CLIENT_XSL_PATH = "/static/com/atomgraph/linkeddatahub/xsl/client.xsl"; @@ -371,9 +373,11 @@ public Application(@Context ServletConfig servletConfig) throws URISyntaxExcepti servletConfig.getServletContext().getInitParameter(LDHC.maxTotalConn.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.maxTotalConn.getURI())) : null, servletConfig.getServletContext().getInitParameter(LDHC.maxRequestRetries.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.maxRequestRetries.getURI())) : null, System.getProperty("com.atomgraph.linkeddatahub.connectionRequestTimeout") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.connectionRequestTimeout")) : - servletConfig.getServletContext().getInitParameter(LDHC.connectionRequestTimeout.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.connectionRequestTimeout.getURI())) : null, + servletConfig.getServletContext().getInitParameter(LDHC.connectionRequestTimeout.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.connectionRequestTimeout.getURI())) : + DEFAULT_CONNECTION_REQUEST_TIMEOUT, // unset means Apache waits for a pooled connection forever; a thread parked on the pool is never coming back on its own System.getProperty("com.atomgraph.linkeddatahub.socketTimeout") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.socketTimeout")) : null, System.getProperty("com.atomgraph.linkeddatahub.connectTimeout") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.connectTimeout")) : null, + System.getProperty("com.atomgraph.linkeddatahub.selfRequestTimeout") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.selfRequestTimeout")) : null, System.getProperty("com.atomgraph.linkeddatahub.connectionTimeToLive") != null ? Long.valueOf(System.getProperty("com.atomgraph.linkeddatahub.connectionTimeToLive")) : null, System.getProperty("com.atomgraph.linkeddatahub.validateAfterInactivity") != null ? Integer.valueOf(System.getProperty("com.atomgraph.linkeddatahub.validateAfterInactivity")) : null, servletConfig.getServletContext().getInitParameter(LDHC.maxImportThreads.getURI()) != null ? Integer.valueOf(servletConfig.getServletContext().getInitParameter(LDHC.maxImportThreads.getURI())) : null, @@ -463,7 +467,7 @@ public Application(final ServletConfig servletConfig, final MediaTypes mediaType final String uploadRootString, final String sefRootString, final String sefCompilerString, final String clientStylesheetString, final boolean invalidateCache, final Integer cookieMaxAge, final boolean enableLinkedDataProxy, final boolean allowInternalUrls, final Integer maxContentLength, final Integer maxConnPerRoute, final Integer maxTotalConn, final Integer maxRequestRetries, final Integer connectionRequestTimeout, - final Integer socketTimeout, final Integer connectTimeout, final Long connectionTimeToLive, final Integer validateAfterInactivity, final Integer maxImportThreads, + final Integer socketTimeout, final Integer connectTimeout, final Integer selfRequestTimeout, final Long connectionTimeToLive, final Integer validateAfterInactivity, final Integer maxImportThreads, final String notificationAddressString, final boolean enableWebIDSignUp, final String oidcRefreshTokensPropertiesPath, final String frontendProxyString, final String backendProxyAdminString, final String backendProxyEndUserString, final String mailUser, final String mailPassword, final String smtpHost, final String smtpPort, @@ -760,7 +764,7 @@ public Application(final ServletConfig servletConfig, final MediaTypes mediaType if (proxyHostname != null) { - ClientRequestFilter rewriteFilter = new ClientUriRewriteFilter(baseURI.getHost(), proxyScheme, proxyHostname, proxyPort); // proxyPort can be null + ClientRequestFilter rewriteFilter = new ClientUriRewriteFilter(baseURI.getHost(), proxyScheme, proxyHostname, proxyPort, selfRequestTimeout); // proxyPort can be null client.register(rewriteFilter); externalClient.register(rewriteFilter); diff --git a/src/main/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilter.java b/src/main/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilter.java index 6c04fb5eb..ce64b465b 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilter.java +++ b/src/main/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilter.java @@ -22,12 +22,20 @@ import jakarta.ws.rs.client.ClientRequestFilter; import jakarta.ws.rs.core.HttpHeaders; import jakarta.ws.rs.core.UriBuilder; +import org.glassfish.jersey.client.ClientProperties; import org.slf4j.Logger; import org.slf4j.LoggerFactory; /** * Client request filter that rewrites target URLs matching the configured host to internal proxy URLs. * This improves performance by routing internal requests through the Docker network instead of external network. + *

+ * A request to this instance's own URL differs from every other outbound request in one way: it is answered by + * one of this instance's own request threads, and the thread that sent it may be one of them. A server-side + * render that asks its own dataspace for labels holds a thread while it waits; when every thread is such a + * render, none is free to answer any of them, and nothing completes until a read times out. So a self-call + * waits a few seconds and then fails, and the caller renders without it (the stylesheets catch the failure), + * rather than waiting for the client's read timeout, which is sized for a stalled backend. * * @author {@literal Martynas Jusevičius } */ @@ -38,7 +46,7 @@ public class ClientUriRewriteFilter implements ClientRequestFilter private final String host; private final String proxyScheme, proxyHost; - private final Integer proxyPort; + private final Integer proxyPort, selfRequestTimeout; /** * Constructs filter from URI components. @@ -47,13 +55,15 @@ public class ClientUriRewriteFilter implements ClientRequestFilter * @param proxyScheme proxy scheme to rewrite to (e.g., "http") * @param proxyHost proxy hostname to rewrite to (e.g., "nginx") * @param proxyPort proxy port to rewrite to (e.g., 9443) + * @param selfRequestTimeout connect and read timeout in milliseconds for requests to the matched host, or null to leave the client's */ - public ClientUriRewriteFilter(String host, String proxyScheme, String proxyHost, Integer proxyPort) + public ClientUriRewriteFilter(String host, String proxyScheme, String proxyHost, Integer proxyPort, Integer selfRequestTimeout) { this.host = host; this.proxyScheme = proxyScheme; this.proxyHost = proxyHost; this.proxyPort = proxyPort; + this.selfRequestTimeout = selfRequestTimeout; } @Override @@ -80,6 +90,14 @@ public void filter(ClientRequestContext cr) throws IOException newHost = subdomainPrefix + getProxyHost(); } + // the answer has to come from one of this instance's own request threads: bound the wait, unless the + // caller set its own. The Apache connector reads both properties per request. + if (getSelfRequestTimeout() != null) + { + if (cr.getProperty(ClientProperties.READ_TIMEOUT) == null) cr.setProperty(ClientProperties.READ_TIMEOUT, getSelfRequestTimeout()); + if (cr.getProperty(ClientProperties.CONNECT_TIMEOUT) == null) cr.setProperty(ClientProperties.CONNECT_TIMEOUT, getSelfRequestTimeout()); + } + // cannot use the URI class because query string with special chars such as '+' gets decoded URI newUri = UriBuilder.fromUri(cr.getUri()).scheme(newScheme).host(newHost).port(getProxyPort()).build(); @@ -127,4 +145,14 @@ public Integer getProxyPort() return proxyPort; } + /** + * Connect and read timeout for requests to the matched host. + * + * @return timeout in milliseconds, or null if the client's own applies + */ + public Integer getSelfRequestTimeout() + { + return selfRequestTimeout; + } + } diff --git a/src/main/java/com/atomgraph/linkeddatahub/writer/function/SendHTTPRequest.java b/src/main/java/com/atomgraph/linkeddatahub/writer/function/SendHTTPRequest.java index 647356ef8..0153fefb1 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/writer/function/SendHTTPRequest.java +++ b/src/main/java/com/atomgraph/linkeddatahub/writer/function/SendHTTPRequest.java @@ -22,6 +22,7 @@ import java.io.InputStream; import java.util.stream.Collectors; import jakarta.ws.rs.client.Client; +import jakarta.ws.rs.ProcessingException; import jakarta.ws.rs.client.Entity; import jakarta.ws.rs.core.MultivaluedHashMap; import jakarta.ws.rs.core.MultivaluedMap; @@ -146,7 +147,10 @@ public XdmValue call(XdmValue[] arguments) throws SaxonApiException return XdmEmptySequence.getInstance(); } - catch (IOException | ParserConfigurationException | SAXException ex) + // ProcessingException is how the client reports a connect or read timeout, a runtime exception the stylesheet's + // xsl:try cannot catch unless it reaches it as a SaxonApiException like the others: a label lookup that timed out + // is a page without that label, not a failed render + catch (IOException | ParserConfigurationException | SAXException | ProcessingException ex) { throw new SaxonApiException(ex); } diff --git a/src/test/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilterTest.java b/src/test/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilterTest.java index ce3d6e89f..f63d7133c 100644 --- a/src/test/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilterTest.java +++ b/src/test/java/com/atomgraph/linkeddatahub/client/filter/ClientUriRewriteFilterTest.java @@ -31,9 +31,11 @@ import java.net.URI; import java.util.Collection; import java.util.Date; +import java.util.HashMap; import java.util.List; import java.util.Locale; import java.util.Map; +import org.glassfish.jersey.client.ClientProperties; import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.*; @@ -49,6 +51,7 @@ private static class StubRequestContext implements ClientRequestContext { private URI uri; private final MultivaluedMap headers = new MultivaluedHashMap<>(); + private final Map properties = new HashMap<>(); StubRequestContext(URI uri) { this.uri = uri; } @@ -56,10 +59,10 @@ private static class StubRequestContext implements ClientRequestContext @Override public void setUri(URI uri) { this.uri = uri; } @Override public MultivaluedMap getHeaders() { return headers; } - @Override public Object getProperty(String name) { throw new UnsupportedOperationException(); } - @Override public Collection getPropertyNames() { throw new UnsupportedOperationException(); } - @Override public void setProperty(String name, Object object) { throw new UnsupportedOperationException(); } - @Override public void removeProperty(String name) { throw new UnsupportedOperationException(); } + @Override public Object getProperty(String name) { return properties.get(name); } + @Override public Collection getPropertyNames() { return properties.keySet(); } + @Override public void setProperty(String name, Object object) { properties.put(name, object); } + @Override public void removeProperty(String name) { properties.remove(name); } @Override public String getMethod() { throw new UnsupportedOperationException(); } @Override public void setMethod(String method) { throw new UnsupportedOperationException(); } @Override public MultivaluedMap getStringHeaders() { throw new UnsupportedOperationException(); } @@ -88,18 +91,63 @@ private static class StubRequestContext implements ClientRequestContext @Test public void testNoRewriteForNonMatchingHost() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://other.org/path")); filter.filter(ctx); assertEquals(URI.create("https://other.org/path"), ctx.getUri()); assertTrue(ctx.getHeaders().isEmpty()); + assertTrue(ctx.getPropertyNames().isEmpty()); + } + + /** Non-matching host with a self-request timeout configured: the timeout is for own URLs only. */ + @Test + public void testNoTimeoutForNonMatchingHost() throws IOException + { + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, 5000); + StubRequestContext ctx = new StubRequestContext(URI.create("https://other.org/path")); + filter.filter(ctx); + assertTrue(ctx.getPropertyNames().isEmpty()); + } + + /** Own URL: the self-request timeout bounds both connect and read, as Integer, which the Apache connector reads per request. */ + @Test + public void testSelfRequestTimeoutSetOnOwnUrl() throws IOException + { + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, 5000); + StubRequestContext ctx = new StubRequestContext(URI.create("https://admin.example.com/sparql")); + filter.filter(ctx); + assertEquals(URI.create("http://nginx:9443/sparql"), ctx.getUri()); + assertEquals(Integer.valueOf(5000), ctx.getProperty(ClientProperties.READ_TIMEOUT)); + assertEquals(Integer.valueOf(5000), ctx.getProperty(ClientProperties.CONNECT_TIMEOUT)); + } + + /** Own URL without a self-request timeout: the client's own timeouts apply, nothing is set. */ + @Test + public void testNoSelfRequestTimeoutLeavesPropertiesUnset() throws IOException + { + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); + StubRequestContext ctx = new StubRequestContext(URI.create("https://example.com/sparql")); + filter.filter(ctx); + assertTrue(ctx.getPropertyNames().isEmpty()); + } + + /** A caller that set its own timeout on the request keeps it. */ + @Test + public void testCallerTimeoutNotOverridden() throws IOException + { + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, 5000); + StubRequestContext ctx = new StubRequestContext(URI.create("https://example.com/sparql")); + ctx.setProperty(ClientProperties.READ_TIMEOUT, 60000); + filter.filter(ctx); + assertEquals(Integer.valueOf(60000), ctx.getProperty(ClientProperties.READ_TIMEOUT)); + assertEquals(Integer.valueOf(5000), ctx.getProperty(ClientProperties.CONNECT_TIMEOUT)); } /** Exact host match: URI host is rewritten to proxyHost, scheme to proxyScheme. */ @Test public void testRewriteExactHost() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://example.com/path?q=1")); filter.filter(ctx); assertEquals(URI.create("http://nginx:9443/path?q=1"), ctx.getUri()); @@ -110,7 +158,7 @@ public void testRewriteExactHost() throws IOException @Test public void testRewriteExactHostWithPort() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://example.com:4443/path")); filter.filter(ctx); assertEquals(URI.create("http://nginx:9443/path"), ctx.getUri()); @@ -126,7 +174,7 @@ public void testRewriteExactHostWithPort() throws IOException @Test public void testRewriteSubdomainPreservesSubdomainWithSameDomainProxy() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "https", "example.com", 5443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "https", "example.com", 5443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://admin.example.com/acl/agents/123/")); filter.filter(ctx); assertEquals(URI.create("https://admin.example.com:5443/acl/agents/123/"), ctx.getUri()); @@ -141,7 +189,7 @@ public void testRewriteSubdomainPreservesSubdomainWithSameDomainProxy() throws I @Test public void testRewriteSubdomainWithInternalProxyUsesProxyHostOnly() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://admin.example.com/path")); filter.filter(ctx); assertEquals(URI.create("http://nginx:9443/path"), ctx.getUri()); @@ -152,7 +200,7 @@ public void testRewriteSubdomainWithInternalProxyUsesProxyHostOnly() throws IOEx @Test public void testQueryStringNotDecoded() throws IOException { - ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443); + ClientUriRewriteFilter filter = new ClientUriRewriteFilter("example.com", "http", "nginx", 9443, null); StubRequestContext ctx = new StubRequestContext(URI.create("https://example.com/sparql?query=ASK+%7B%7D")); filter.filter(ctx); assertEquals("query=ASK+%7B%7D", ctx.getUri().getRawQuery()); diff --git a/tests/load/docker-compose.load-tests.yml b/tests/load/docker-compose.load-tests.yml index 3833f0973..edebd6737 100644 --- a/tests/load/docker-compose.load-tests.yml +++ b/tests/load/docker-compose.load-tests.yml @@ -4,18 +4,24 @@ # -f tests/load/docker-compose.load-tests.yml --env-file tests/http/.env up --build -d # # A stack sized so that a burst the test runner can produce reproduces what only real traffic -# reaches on a production connector of 200 threads. The ratio is what matters and it mirrors -# production: more request threads than outbound connections per route, so a burst of renders -# first exhausts the client pool and then the connector, in that order. +# reaches on a production connector of 200 threads. The ratio is what matters and it mirrors the +# image's defaults: the client pool holds as many connections per route as the connector has +# threads, so a self-call never waits for a pool slot behind other self-calls, only on its own +# bounded read. (With a pool of 4 against these 16 threads, as before the fix, a burst drained +# one bounded wait at a time and recovery took over three minutes.) services: linkeddatahub: environment: - HTTP_MAX_THREADS=16 # Tomcat's connector behind the proxy; default 200 - - MAX_CONN_PER_ROUTE=4 # the platform's outbound client pool, per host; production had 40 - - MAX_TOTAL_CONN=8 + - MAX_CONN_PER_ROUTE=16 # the platform's outbound client pool, per host; the image's 200 scaled with the connector + - MAX_TOTAL_CONN=32 # far past any bound the tests use: a wedge that resolves only because the platform's reads # time out is still the wedge, and the dev default of two minutes would hide it behind the probe - CLIENT_SOCKET_TIMEOUT=600000 + volumes: + # the base mounts the repository's ./settings, a developer's own package imports included; the + # burst renders the stock dataspace, so this stack starts from settings of its own, kept empty + - ./tests/load/settings:/var/www/linkeddatahub/settings fuseki: # the HTTP suite publishes the triplestore for its fixtures; this suite only talks to the front # door, and a developer's own stack may already hold the port diff --git a/tests/load/render-burst-no-deadlock.sh b/tests/load/render-burst-no-deadlock.sh index d7206fb2b..961ac74e7 100755 --- a/tests/load/render-burst-no-deadlock.sh +++ b/tests/load/render-burst-no-deadlock.sh @@ -14,20 +14,25 @@ set -euo pipefail # do not exist across the dataspace origins, every 403 and 404 page is a full render, and the 200 # threads filled within a minute. The pages were error pages, the callbacks were not. # -# The stack runs with a 16-thread connector and a 4-per-route client pool -# (docker-compose.load-tests.yml), production's ratio at a size the runner can saturate: a burst of +# The stack runs with a 16-thread connector and a client pool sized to it +# (docker-compose.load-tests.yml), the image's ratio at a size the runner can saturate: a burst of # three times the connector is enough. The requests are anonymous GETs of documents that do not # exist, which is what the scanner sent. The bound is the assertion: once the burst has been # answered or given up on, a plain request must get an HTTP status within 30 s. curl's 000 and the # proxy's 502/503/504 are the deadlock, whatever the platform would eventually have answered. # -# Measured on 6.0.0 with this stack: without a fix the platform answers nothing for as long as its -# reads take to time out (the override sets that to ten minutes). A pool-wait timeout -# (CONNECTION_REQUEST_TIMEOUT=10000) is a mitigation, not a fix - the renders queued on the pool -# fail in ten-second waves and the platform answered again 113 s after the burst began, which this -# bound is right to reject. It passes once a render no longer needs a request thread of its own -# to be answered: the callbacks answered in-process, or through a client of their own that cannot -# take the connector down with it. +# Measured on 6.0.0 as released, with this stack: the platform answers nothing for as long as its +# reads take to time out (the override sets that to ten minutes); a shorter pool-wait alone +# (CONNECTION_REQUEST_TIMEOUT=10000) only shortens that to 113 s. What makes it pass is two +# things together. ClientUriRewriteFilter, which is what sends a request to this instance's own +# URL through the proxy, gives that request a connect and read timeout of its own +# (CLIENT_SELF_REQUEST_TIMEOUT, 5 s by default), so a render that cannot get its labels gives its +# thread back within the bound. And the client pool is sized to the connector (MAX_CONN_PER_ROUTE +# at least the thread count), so a self-call waits only on that read and never for a pool slot +# behind other self-calls: with a pool of 4 here the bounded waits still drained one slot at a +# time and recovery took 218 s; sized to the 16 threads it took 78 s, and the burst that +# provoked it drains as rendered pages without their labels (SendHTTPRequest hands the timeout +# to xsl:try as a SaxonApiException, so a lookup that gave up is a missing label, not a 500). burst=$(( HTTP_MAX_THREADS * 3 )) bound=30 diff --git a/tests/load/settings/.gitkeep b/tests/load/settings/.gitkeep new file mode 100644 index 000000000..e69de29bb From ef9fb0b404df4efca39a99721da5c9bd29e294d4 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 20:20:28 +0200 Subject: [PATCH 11/16] Package ontologies no longer take their dataspace down, and packages.linkeddatahub.com is no longer shadowed by bundled copies. prefix-mapping.ttl mapped the whole https://packages.linkeddatahub.com/ prefix to the bundled packages.ttl, so every URI on that origin resolved to the catalog - the packages dataspace's own translations.rdf included, and its every HTML page failed with an empty ac:label() in the breadcrumb; the three mappings and the bundled packages.ttl, editor/taxonomy/package.ttl and ns.ttl go, and the registry is read over HTTP. With the mappings gone a descriptor naming its ontology's document (lds:ontology ) while the ontology inside is loaded two graphs of that name into the closure, and ontapi refused the second: declarePackageImports now imports the ontology IRI the resolved graph declares, and loadOntology assembles the closure without the package ontologies, retracting their imports, when it cannot be assembled with them. OntologyFilterTest covers both. Co-Authored-By: Claude Fable 5.1 --- .../server/filter/request/OntologyFilter.java | 52 ++- .../packages/editor/taxonomy/ns.ttl | 318 ------------------ .../packages/editor/taxonomy/package.ttl | 21 -- .../com/linkeddatahub/packages/packages.ttl | 12 - src/main/resources/prefix-mapping.ttl | 9 +- .../linkeddatahub/xsl/client/packages.xsl | 3 +- .../filter/request/OntologyFilterTest.java | 63 ++++ 7 files changed, 105 insertions(+), 373 deletions(-) delete mode 100644 src/main/resources/com/linkeddatahub/packages/editor/taxonomy/ns.ttl delete mode 100644 src/main/resources/com/linkeddatahub/packages/editor/taxonomy/package.ttl delete mode 100644 src/main/resources/com/linkeddatahub/packages/packages.ttl diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java index 23f3bc554..39947e9c7 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilter.java @@ -25,6 +25,7 @@ import java.io.IOException; import java.net.URI; import java.net.URISyntaxException; +import java.util.ArrayList; import java.util.List; import java.util.Optional; import jakarta.annotation.Priority; @@ -194,7 +195,8 @@ public OntModel getOntology(Dataspace app, String uri) * ontapi resolves it — along with its own transitive imports — as part of the closure, through the * same scoped repository view as every other import. The declaration is derived from the * application's ldh:import data on every load and never persisted, so the ldh:import triples remain - * the single source of truth. A package ontology that cannot be resolved is skipped so a broken + * the single source of truth. A package ontology that cannot be resolved is skipped, and a closure + * that cannot be assembled with the package ontologies is assembled without them, so a broken * package cannot take the application ontology down. * * @param repository graph repository @@ -204,9 +206,20 @@ public OntModel getOntology(Dataspace app, String uri) */ public static UnionGraph loadOntology(PrefixGraphRepository repository, String uri, List packageOntologies) { - if (!packageOntologies.isEmpty()) declarePackageImports(repository, uri, packageOntologies); + if (packageOntologies.isEmpty()) return loadOntology(repository, uri); - return loadOntology(repository, uri); + List imports = declarePackageImports(repository, uri, packageOntologies); + try + { + return loadOntology(repository, uri); + } + catch (RuntimeException ex) // e.g. ontapi refusing a package graph that collides with another in the closure + { + if (log.isErrorEnabled()) log.error("Could not assemble ontology '{}' with package ontologies {}, assembling it without them", uri, packageOntologies, ex); + Graph base = repository.get(uri); + imports.forEach(base::delete); + return loadOntology(repository, uri); + } } /** @@ -220,15 +233,17 @@ public static UnionGraph loadOntology(PrefixGraphRepository repository, String u * @param repository graph repository * @param uri ontology URI * @param packageOntologies package ontology URIs + * @return the owl:imports triples added to the base graph */ - public static void declarePackageImports(PrefixGraphRepository repository, String uri, List packageOntologies) + public static List declarePackageImports(PrefixGraphRepository repository, String uri, List packageOntologies) { + List imports = new ArrayList<>(); Graph base = repository.get(uri); Optional name = Graphs.findOntologyNameNode(base); if (name.isEmpty()) { if (log.isErrorEnabled()) log.error("Ontology with URI '{}' carries no ontology header, cannot import packages {} into it", uri, packageOntologies); - return; + return imports; } for (URI packageOntology : packageOntologies) @@ -236,33 +251,46 @@ public static void declarePackageImports(PrefixGraphRepository repository, Strin // the model is constructed with ignoreUnresolvedImports, which silently substitutes an empty // graph for an import it cannot resolve — resolve it here so that a broken package is reported // instead of composing as nothing - if (!isResolvable(repository, packageOntology.toString())) + Graph packageGraph = resolve(repository, packageOntology.toString()); + if (packageGraph == null) { if (log.isErrorEnabled()) log.error("Could not load package ontology '{}', skipping it", packageOntology); continue; } - base.add(Triple.create(name.get(), OWL.imports.asNode(), NodeFactory.createURI(packageOntology.toString()))); + // import the ontology IRI the graph declares, not the URI it was resolved from: a package may name + // the document (ns/) while its ontology is ns/#, and ontapi, which keys the closure by ontology IRI, + // would resolve the IRI again into a second graph with the same name and refuse it + Node packageName = Graphs.findOntologyNameNode(packageGraph).filter(Node::isURI). + orElse(NodeFactory.createURI(packageOntology.toString())); + Triple declaration = Triple.create(name.get(), OWL.imports.asNode(), packageName); + if (!base.contains(declaration)) + { + base.add(declaration); + imports.add(declaration); + } } + + return imports; } /** - * Returns true if the repository can supply a graph for the given ID. + * Returns the graph the repository supplies for the given ID, or null if it cannot supply one. * * @param repository graph repository * @param id graph ID - * @return true if resolvable + * @return graph or null */ - public static boolean isResolvable(PrefixGraphRepository repository, String id) + public static Graph resolve(PrefixGraphRepository repository, String id) { try { - return repository.get(id) != null; + return repository.get(id); } catch (RuntimeException ex) // unmapped location, 404, connection refused, unparseable document... { if (log.isDebugEnabled()) log.debug("Could not resolve graph with ID '{}'", id, ex); - return false; + return null; } } diff --git a/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/ns.ttl b/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/ns.ttl deleted file mode 100644 index 11ee744e4..000000000 --- a/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/ns.ttl +++ /dev/null @@ -1,318 +0,0 @@ -@prefix : <#> . -@prefix ldh: . -@prefix ac: . -@prefix rdfs: . -@prefix owl: . -@prefix sp: . -@prefix spin: . -@prefix dct: . -@prefix skos: . - -: a owl:Ontology ; - owl:imports ; - rdfs:label "SKOS package ontology". - -# Concept - -skos:Concept spin:constructor :ConceptConstructor ; - spin:constraint :MissingPrefLabel , :MissingInScheme . - -# Every natural-language value the constructors below construct - the labels, the title, the -# definitions and the description - is declared rdf:langString, not xsd:string: such a value is -# natural language, and SKOS's own integrity rules assume it carries a tag (a concept may have only -# one prefLabel PER LANGUAGE, which is not even expressible without one). A constructor declaring -# xsd:string produced a form field with a datatype and no language control, so a label added through -# the form came out untagged beside the tagged ones already in the data - visible as a value with an -# xsd:string annotation next to one showing en. -:ConceptConstructor a ldh:Constructor ; - rdfs:label "Concept constructor" ; - dct:title "Concept constructor" ; - sp:text """ - PREFIX skos: - PREFIX rdf: - - CONSTRUCT { - $this skos:prefLabel [ a rdf:langString ] ; - skos:altLabel [ a rdf:langString ] ; - skos:definition [ a rdf:langString ] ; - skos:scopeNote [ a rdf:langString ] ; - skos:broader [ a skos:Concept ] ; - skos:narrower [ a skos:Concept ] ; - skos:related [ a skos:Concept ] ; - skos:inScheme [ a skos:ConceptScheme ] ; - skos:topConceptOf [ a skos:ConceptScheme ] . - } - WHERE {}""" ; - rdfs:isDefinedBy : . - -# The prefLabel every view below joins is a SORT KEY, not a selector: it is never projected, so -# DISTINCT keeps one row per concept whatever languages it carries. These queries used to filter it to -# English or untagged, which silently dropped every concept labelled in another language - and since -# the form's language control follows the browser, a concept created through the UI lands in exactly -# that state. Measured on the fixture: Juice's Broader block showed Cold drinks but not the concept -# just created above it, while that concept's Narrower block did show Juice, which also carries an en -# label. Which language a reader wants is decided at render time, where the label chain already picks -# per value; a stored query has no way to know it. The join stays, because a concept with no prefLabel -# at all has no sort key and the package makes one mandatory anyway (:MissingPrefLabel), and the order -# among a multi-label concept's own labels is left to the engine - that is what an unprojected sort -# key means. - -# narrower - -skos:narrower ldh:view :NarrowerConcepts . - -:NarrowerConcepts a ldh:View ; - dct:title "Narrower concepts" ; - spin:query :SelectNarrowerConcepts ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectNarrowerConcepts a sp:Select ; - rdfs:label "Select narrower concepts" ; - dct:title "Select narrower concepts" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?narrower -WHERE - { GRAPH ?graph - { { $about skos:narrower ?narrower } - UNION - { ?narrower skos:broader $about } - GRAPH ?narrowerGraph - { ?narrower skos:prefLabel ?prefLabel } - } - } -ORDER BY ?prefLabel -""" ; - rdfs:isDefinedBy : . - -# broader - -skos:broader ldh:view :BroaderConcepts . - -:BroaderConcepts a ldh:View ; - dct:title "Broader concepts" ; - spin:query :SelectBroaderConcepts ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectBroaderConcepts a sp:Select ; - rdfs:label "Select broader concepts" ; - dct:title "Select broader concepts" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?broader -WHERE - { GRAPH ?graph - { { $about skos:broader ?broader } - UNION - { ?broader skos:narrower $about } - GRAPH ?broaderGraph - { ?broader skos:prefLabel ?prefLabel } - } - } -ORDER BY ?prefLabel -""" ; - rdfs:isDefinedBy : . - -# Collection - -skos:Collection spin:constructor :CollectionConstructor ; - spin:constraint :MissingPrefLabel . - -:CollectionConstructor a ldh:Constructor ; - rdfs:label "Collection constructor" ; - dct:title "Collection constructor" ; - sp:text """ - PREFIX skos: - PREFIX rdf: - PREFIX xsd: - - CONSTRUCT { - $this skos:prefLabel [ a rdf:langString ] ; - skos:altLabel [ a rdf:langString ] ; - skos:definition [ a rdf:langString ] ; - skos:notation [ a xsd:string ] ; - skos:member [ a skos:Concept ] ; - skos:inScheme [ a skos:ConceptScheme ] . - } - WHERE {}""" ; - rdfs:isDefinedBy : . - -skos:member ldh:view :CollectionMembers . - -:CollectionMembers a ldh:View ; - dct:title "Collection members" ; - spin:query :SelectCollectionMembers ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectCollectionMembers a sp:Select ; - rdfs:label "Select collection members" ; - dct:title "Select collection members" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?member -WHERE - { GRAPH ?graph - { $about skos:member ?member . - GRAPH ?memberGraph - { ?member skos:prefLabel ?prefLabel } - } - } -ORDER BY ?prefLabel -""" ; - rdfs:isDefinedBy : . - -# ConceptScheme - -skos:ConceptScheme spin:constructor :ConceptSchemeConstructor . - -:ConceptSchemeConstructor a ldh:Constructor ; - rdfs:label "Concept scheme constructor" ; - dct:title "Concept scheme constructor" ; - sp:text """ - PREFIX skos: - PREFIX rdf: - PREFIX dct: - - CONSTRUCT { - $this dct:title [ a rdf:langString ] ; - dct:description [ a rdf:langString ] ; - skos:hasTopConcept [ a skos:Concept ] . - } - WHERE {}""" ; - rdfs:isDefinedBy : . - -skos:hasTopConcept ldh:view :TopConcepts . - -:TopConcepts a ldh:View ; - dct:title "Top concepts" ; - spin:query :SelectTopConcepts ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -# the entry level of a taxonomy, and the root of the concept tree. Both directions count as a top -# Every hierarchy query below reads BOTH directions, for the same reason: SKOS declares -# skos:broader/skos:narrower and skos:hasTopConcept/skos:topConceptOf as inverse pairs and leaves the -# choice to the modeller, so a parent may point down or a child may point up and both are correct. -# Reading one direction made a view disagree with the concept tree, which unions them: measured on the -# fixture, Hot drinks' Narrower block showed 1 child where the tree showed 3, because Coffee and Tea -# assert skos:broader upwards, and Juice's Broader block showed nothing while the tree nested it under -# Hot drinks. The union stays INSIDE the link's GRAPH rather than becoming a sibling block, because -# SPARQLBuilder's round-trip merges sibling GRAPH blocks into one and would scope the child's label to -# the link's document. -:SelectTopConcepts a sp:Select ; - rdfs:label "Select top concepts" ; - dct:title "Select top concepts" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?concept -WHERE - { GRAPH ?graph - { { $about skos:hasTopConcept ?concept } - UNION - { ?concept skos:topConceptOf $about } - } - } -""" ; - rdfs:isDefinedBy : . - -skos:inScheme ldh:view :ConceptScheme . - -:ConceptScheme a ldh:View ; - dct:title "Scheme" ; - spin:query :SelectConceptScheme ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectConceptScheme a sp:Select ; - rdfs:label "Select the scheme a concept belongs to" ; - dct:title "Select the scheme a concept belongs to" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?scheme -WHERE - { GRAPH ?graph - { $about skos:inScheme ?scheme } - } -""" ; - rdfs:isDefinedBy : . - -skos:inScheme ldh:inverseView :ConceptsInScheme , :OrphanConcepts . - -:ConceptsInScheme a ldh:View ; - dct:title "Concepts in scheme" ; - spin:query :SelectConceptsInScheme ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectConceptsInScheme a sp:Select ; - rdfs:label "Select concepts in scheme" ; - dct:title "Select concepts in scheme" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?concept -WHERE - { GRAPH ?graph - { ?concept skos:inScheme $about ; - skos:prefLabel ?prefLabel - } - } -ORDER BY ?prefLabel -""" ; - rdfs:isDefinedBy : . - -# Concepts a scheme lists but nothing places in its hierarchy: neither a top concept nor narrower -# than anything. A taxonomy with orphans is not wrong, but they are invisible to a tree that -# descends from the top, so the scheme surfaces them rather than letting them go unreachable -:OrphanConcepts a ldh:View ; - dct:title "Unplaced concepts" ; - spin:query :SelectOrphanConcepts ; - ac:mode ac:TableMode ; - ldh:showWhenEmpty false ; - rdfs:isDefinedBy : . - -:SelectOrphanConcepts a sp:Select ; - rdfs:label "Select concepts with no place in the hierarchy" ; - dct:title "Select concepts with no place in the hierarchy" ; - sp:text """ -PREFIX skos: - -SELECT DISTINCT ?concept -WHERE - { GRAPH ?graph - { ?concept skos:inScheme $about } - FILTER NOT EXISTS { GRAPH ?topGraph { $about skos:hasTopConcept ?concept } } - FILTER NOT EXISTS { GRAPH ?topGraph { ?concept skos:topConceptOf $about } } - FILTER NOT EXISTS { GRAPH ?broaderGraph { ?concept skos:broader ?parent } } - FILTER NOT EXISTS { GRAPH ?narrowerGraph { ?parent skos:narrower ?concept } } - } -""" ; - rdfs:isDefinedBy : . - -# Constraints - -:MissingPrefLabel a ldh:MissingPropertyValue ; - rdfs:label "Missing skos:prefLabel" ; - sp:arg1 skos:prefLabel ; - rdfs:isDefinedBy : . - -# a concept outside every scheme cannot be reached from a scheme's tree, so the package treats -# skos:inScheme as mandatory even though SKOS itself does not -:MissingInScheme a ldh:MissingPropertyValue ; - rdfs:label "Missing skos:inScheme" ; - sp:arg1 skos:inScheme ; - rdfs:isDefinedBy : . diff --git a/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/package.ttl b/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/package.ttl deleted file mode 100644 index 3e695b39b..000000000 --- a/src/main/resources/com/linkeddatahub/packages/editor/taxonomy/package.ttl +++ /dev/null @@ -1,21 +0,0 @@ -@base . -@prefix : <#> . -@prefix lds: . -@prefix ac: . -@prefix rdfs: . -@prefix dct: . -@prefix foaf: . - - a lds:Package ; - rdfs:label "Taxonomy Editor" ; - dct:title "Taxonomy Editor for LinkedDataHub" ; - dct:description "Turns a dataspace into a taxonomy editor: SKOS concepts, schemes and collections gain a concept tree beside the content, hierarchy views that read both assertion directions, and constructors and constraints that keep a concept labelled and in a scheme." ; - dct:creator ; - lds:ontology ; - # the published copy, which the application takes its own copy of at import and then serves from - # its own origin - so what a running instance compiles cannot change under it afterwards - ac:stylesheet . - - a foaf:Organization ; - foaf:name "AtomGraph" ; - foaf:homepage . diff --git a/src/main/resources/com/linkeddatahub/packages/packages.ttl b/src/main/resources/com/linkeddatahub/packages/packages.ttl deleted file mode 100644 index e6ac3f40a..000000000 --- a/src/main/resources/com/linkeddatahub/packages/packages.ttl +++ /dev/null @@ -1,12 +0,0 @@ -@base . -@prefix rdfs: . -@prefix dct: . -@prefix foaf: . - - a foaf:Document ; - dct:title "LinkedDataHub packages" ; - dct:description "Catalog of packages available for LinkedDataHub applications" ; - rdfs:member . - - dct:title "Taxonomy Editor" ; - dct:description "Taxonomy editing on SKOS: a concept tree beside the content, hierarchy views in both assertion directions, and constructors and constraints for concepts, schemes and collections" . diff --git a/src/main/resources/prefix-mapping.ttl b/src/main/resources/prefix-mapping.ttl index 3c982e15c..b984c41d6 100644 --- a/src/main/resources/prefix-mapping.ttl +++ b/src/main/resources/prefix-mapping.ttl @@ -35,12 +35,5 @@ [ lm:prefix "http://www.w3.org/2004/02/skos/core" ; lm:altName "com/atomgraph/client/skos.owl" ] , [ lm:prefix "http://usefulinc.com/ns/doap" ; lm:altName "com/atomgraph/client/doap.owl" ] , [ lm:prefix "http://www.w3.org/2011/http" ; lm:altName "com/atomgraph/linkeddatahub/http.owl" ] , - [ lm:prefix "http://www.w3.org/2011/http-statusCodes" ; lm:altName "com/atomgraph/linkeddatahub/http-statusCodes.rdf" ] , - - # packages - the bundled copies stand in until they are served from packages.linkeddatahub.com. - # These are a fallback, not an override: OntologyRepository asks the store before it consults a - # mapping, so an application's own materialized copy of a package ontology wins over the shipped file - [ lm:prefix "https://packages.linkeddatahub.com/" ; lm:altName "com/linkeddatahub/packages/packages.ttl" ] , - [ lm:prefix "https://packages.linkeddatahub.com/editor/taxonomy/" ; lm:altName "com/linkeddatahub/packages/editor/taxonomy/package.ttl" ] , - [ lm:prefix "https://raw.githubusercontent.com/AtomGraph/LinkedDataHub-Apps/refs/heads/develop/packages/editor/taxonomy/ns.ttl" ; lm:altName "com/linkeddatahub/packages/editor/taxonomy/ns.ttl" ] + [ lm:prefix "http://www.w3.org/2011/http-statusCodes" ; lm:altName "com/atomgraph/linkeddatahub/http-statusCodes.rdf" ] . diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/packages.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/packages.xsl index eb9789fe2..c5c4de7ec 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/packages.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client/packages.xsl @@ -29,8 +29,7 @@ version="3.0" next request. The package checkboxes are RDF/POST inputs in the settings form, so its Save submits the ldh:import triples together with the other settings. --> - + diff --git a/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilterTest.java b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilterTest.java index 9478f75d2..a68f64645 100644 --- a/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilterTest.java +++ b/src/test/java/com/atomgraph/linkeddatahub/server/filter/request/OntologyFilterTest.java @@ -17,8 +17,17 @@ package com.atomgraph.linkeddatahub.server.filter.request; import com.atomgraph.client.util.jena.PrefixGraphRepository; +import java.net.URI; +import java.util.List; import org.apache.jena.graph.Graph; +import org.apache.jena.graph.Node; +import org.apache.jena.graph.NodeFactory; +import org.apache.jena.graph.Triple; +import org.apache.jena.ontapi.UnionGraph; import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.vocabulary.OWL; +import org.apache.jena.vocabulary.RDF; +import org.apache.jena.vocabulary.RDFS; import org.junit.jupiter.api.Test; import static org.junit.jupiter.api.Assertions.*; @@ -65,4 +74,58 @@ public void testAddDocumentModelSkipsWhenDocumentURIMapped() assertFalse(repository.isCached(docURI), "mapped document URI should not be cached as a secondary key"); } + /** + * A package descriptor may name the ontology's document (ns/) while the ontology inside is ns/#. The + * two URIs resolve to two graphs with the same ontology name; importing the declared name keeps the + * second one out of the closure. + */ + @Test + public void testPackageImportUsesDeclaredOntologyIRI() + { + PrefixGraphRepository repository = new PrefixGraphRepository(null); + repository.put(APP, ontology(APP)); + repository.put("http://example.org/pkg/ns/", ontology("http://example.org/pkg/ns/#", LABELLED)); + repository.put("http://example.org/pkg/ns/#", ontology("http://example.org/pkg/ns/#", LABELLED)); // a separate load of the same document + + UnionGraph union = OntologyFilter.loadOntology(repository, APP, List.of(URI.create("http://example.org/pkg/ns/"))); + + assertTrue(union.contains(LABELLED), "package ontology should be in the closure"); + } + + /** A package ontology that breaks the closure is left out of it, and the application ontology still loads. */ + @Test + public void testCollidingPackageFallsBackToApplicationOntology() + { + PrefixGraphRepository repository = new PrefixGraphRepository(null); + Graph app = ontology(APP); + repository.put(APP, app); + Graph pkg = ontology("http://example.org/pkg#", LABELLED); + pkg.add(Triple.create(uri("http://example.org/pkg#"), OWL.imports.asNode(), uri("http://example.org/alias"))); + repository.put("http://example.org/pkg#", pkg); + repository.put("http://example.org/alias", ontology("http://example.org/pkg#")); // another graph with the package's name + + UnionGraph union = OntologyFilter.loadOntology(repository, APP, List.of(URI.create("http://example.org/pkg#"))); + + assertTrue(union.contains(uri(APP), RDF.type.asNode(), OWL.Ontology.asNode()), "application ontology should load"); + assertFalse(union.contains(LABELLED), "the colliding package should be left out"); + assertFalse(app.contains(uri(APP), OWL.imports.asNode(), uri("http://example.org/pkg#")), "the package import should be retracted"); + } + + private static final String APP = "http://example.org/app#"; + + private static final Triple LABELLED = Triple.create(uri("http://example.org/pkg/ns/#Thing"), RDFS.label.asNode(), NodeFactory.createLiteralString("Thing")); + + private static Graph ontology(String name, Triple... triples) + { + Graph graph = ModelFactory.createDefaultModel().getGraph(); + graph.add(Triple.create(uri(name), RDF.type.asNode(), OWL.Ontology.asNode())); + for (Triple triple : triples) graph.add(triple); + return graph; + } + + private static Node uri(String uri) + { + return NodeFactory.createURI(uri); + } + } From 71412f239ee98191c92852aac1f180bc42f06b05 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 21:20:53 +0200 Subject: [PATCH 12/16] The package tests import from a registry in the test stack instead of https://packages.linkeddatahub.com/, so they depend on neither its reachability nor its current content; the materialization test failed when the published descriptor still named its ontology while the test pinned the bundled copy's IRI. tests/http/config declares a packages.localhost:4443 dataspace on the root's datasets, as test.localhost is; tests/packages holds a copy of the taxonomy editor package and publish.sh, which pushes it and declares its stylesheet by the URI of the upload the push made; run.sh publishes it before the datasets are stored, so every reset restores it, and exports PACKAGES_BASE_URL, which the three package tests import from. The UI suite publishes the same registry onto packages. and imports from it unless UI_TESTS_TAXONOMY_PACKAGE names another package. ldh uploads .xsl and .xslt as text/xsl, which Files.probeContentType does not detect: a stylesheet uploaded as application/octet-stream answers the platform's text/xsl request with 406. Co-Authored-By: Claude Fable 5.1 --- .../linkeddatahub/cli/command/AddFile.java | 15 + .../cli/command/AddFileMultiPartTest.java | 12 + tests/http/config/dataspaces.trig | 24 + tests/http/config/system.trig | 40 ++ .../misc/PATCH-settings-package-import.sh | 2 +- .../PATCH-settings-package-materialization.sh | 5 +- .../misc/PATCH-settings-package-ontology.sh | 2 +- tests/http/run.sh | 4 + tests/packages/.ldhignore | 1 + tests/packages/editor.ttl | 18 + tests/packages/editor/taxonomy.ttl | 31 ++ tests/packages/editor/taxonomy/ns.ttl | 326 +++++++++++++ tests/packages/editor/taxonomy/skos.xsl | 448 ++++++++++++++++++ tests/packages/publish.sh | 28 ++ tests/packages/root.ttl | 27 ++ tests/ui/README.md | 3 +- tests/ui/lib/taxonomy.mjs | 21 +- 17 files changed, 998 insertions(+), 9 deletions(-) create mode 100644 tests/packages/.ldhignore create mode 100644 tests/packages/editor.ttl create mode 100644 tests/packages/editor/taxonomy.ttl create mode 100644 tests/packages/editor/taxonomy/ns.ttl create mode 100644 tests/packages/editor/taxonomy/skos.xsl create mode 100755 tests/packages/publish.sh create mode 100644 tests/packages/root.ttl diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java index bae06e6f4..7872d4529 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java @@ -28,6 +28,8 @@ import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; +import java.util.Locale; +import java.util.Map; import org.apache.jena.vocabulary.DCTerms; import org.apache.jena.vocabulary.RDF; import org.glassfish.jersey.media.multipart.FormDataMultiPart; @@ -145,8 +147,21 @@ public static FormDataMultiPart buildMultiPart(Path file, String contentType, St return multiPart; } + /** + * Media types of the file extensions the platform serves by type but the JDK does not detect: a + * package stylesheet uploaded as application/octet-stream answers the platform's + * text/xsl request with 406. + */ + static final Map EXTENSION_TYPES = Map.of( + "xsl", "text/xsl", + "xslt", "text/xsl"); + static String detectContentType(Path file) throws IOException { + String name = file.getFileName().toString(); + String extension = name.substring(name.lastIndexOf('.') + 1).toLowerCase(Locale.ROOT); + if (name.contains(".") && EXTENSION_TYPES.containsKey(extension)) return EXTENSION_TYPES.get(extension); + String detected = Files.probeContentType(file); return detected != null ? detected : "application/octet-stream"; } diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/AddFileMultiPartTest.java b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/AddFileMultiPartTest.java index a40c4ae8f..898f1afb8 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/AddFileMultiPartTest.java +++ b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/AddFileMultiPartTest.java @@ -68,4 +68,16 @@ public void descriptionAppendsTrailingPair() throws Exception } } + @Test + public void stylesheetIsDetectedAsXSL() throws Exception + { + for (String name : List.of("skos.xsl", "Skos.XSLT")) + { + Path file = tempDir.resolve(name); + Files.writeString(file, ""); + + assertEquals("text/xsl", AddFile.detectContentType(file), name); + } + } + } diff --git a/tests/http/config/dataspaces.trig b/tests/http/config/dataspaces.trig index ab616a366..3fa2f7b04 100644 --- a/tests/http/config/dataspaces.trig +++ b/tests/http/config/dataspaces.trig @@ -55,3 +55,27 @@ ac:stylesheet ; lds:public true . } + +# packages admin + + +{ + a lds:Dataspace ; + dct:title "Packages admin" ; + lds:origin ; + lds:ontology ; + ac:stylesheet . +} + +# packages end-user: the package registry the package tests import from, so that they depend on neither the +# reachability nor the current content of https://packages.linkeddatahub.com/ (run.sh publishes tests/packages) + + +{ + a lds:Dataspace ; + dct:title "Packages" ; + lds:origin ; + lds:ontology ; + ac:stylesheet ; + lds:public true . +} diff --git a/tests/http/config/system.trig b/tests/http/config/system.trig index 25f6cc926..f4826bb9e 100644 --- a/tests/http/config/system.trig +++ b/tests/http/config/system.trig @@ -86,3 +86,43 @@ a:graphStore ; a:quadStore . } + +# packages admin - type + service binding + + +{ + a lds:AdminDataspace ; + lds:service . +} + +# packages admin - service description + + +{ + a sd:Service ; + dct:title "Packages admin service" ; + sd:supportedLanguage sd:SPARQL11Query, sd:SPARQL11Update ; + sd:endpoint ; + a:graphStore ; + a:quadStore . +} + +# packages end-user - type + service binding + + +{ + a lds:EndUserDataspace ; + lds:service . +} + +# packages end-user - service description + + +{ + a sd:Service ; + dct:title "Packages service" ; + sd:supportedLanguage sd:SPARQL11Query, sd:SPARQL11Update ; + sd:endpoint ; + a:graphStore ; + a:quadStore . +} diff --git a/tests/http/misc/PATCH-settings-package-import.sh b/tests/http/misc/PATCH-settings-package-import.sh index 74e6350b2..5da7c66f5 100755 --- a/tests/http/misc/PATCH-settings-package-import.sh +++ b/tests/http/misc/PATCH-settings-package-import.sh @@ -14,7 +14,7 @@ clear_ontology # The update strings mirror the ones generated by the settings modal's package Save button. app_uri="urn:linkeddatahub:apps/end-user" -package_uri="https://packages.linkeddatahub.com/editor/taxonomy/#this" +package_uri="${PACKAGES_BASE_URL}editor/taxonomy/#this" # the fixture registry, tests/packages slug="taxonomy-package-probe" doc="${END_USER_BASE_URL}${slug}/" diff --git a/tests/http/misc/PATCH-settings-package-materialization.sh b/tests/http/misc/PATCH-settings-package-materialization.sh index 5b4867065..8e6b0236f 100755 --- a/tests/http/misc/PATCH-settings-package-materialization.sh +++ b/tests/http/misc/PATCH-settings-package-materialization.sh @@ -14,8 +14,9 @@ clear_ontology # the copy is what makes the package's constructors editable rather than merely visible. app_uri="urn:linkeddatahub:apps/end-user" -package_uri="https://packages.linkeddatahub.com/editor/taxonomy/#this" -package_ontology="https://raw.githubusercontent.com/AtomGraph/LinkedDataHub-Apps/refs/heads/develop/packages/editor/taxonomy/ns.ttl#" +package_uri="${PACKAGES_BASE_URL}editor/taxonomy/#this" # the fixture registry, tests/packages +# the ontology IRI ns.ttl declares, served by the registry as the ns/ document +package_ontology="${PACKAGES_BASE_URL}editor/taxonomy/ns/#" # the document URI is derived from the package URI's path, so it is predictable rather than a digest doc="${ADMIN_BASE_URL}ontologies/editor-taxonomy/" diff --git a/tests/http/misc/PATCH-settings-package-ontology.sh b/tests/http/misc/PATCH-settings-package-ontology.sh index f99518787..e51aaa703 100755 --- a/tests/http/misc/PATCH-settings-package-ontology.sh +++ b/tests/http/misc/PATCH-settings-package-ontology.sh @@ -14,7 +14,7 @@ clear_ontology # visible on the /ns endpoint after the PATCH and disappears again after removal. app_uri="urn:linkeddatahub:apps/end-user" -package_uri="https://packages.linkeddatahub.com/editor/taxonomy/#this" +package_uri="${PACKAGES_BASE_URL}editor/taxonomy/#this" # the fixture registry, tests/packages query='SELECT ?text WHERE { ?constructor . ?constructor ?text . }' diff --git a/tests/http/run.sh b/tests/http/run.sh index 86925cf14..08cbd2932 100755 --- a/tests/http/run.sh +++ b/tests/http/run.sh @@ -333,6 +333,7 @@ export END_USER_ENDPOINT_URL="http://localhost:3030/end-user/" export ADMIN_ENDPOINT_URL="http://localhost:3030/admin/" export END_USER_BASE_URL="https://localhost:4443/" export ADMIN_BASE_URL="https://admin.localhost:4443/" +export PACKAGES_BASE_URL="https://packages.localhost:4443/" # the fixture package registry, tests/packages export END_USER_VARNISH_SERVICE="varnish-end-user" export ADMIN_VARNISH_SERVICE="varnish-admin" export FRONTEND_VARNISH_SERVICE="varnish-frontend" @@ -352,6 +353,9 @@ run_tests "signup" "signup.sh" export AGENT_URI="$(webid-uri.sh "$AGENT_CERT_FILE")" printf "### Signed up agent URI: %s\n" "$AGENT_URI" +# publish the fixture package registry before the datasets are stored, so that every test's reset restores it +"$HTTP_TEST_ROOT/../packages/publish.sh" "$PACKAGES_BASE_URL" "$OWNER_CERT_KEYSTORE" "$OWNER_CERT_PWD" + # store the end-user and admin datasets export TMP_END_USER_DATASET=$(mktemp) export TMP_ADMIN_DATASET=$(mktemp) diff --git a/tests/packages/.ldhignore b/tests/packages/.ldhignore new file mode 100644 index 000000000..305fa74dd --- /dev/null +++ b/tests/packages/.ldhignore @@ -0,0 +1 @@ +publish.sh diff --git a/tests/packages/editor.ttl b/tests/packages/editor.ttl new file mode 100644 index 000000000..b32ad7970 --- /dev/null +++ b/tests/packages/editor.ttl @@ -0,0 +1,18 @@ +@prefix ldh: . +@prefix rdf: . +@prefix dh: . +@prefix dct: . + +<> a dh:Container ; + dct:title "Editors" ; + dct:description "Packages that turn a dataspace into an editor for a kind of data, each one pairing an ontology with the stylesheet that renders it." ; + rdf:_1 <#content> ; + rdf:_2 <#select-children> . + +<#content> a ldh:XHTML ; + rdf:value """

+

Packages that turn a dataspace into an editor for a kind of data, each one pairing an ontology with the stylesheet that renders it.

+
"""^^rdf:XMLLiteral . + +<#select-children> a ldh:Object ; + rdf:value ldh:ChildrenView . diff --git a/tests/packages/editor/taxonomy.ttl b/tests/packages/editor/taxonomy.ttl new file mode 100644 index 000000000..460c24cb4 --- /dev/null +++ b/tests/packages/editor/taxonomy.ttl @@ -0,0 +1,31 @@ +@prefix lds: . +@prefix ldh: . +@prefix rdf: . +@prefix dh: . +@prefix dct: . +@prefix foaf: . + +<> a dh:Container ; + dct:title "Taxonomy" ; + dct:description "Taxonomy editing on SKOS: a concept tree beside the content, hierarchy views in both assertion directions, and constructors and constraints for concepts, schemes and collections." ; + foaf:primaryTopic <#this> ; + rdf:_1 <#content> ; + rdf:_2 <#select-children> . + +<#content> a ldh:XHTML ; + rdf:value """
+

Taxonomy editing on SKOS: a concept tree beside the content, hierarchy views in both assertion directions, and constructors and constraints for concepts, schemes and collections.

+
"""^^rdf:XMLLiteral . + +<#select-children> a ldh:Object ; + rdf:value ldh:ChildrenView . + +<#this> a lds:Package ; + dct:title "Taxonomy Editor" ; + dct:description "Turns a dataspace into a taxonomy editor: SKOS concepts, schemes and collections gain a concept tree beside the content, hierarchy views that read both assertion directions, and constructors and constraints that keep a concept labelled and in a scheme." ; + dct:creator ; + lds:ontology . + + a foaf:Organization ; + foaf:name "AtomGraph" ; + foaf:homepage . diff --git a/tests/packages/editor/taxonomy/ns.ttl b/tests/packages/editor/taxonomy/ns.ttl new file mode 100644 index 000000000..2e98c7bc3 --- /dev/null +++ b/tests/packages/editor/taxonomy/ns.ttl @@ -0,0 +1,326 @@ +@prefix : <#> . +@prefix ldh: . +@prefix ac: . +@prefix rdfs: . +@prefix owl: . +@prefix sp: . +@prefix spin: . +@prefix dct: . +@prefix skos: . +@prefix dh: . +@prefix foaf: . + +# the document the registry serves this file as (ns/), which is what the package's lds:ontology dereferences to + +<> a dh:Item ; + dct:title "Taxonomy Editor ontology" ; + foaf:primaryTopic : . + +: a owl:Ontology ; + owl:imports ; + rdfs:label "SKOS package ontology". + +# Concept + +skos:Concept spin:constructor :ConceptConstructor ; + spin:constraint :MissingPrefLabel , :MissingInScheme . + +# Every natural-language value the constructors below construct - the labels, the title, the +# definitions and the description - is declared rdf:langString, not xsd:string: such a value is +# natural language, and SKOS's own integrity rules assume it carries a tag (a concept may have only +# one prefLabel PER LANGUAGE, which is not even expressible without one). A constructor declaring +# xsd:string produced a form field with a datatype and no language control, so a label added through +# the form came out untagged beside the tagged ones already in the data - visible as a value with an +# xsd:string annotation next to one showing en. +:ConceptConstructor a ldh:Constructor ; + rdfs:label "Concept constructor" ; + dct:title "Concept constructor" ; + sp:text """ + PREFIX skos: + PREFIX rdf: + + CONSTRUCT { + $this skos:prefLabel [ a rdf:langString ] ; + skos:altLabel [ a rdf:langString ] ; + skos:definition [ a rdf:langString ] ; + skos:scopeNote [ a rdf:langString ] ; + skos:broader [ a skos:Concept ] ; + skos:narrower [ a skos:Concept ] ; + skos:related [ a skos:Concept ] ; + skos:inScheme [ a skos:ConceptScheme ] ; + skos:topConceptOf [ a skos:ConceptScheme ] . + } + WHERE {}""" ; + rdfs:isDefinedBy : . + +# The prefLabel every view below joins is a SORT KEY, not a selector: it is never projected, so +# DISTINCT keeps one row per concept whatever languages it carries. These queries used to filter it to +# English or untagged, which silently dropped every concept labelled in another language - and since +# the form's language control follows the browser, a concept created through the UI lands in exactly +# that state. Measured on the fixture: Juice's Broader block showed Cold drinks but not the concept +# just created above it, while that concept's Narrower block did show Juice, which also carries an en +# label. Which language a reader wants is decided at render time, where the label chain already picks +# per value; a stored query has no way to know it. The join stays, because a concept with no prefLabel +# at all has no sort key and the package makes one mandatory anyway (:MissingPrefLabel), and the order +# among a multi-label concept's own labels is left to the engine - that is what an unprojected sort +# key means. + +# narrower + +skos:narrower ldh:view :NarrowerConcepts . + +:NarrowerConcepts a ldh:View ; + dct:title "Narrower concepts" ; + spin:query :SelectNarrowerConcepts ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectNarrowerConcepts a sp:Select ; + rdfs:label "Select narrower concepts" ; + dct:title "Select narrower concepts" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?narrower +WHERE + { GRAPH ?graph + { { $about skos:narrower ?narrower } + UNION + { ?narrower skos:broader $about } + GRAPH ?narrowerGraph + { ?narrower skos:prefLabel ?prefLabel } + } + } +ORDER BY ?prefLabel +""" ; + rdfs:isDefinedBy : . + +# broader + +skos:broader ldh:view :BroaderConcepts . + +:BroaderConcepts a ldh:View ; + dct:title "Broader concepts" ; + spin:query :SelectBroaderConcepts ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectBroaderConcepts a sp:Select ; + rdfs:label "Select broader concepts" ; + dct:title "Select broader concepts" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?broader +WHERE + { GRAPH ?graph + { { $about skos:broader ?broader } + UNION + { ?broader skos:narrower $about } + GRAPH ?broaderGraph + { ?broader skos:prefLabel ?prefLabel } + } + } +ORDER BY ?prefLabel +""" ; + rdfs:isDefinedBy : . + +# Collection + +skos:Collection spin:constructor :CollectionConstructor ; + spin:constraint :MissingPrefLabel . + +:CollectionConstructor a ldh:Constructor ; + rdfs:label "Collection constructor" ; + dct:title "Collection constructor" ; + sp:text """ + PREFIX skos: + PREFIX rdf: + PREFIX xsd: + + CONSTRUCT { + $this skos:prefLabel [ a rdf:langString ] ; + skos:altLabel [ a rdf:langString ] ; + skos:definition [ a rdf:langString ] ; + skos:notation [ a xsd:string ] ; + skos:member [ a skos:Concept ] ; + skos:inScheme [ a skos:ConceptScheme ] . + } + WHERE {}""" ; + rdfs:isDefinedBy : . + +skos:member ldh:view :CollectionMembers . + +:CollectionMembers a ldh:View ; + dct:title "Collection members" ; + spin:query :SelectCollectionMembers ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectCollectionMembers a sp:Select ; + rdfs:label "Select collection members" ; + dct:title "Select collection members" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?member +WHERE + { GRAPH ?graph + { $about skos:member ?member . + GRAPH ?memberGraph + { ?member skos:prefLabel ?prefLabel } + } + } +ORDER BY ?prefLabel +""" ; + rdfs:isDefinedBy : . + +# ConceptScheme + +skos:ConceptScheme spin:constructor :ConceptSchemeConstructor . + +:ConceptSchemeConstructor a ldh:Constructor ; + rdfs:label "Concept scheme constructor" ; + dct:title "Concept scheme constructor" ; + sp:text """ + PREFIX skos: + PREFIX rdf: + PREFIX dct: + + CONSTRUCT { + $this dct:title [ a rdf:langString ] ; + dct:description [ a rdf:langString ] ; + skos:hasTopConcept [ a skos:Concept ] . + } + WHERE {}""" ; + rdfs:isDefinedBy : . + +skos:hasTopConcept ldh:view :TopConcepts . + +:TopConcepts a ldh:View ; + dct:title "Top concepts" ; + spin:query :SelectTopConcepts ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +# the entry level of a taxonomy, and the root of the concept tree. Both directions count as a top +# Every hierarchy query below reads BOTH directions, for the same reason: SKOS declares +# skos:broader/skos:narrower and skos:hasTopConcept/skos:topConceptOf as inverse pairs and leaves the +# choice to the modeller, so a parent may point down or a child may point up and both are correct. +# Reading one direction made a view disagree with the concept tree, which unions them: measured on the +# fixture, Hot drinks' Narrower block showed 1 child where the tree showed 3, because Coffee and Tea +# assert skos:broader upwards, and Juice's Broader block showed nothing while the tree nested it under +# Hot drinks. The union stays INSIDE the link's GRAPH rather than becoming a sibling block, because +# SPARQLBuilder's round-trip merges sibling GRAPH blocks into one and would scope the child's label to +# the link's document. +:SelectTopConcepts a sp:Select ; + rdfs:label "Select top concepts" ; + dct:title "Select top concepts" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?concept +WHERE + { GRAPH ?graph + { { $about skos:hasTopConcept ?concept } + UNION + { ?concept skos:topConceptOf $about } + } + } +""" ; + rdfs:isDefinedBy : . + +skos:inScheme ldh:view :ConceptScheme . + +:ConceptScheme a ldh:View ; + dct:title "Scheme" ; + spin:query :SelectConceptScheme ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectConceptScheme a sp:Select ; + rdfs:label "Select the scheme a concept belongs to" ; + dct:title "Select the scheme a concept belongs to" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?scheme +WHERE + { GRAPH ?graph + { $about skos:inScheme ?scheme } + } +""" ; + rdfs:isDefinedBy : . + +skos:inScheme ldh:inverseView :ConceptsInScheme , :OrphanConcepts . + +:ConceptsInScheme a ldh:View ; + dct:title "Concepts in scheme" ; + spin:query :SelectConceptsInScheme ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectConceptsInScheme a sp:Select ; + rdfs:label "Select concepts in scheme" ; + dct:title "Select concepts in scheme" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?concept +WHERE + { GRAPH ?graph + { ?concept skos:inScheme $about ; + skos:prefLabel ?prefLabel + } + } +ORDER BY ?prefLabel +""" ; + rdfs:isDefinedBy : . + +# Concepts a scheme lists but nothing places in its hierarchy: neither a top concept nor narrower +# than anything. A taxonomy with orphans is not wrong, but they are invisible to a tree that +# descends from the top, so the scheme surfaces them rather than letting them go unreachable +:OrphanConcepts a ldh:View ; + dct:title "Unplaced concepts" ; + spin:query :SelectOrphanConcepts ; + ac:mode ac:TableMode ; + ldh:showWhenEmpty false ; + rdfs:isDefinedBy : . + +:SelectOrphanConcepts a sp:Select ; + rdfs:label "Select concepts with no place in the hierarchy" ; + dct:title "Select concepts with no place in the hierarchy" ; + sp:text """ +PREFIX skos: + +SELECT DISTINCT ?concept +WHERE + { GRAPH ?graph + { ?concept skos:inScheme $about } + FILTER NOT EXISTS { GRAPH ?topGraph { $about skos:hasTopConcept ?concept } } + FILTER NOT EXISTS { GRAPH ?topGraph { ?concept skos:topConceptOf $about } } + FILTER NOT EXISTS { GRAPH ?broaderGraph { ?concept skos:broader ?parent } } + FILTER NOT EXISTS { GRAPH ?narrowerGraph { ?parent skos:narrower ?concept } } + } +""" ; + rdfs:isDefinedBy : . + +# Constraints + +:MissingPrefLabel a ldh:MissingPropertyValue ; + rdfs:label "Missing skos:prefLabel" ; + sp:arg1 skos:prefLabel ; + rdfs:isDefinedBy : . + +# a concept outside every scheme cannot be reached from a scheme's tree, so the package treats +# skos:inScheme as mandatory even though SKOS itself does not +:MissingInScheme a ldh:MissingPropertyValue ; + rdfs:label "Missing skos:inScheme" ; + sp:arg1 skos:inScheme ; + rdfs:isDefinedBy : . diff --git a/tests/packages/editor/taxonomy/skos.xsl b/tests/packages/editor/taxonomy/skos.xsl new file mode 100644 index 000000000..f5ee6085d --- /dev/null +++ b/tests/packages/editor/taxonomy/skos.xsl @@ -0,0 +1,448 @@ + + + + + + + + + + +]> + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
+ +
+

+ +

+ + +
    + + + + + + + +
+
+
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    +
  • + + + + +
    • +
+ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/tests/packages/publish.sh b/tests/packages/publish.sh new file mode 100755 index 000000000..64a9fb1f5 --- /dev/null +++ b/tests/packages/publish.sh @@ -0,0 +1,28 @@ +#!/usr/bin/env bash +# Publishes this fixture package registry onto a LinkedDataHub dataspace, the way +# LinkedDataHub-Apps/packages/install.sh publishes https://packages.linkeddatahub.com/: pushes the document +# tree, whose folders are the package URIs, and declares each package's stylesheet, which the push uploads +# as text/xsl, by the URI of that upload. +# +# The package tests import from here rather than from the public registry, so that they depend on neither +# its reachability nor its current content. editor/taxonomy is a copy of the taxonomy editor package from +# LinkedDataHub-Apps; what the tests assert about it (its constructors, views and stylesheet rules) is +# pinned by this copy and changes only when it is updated here. +# +# Usage: publish.sh BASE_URI KEYSTORE PASSWORD +set -euo pipefail + +base="$1" +cert="$2" +password="$3" +dir="$(cd "$(dirname "$0")" && pwd)" + +ldh push -c "$cert" -p "$password" -b "$base" --dir "$dir" "$base" > /dev/null + +# a package is the folder its stylesheet is in, e.g. editor/taxonomy/skos.xsl -> ${base}editor/taxonomy/#this +(cd "$dir" && find . -name '*.xsl' | sed 's|^\./||' | sort) | while read -r stylesheet; do + package_doc="${base}$(dirname "$stylesheet")/" + upload="${base}uploads/$(shasum -a 1 "$dir/$stylesheet" | cut -d' ' -f1)" + echo "INSERT { <${package_doc}#this> <${upload}> } WHERE { }" | + ldh patch -c "$cert" -p "$password" "$package_doc" +done diff --git a/tests/packages/root.ttl b/tests/packages/root.ttl new file mode 100644 index 000000000..786e77c0a --- /dev/null +++ b/tests/packages/root.ttl @@ -0,0 +1,27 @@ +@prefix def: . +@prefix ldh: . +@prefix rdf: . +@prefix rdfs: . +@prefix dct: . + +# The package catalog. The dataspace settings modal and `ldh packages list` read this document and +# offer each rdfs:member, labelled with the dct:title and dct:description given here, so a new +# package is listed by adding it below with the same title and description as its descriptor. + +<> a def:Root ; + dct:title "LinkedDataHub packages" ; + dct:description "Catalog of packages available for LinkedDataHub applications" ; + rdfs:member ; + rdf:_1 <#content> ; + rdf:_2 <#select-children> . + +<#content> a ldh:XHTML ; + rdf:value """
+

Packages add a vocabulary to a dataspace together with the views, forms and templates that render it. A dataspace imports one from its settings.

+
"""^^rdf:XMLLiteral . + +<#select-children> a ldh:Object ; + rdf:value ldh:ChildrenView . + + dct:title "Taxonomy Editor" ; + dct:description "Turns a dataspace into a taxonomy editor: SKOS concepts, schemes and collections gain a concept tree beside the content, hierarchy views that read both assertion directions, and constructors and constraints that keep a concept labelled and in a scheme." . diff --git a/tests/ui/README.md b/tests/ui/README.md index c1662a6ba..4b032aaba 100644 --- a/tests/ui/README.md +++ b/tests/ui/README.md @@ -65,7 +65,8 @@ not remembered. | `UI_TESTS_SKIP_SEED=1` | Reuse whatever is already there — for iterating on one spec | | `UI_TESTS_KEEP_FIXTURES=1` | Leave the container behind to inspect it in a browser | | `UI_TESTS_ITEMS=n` | Fewer children (default 25 — enough for a second pager page) | -| `UI_TESTS_TAXONOMY_PACKAGE=uri` | A different taxonomy package to import (default: the bundled taxonomy editor) | +| `UI_TESTS_TAXONOMY_PACKAGE=uri` | A different taxonomy package to import, as it is (default: the taxonomy editor from the fixture registry `tests/packages`, published onto the stack's `packages.` dataspace) | +| `PACKAGES_BASE_URL=url` | The dataspace the fixture registry is published onto (default: `packages.` + the end-user host) | | `REMOTE_END_USER_BASE_URL=url` | The second end-user dataspace the cross-origin fixture is seeded into. Unset, the preflight probes the origins declared in `config/dataspaces.trig` and `tests/http/config/dataspaces.trig` and takes the one the stack answers for | ### The remote document diff --git a/tests/ui/lib/taxonomy.mjs b/tests/ui/lib/taxonomy.mjs index 54d0ca4b2..19b73adfc 100644 --- a/tests/ui/lib/taxonomy.mjs +++ b/tests/ui/lib/taxonomy.mjs @@ -10,10 +10,12 @@ // drinks names it as narrower, and the two top concepts arrive one from each direction. // Depth is deliberate too - espresso sits three hops below the scheme, which is what makes // the reveal a walk rather than a single lookup. +import { execFileSync } from 'node:child_process'; +import { join } from 'node:path'; import { get } from './http.mjs'; import { ldh } from './fixtures.mjs'; import { READ_MODE, inMode } from './mode.mjs'; -import { adminBase, endUserBase } from './stack.mjs'; +import { adminBase, endUserBase, ownerKeystore, ownerPassword, repoRoot } from './stack.mjs'; const slug = 'ui-taxonomy'; const SKOS = 'http://www.w3.org/2004/02/skos/core#'; @@ -21,8 +23,14 @@ const FOAF = 'http://xmlns.com/foaf/0.1/'; // The package under test. Without it the tree does not exist at all: the column, the // hierarchy queries and the reveal are all the package stylesheet's, not the platform's. -export const taxonomyPackage = process.env.UI_TESTS_TAXONOMY_PACKAGE - ?? 'https://packages.linkeddatahub.com/editor/taxonomy/#this'; +// It is imported from the fixture registry, tests/packages, which seedTaxonomy publishes onto the +// stack's packages dataspace, so the suite depends on neither the reachability nor the current +// content of https://packages.linkeddatahub.com/. UI_TESTS_TAXONOMY_PACKAGE names another package, +// which is imported as it is. +const endUserURL = new URL(endUserBase); +export const packagesBase = process.env.PACKAGES_BASE_URL ?? `${endUserURL.protocol}//packages.${endUserURL.host}/`; +const fixturePackage = !process.env.UI_TESTS_TAXONOMY_PACKAGE; +export const taxonomyPackage = process.env.UI_TESTS_TAXONOMY_PACKAGE ?? `${packagesBase}editor/taxonomy/#this`; export const taxonomy = { container: `${endUserBase}${slug}/` }; @@ -107,7 +115,7 @@ function parentTriple(node) { } async function packageInstalled() { - const { stdout } = await ldh(['packages', 'list']); + const { stdout } = await ldh(['packages', 'list', ...(fixturePackage ? ['--registry', packagesBase] : [])]); return stdout.split('\n').some(line => { const [state, uri] = line.split('\t'); return state === 'installed' && uri === taxonomyPackage; @@ -120,6 +128,11 @@ async function packageInstalled() { let addedPackage = false; export async function seedTaxonomy() { + if (fixturePackage) { + execFileSync(join(repoRoot, 'tests/packages/publish.sh'), [packagesBase, ownerKeystore, ownerPassword()], + { stdio: ['ignore', 'ignore', 'inherit'] }); + } + if (!await packageInstalled()) { await ldh(['packages', 'add', '--package', taxonomyPackage]); addedPackage = true; From fbafc5ae4f9cf5dea34c5010a0307af00ec829ea Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 21:34:19 +0200 Subject: [PATCH 13/16] The taxonomy editor package is called the Taxonomy Editor, not the SKOS package: the fixture's ontology label and the UI README say so, and client.xsl's comment on keeping the server-rendered body no longer names a package at all, since a platform stylesheet has no business knowing one. Co-Authored-By: Claude Fable 5.1 --- .../webapp/static/com/atomgraph/linkeddatahub/xsl/client.xsl | 2 +- tests/packages/editor/taxonomy/ns.ttl | 2 +- tests/ui/README.md | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client.xsl b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client.xsl index b2280799e..27024acf6 100644 --- a/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client.xsl +++ b/src/main/webapp/static/com/atomgraph/linkeddatahub/xsl/client.xsl @@ -375,7 +375,7 @@ WHERE - + diff --git a/tests/packages/editor/taxonomy/ns.ttl b/tests/packages/editor/taxonomy/ns.ttl index 2e98c7bc3..9c4560b94 100644 --- a/tests/packages/editor/taxonomy/ns.ttl +++ b/tests/packages/editor/taxonomy/ns.ttl @@ -18,7 +18,7 @@ : a owl:Ontology ; owl:imports ; - rdfs:label "SKOS package ontology". + rdfs:label "Taxonomy Editor ontology". # Concept diff --git a/tests/ui/README.md b/tests/ui/README.md index 4b032aaba..6e125f36d 100644 --- a/tests/ui/README.md +++ b/tests/ui/README.md @@ -163,7 +163,7 @@ these axes in its own prose, so the folder is the README's vocabulary rather tha Two things that are *not* in the path, because neither is a fact about the component: which app serves the fixture (`overlays/modal/ontology-import` runs against the admin origin, `constructor-editor` against both), and who owns the markup (`document/content-aside/concept-tree` -is the SKOS package's, rendered in the platform's slot). +is the Taxonomy Editor package's, rendered in the platform's slot). Helpers stay flat in `lib/`, named for the component whose vocabulary they carry. `specs/` is containment; `lib/` is vocabulary. A helper is reached across regions — a block spec opens the From a76d553f08bd913b63a4db1fd890587a9f65565f Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 21:50:28 +0200 Subject: [PATCH 14/16] Every write to a document is held to the constraints of what it writes. A PUT whose body did not type the document was validated while the document was untyped, then typed dh:Item by the server and written, so an ontology file pushed as a document came out a dh:Item without a title; put() now checks a document it types itself before writing it, with validateConstraints, which runs the SPIN and SHACL checks alone - validate() would announce the model's authorizations a second time. A POST, plain or multipart, validated only its body, where a constraint spanning the document had nothing to apply to; both now check the document as it will be written, the upload before any file is stored, and the upload still needs no If-Match since it is appended rather than written back. The multipart PUT, the document form's, wrote the form's triples as the graph and skipped put(Model) altogether - no type or container, no created/creator/owner, no dct:modified, no If-Match; it now writes the files and hands the model to put(Model). PATCH and the new checks share describeViolations, so a 422 describes only the violating resources. The PUT tests that created untitled documents title them, PUT-empty.sh becomes PUT-empty-422.sh, and new tests cover the untitled PUT (PUT-missing-title-422-body), the POST whose document becomes invalid (POST-invalid-content-block-422, POST-multipart-invalid-content-block-422) and the multipart PUT (PUT-multipart-metadata, -412, -created, -missing-title-422). Co-Authored-By: Claude Fable 5.1 --- .../impl/DocumentHierarchyGraphStoreImpl.java | 106 +++++++++++++++--- .../POST-invalid-content-block-422.sh | 85 ++++++++++++++ ...OST-multipart-invalid-content-block-422.sh | 89 +++++++++++++++ .../{PUT-empty.sh => PUT-empty-422.sh} | 4 +- .../document-hierarchy/PUT-item-metadata.sh | 4 +- tests/http/document-hierarchy/PUT-location.sh | 1 + .../PUT-missing-title-422-body.sh | 95 ++++++++++++++++ .../document-hierarchy/PUT-multipart-412.sh | 76 +++++++++++++ .../PUT-multipart-created.sh | 65 +++++++++++ .../PUT-multipart-metadata.sh | 95 ++++++++++++++++ .../PUT-multipart-missing-title-422.sh | 51 +++++++++ .../document-hierarchy/PUT-no-slash-308.sh | 2 + .../PUT-relative-uri-turtle.sh | 1 + tests/http/proxy/PUT-proxied-location.sh | 1 + 14 files changed, 655 insertions(+), 20 deletions(-) create mode 100755 tests/http/document-hierarchy/POST-invalid-content-block-422.sh create mode 100755 tests/http/document-hierarchy/POST-multipart-invalid-content-block-422.sh rename tests/http/document-hierarchy/{PUT-empty.sh => PUT-empty-422.sh} (84%) create mode 100755 tests/http/document-hierarchy/PUT-missing-title-422-body.sh create mode 100755 tests/http/document-hierarchy/PUT-multipart-412.sh create mode 100755 tests/http/document-hierarchy/PUT-multipart-created.sh create mode 100755 tests/http/document-hierarchy/PUT-multipart-metadata.sh create mode 100755 tests/http/document-hierarchy/PUT-multipart-missing-title-422.sh diff --git a/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/DocumentHierarchyGraphStoreImpl.java b/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/DocumentHierarchyGraphStoreImpl.java index 687be2d56..b2b940233 100644 --- a/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/DocumentHierarchyGraphStoreImpl.java +++ b/src/main/java/com/atomgraph/linkeddatahub/server/model/impl/DocumentHierarchyGraphStoreImpl.java @@ -387,9 +387,15 @@ public Response post(Model model) if (log.isDebugEnabled()) log.debug("POST Model to named graph with URI: {}", getURI()); // First remove old dct:modified values from the triplestore, then add new data existingModel.createResource(getURI().toString()).removeAll(DCTerms.modified); - getSystem().getServiceContext(getService()).getGraphStoreClient().putModel(getURI().toString(), existingModel.add(model)); // replace entire graph to avoid accumulating dct:modified Model updatedModel = existingModel.add(model); + // the payload was validated on its own when it was read, where a constraint that spans the document - + // a block the payload appends to a document it does not type - had nothing to apply to; what is written + // is the whole document, so that is what is held to the constraints + validateConstraints(updatedModel); + + getSystem().getServiceContext(getService()).getGraphStoreClient().putModel(getURI().toString(), updatedModel); // replace entire graph to avoid accumulating dct:modified + submitImports(model); return Response.noContent(). @@ -456,15 +462,23 @@ public Response put(Model model) removeAll(SIOC.HAS_PARENT). removeAll(SIOC.HAS_CONTAINER); + boolean typed = false; // whether the document's class is assigned here rather than by the payload if (!getDataspace().getBaseURI().equals(getURI())) // don't update Root document's metadata { if (resource.hasProperty(RDF.type, DH.Container)) resource.addProperty(SIOC.HAS_PARENT, parent); else + { + typed = !resource.hasProperty(RDF.type, DH.Item); resource.addProperty(SIOC.HAS_CONTAINER, parent). addProperty(RDF.type, DH.Item); // TO-DO: replace with foaf:Document? + } } + // the payload was validated when it was read, while the document was still untyped and so held to no + // class's constraints; a document typed here is held to dh:Item's now, before it is written + if (typed) validateConstraints(model); + if (existingModel == null) // creating new graph and attaching it to the document hierarchy { resource.removeAll(DCTerms.created). // remove any client-supplied dct:created values @@ -576,20 +590,12 @@ public Response patch(UpdateRequest updateRequest) { // the whole post-PATCH graph gets validated, but the 422 body must describe only the // violating resources - the full graph would leak every sibling resource into the - // error response, unlike POST/PUT whose echoed model is the request payload - Set roots = new HashSet<>(); - for (ConstraintViolation cv : ex.getConstraintViolations()) - if (cv.getRoot() != null) roots.add(cv.getRoot()); - - throw new SPINConstraintViolationException(ex.getConstraintViolations(), describeResources(roots, dataset.getDefaultModel())); + // error response + throw describeViolations(ex, dataset.getDefaultModel()); } catch (SHACLConstraintViolationException ex) { - Set roots = new HashSet<>(); - for (ReportEntry entry : ex.getValidationReport().getEntries()) - if (!entry.focusNode().isLiteral()) roots.add(dataset.getDefaultModel().asRDFNode(entry.focusNode()).asResource()); - - throw new SHACLConstraintViolationException(ex.getValidationReport(), describeResources(roots, dataset.getDefaultModel())); + throw describeViolations(ex, dataset.getDefaultModel()); } put(dataset.getDefaultModel(), Boolean.FALSE, getURI()); @@ -640,11 +646,15 @@ public Response postMultipart(FormDataMultiPart multiPart) validate(model); if (log.isTraceEnabled()) log.trace("POST Graph Store request with RDF payload: {} payload size(): {}", model, model.size()); - final boolean existingGraph = getSystem().getServiceContext(getService()).getGraphStoreClient().containsModel(getURI().toString()); - if (!existingGraph) throw new NotFoundException("Named graph with URI <" + getURI() + "> not found"); + final Model existingModel = getSystem().getServiceContext(getService()).getGraphStoreClient().getModel(getURI().toString()); + if (existingModel == null) throw new NotFoundException("Named graph with URI <" + getURI() + "> not found"); new Skolemizer(getURI().toString()).apply(model); // skolemize before writing files (they require absolute URIs) + // appended to the store rather than written back whole, so no If-Match is asked of an upload; the + // document it is appended to is still held to the constraints as a whole, before a file is written + validateConstraints(ModelFactory.createDefaultModel().add(existingModel).add(model)); + int fileCount = writeFiles(model, getFileNameBodyPartMap(multiPart)); if (log.isDebugEnabled()) log.debug("# of files uploaded: {} ", fileCount); @@ -665,7 +675,7 @@ public Response postMultipart(FormDataMultiPart multiPart) /** * Handles multipart PUT - * Files are written to storage before the RDF data is passed to the default PUT handler method. + * Files are written to storage before the RDF data is passed to {@link #put(Model)}. * * @param multiPart multipart form data * @return HTTP response @@ -689,8 +699,11 @@ public Response putMultipart(FormDataMultiPart multiPart) int fileCount = writeFiles(model, getFileNameBodyPartMap(multiPart)); if (log.isDebugEnabled()) log.debug("# of files uploaded: {} ", fileCount); - - return put(model, false, getURI()); + + // the same PUT as an RDF body gets - the document's type and container, its created/creator/owner + // metadata, the If-Match precondition and the constraints of a class assigned here - rather than the + // raw graph write, which skipped all of it for the document form + return put(model); } catch (URISyntaxException ex) { @@ -1079,6 +1092,65 @@ public Model validate(Model model) throw new InternalServerErrorException("Could not obtain ValidatingModelProvider instance"); } + /** + * Checks a model against the ontology's SPIN constraints and SHACL shapes, and nothing else: unlike + * {@link #validate(Model)}, which runs the processing a request body gets when it is read, this does not + * announce the model's acl:Authorizations again. A violation is reported with only the + * violating resources described. + * + * @param model RDF model + */ + public void validateConstraints(Model model) + { + MessageBodyReader reader = getProviders().getMessageBodyReader(Model.class, null, null, com.atomgraph.core.MediaType.APPLICATION_NTRIPLES_TYPE); + if (!(reader instanceof ValidatingModelProvider validatingModelProvider)) throw new InternalServerErrorException("Could not obtain ValidatingModelProvider instance"); + + try + { + validatingModelProvider.validate(model); + } + catch (SPINConstraintViolationException ex) + { + throw describeViolations(ex, model); + } + catch (SHACLConstraintViolationException ex) + { + throw describeViolations(ex, model); + } + } + + /** + * Returns a SPIN violation whose model describes only the violating resources. + * + * @param ex violation over the whole model + * @param model validated model + * @return violation with the violating resources' descriptions + */ + public SPINConstraintViolationException describeViolations(SPINConstraintViolationException ex, Model model) + { + Set roots = new HashSet<>(); + for (ConstraintViolation cv : ex.getConstraintViolations()) + if (cv.getRoot() != null) roots.add(cv.getRoot()); + + return new SPINConstraintViolationException(ex.getConstraintViolations(), describeResources(roots, model)); + } + + /** + * Returns a SHACL violation whose model describes only the focus nodes of the report. + * + * @param ex violation over the whole model + * @param model validated model + * @return violation with the focus nodes' descriptions + */ + public SHACLConstraintViolationException describeViolations(SHACLConstraintViolationException ex, Model model) + { + Set roots = new HashSet<>(); + for (ReportEntry entry : ex.getValidationReport().getEntries()) + if (!entry.focusNode().isLiteral()) roots.add(model.asRDFNode(entry.focusNode()).asResource()); + + return new SHACLConstraintViolationException(ex.getValidationReport(), describeResources(roots, model)); + } + /** * Copies the concise bounded descriptions of the given resources from a model. * diff --git a/tests/http/document-hierarchy/POST-invalid-content-block-422.sh b/tests/http/document-hierarchy/POST-invalid-content-block-422.sh new file mode 100755 index 000000000..fddb4446d --- /dev/null +++ b/tests/http/document-hierarchy/POST-invalid-content-block-422.sh @@ -0,0 +1,85 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# create an item with random slug + +slug=$(uuidgen | tr '[:upper:]' '[:lower:]') + +item=$(ldh create item \ + -c "$AGENT_CERT_KEYSTORE" \ + -p "$AGENT_CERT_PWD" \ + -b "$END_USER_BASE_URL" \ + --title "Test item" \ + --slug "$slug" \ + --container "$END_USER_BASE_URL") + +# POST a block typed as something other than ldh:Object/ldh:XHTML into the item. The body does not type the +# item, so validated on its own ldh:InvalidContentBlockType has nothing to apply to; the document it is +# appended to is a dh:Item, and it is the whole document that gets written, so the POST is refused with 422 +# and nothing is appended. + +response=$(curl -k -w "%{http_code}\n" -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X POST \ + -H "If-Match: $(etag "$item" "$AGENT_CERT_FILE" "$AGENT_CERT_PWD" "application/n-triples")" \ + -H "Accept: application/n-triples" \ + -H "Content-Type: application/n-triples" \ + --data-binary @- \ + "$item" < <${item}#bad-block> . +<${item}#bad-block> . +<${item}#bad-block> "Not a valid content block" . +<${item}#bad-block> "CONSTRUCT WHERE {}" . +EOF +) + +status=$(echo "$response" | tail -n 1) +body=$(echo "$response" | sed '$d') + +echo "DEBUG: Expected status: $STATUS_UNPROCESSABLE_ENTITY" +echo "DEBUG: Got status: $status" +if [ "$status" != "$STATUS_UNPROCESSABLE_ENTITY" ]; then + echo "DEBUG: Status mismatch!" + exit 1 +fi + +ntriples=$(echo "$body" | rapper -q --input ntriples --output ntriples /dev/stdin -) +echo "DEBUG: Response body as N-Triples:" +echo "$ntriples" + +# the violation rooted in the document + +expected=" <${item}>" +echo "DEBUG: Expected present: $expected" +if ! echo "$ntriples" | grep -qF "$expected"; then + echo "DEBUG: Violation root missing!" + exit 1 +fi + +# nothing was appended + +item_ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item") + +if grep -qF "<${item}#bad-block>" <<< "$item_ntriples"; then + echo "DEBUG: The refused block was appended!" + exit 1 +fi diff --git a/tests/http/document-hierarchy/POST-multipart-invalid-content-block-422.sh b/tests/http/document-hierarchy/POST-multipart-invalid-content-block-422.sh new file mode 100755 index 000000000..20eb86327 --- /dev/null +++ b/tests/http/document-hierarchy/POST-multipart-invalid-content-block-422.sh @@ -0,0 +1,89 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# create an item with random slug + +slug=$(uuidgen | tr '[:upper:]' '[:lower:]') + +item=$(ldh create item \ + -c "$AGENT_CERT_KEYSTORE" \ + -p "$AGENT_CERT_PWD" \ + -b "$END_USER_BASE_URL" \ + --title "Test item" \ + --slug "$slug" \ + --container "$END_USER_BASE_URL") + +# An upload is a multipart POST appended to the document, without If-Match. It is held to the constraints of +# the document it lands in as a whole, before any file is written: one that also appends a block typed as +# something other than ldh:Object/ldh:XHTML to the dh:Item is refused with 422, and its file is not stored. + +test_file=$(mktemp) +echo "file refused with its document $(uuidgen)" > "$test_file" # content no other test uploads +sha1sum=$(shasum -a 1 "$test_file" | awk '{print $1}') + +status=$(curl -k -w "%{http_code}\n" -o /dev/null -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X POST \ + -H "Accept: application/n-triples" \ + -F "rdf=" \ + -F "sb=file" \ + -F "pu=http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#fileName" \ + -F "ol=@${test_file};type=text/plain" \ + -F "pu=http://purl.org/dc/terms/title" \ + -F "ol=Refused file" \ + -F "pu=http://www.w3.org/1999/02/22-rdf-syntax-ns#type" \ + -F "ou=http://www.semanticdesktop.org/ontologies/2007/03/22/nfo#FileDataObject" \ + -F "su=${item}" \ + -F "pu=http://www.w3.org/1999/02/22-rdf-syntax-ns#_2" \ + -F "ou=${item}#bad-block" \ + -F "su=${item}#bad-block" \ + -F "pu=http://www.w3.org/1999/02/22-rdf-syntax-ns#type" \ + -F "ou=http://spinrdf.org/sp#Construct" \ + "$item") + +rm -f "$test_file" + +echo "DEBUG: Expected status: $STATUS_UNPROCESSABLE_ENTITY Got: $status" +if [ "$status" != "$STATUS_UNPROCESSABLE_ENTITY" ]; then + exit 1 +fi + +# neither the block nor the file's description was appended + +item_ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item") + +if grep -qF "#bad-block>" <<< "$item_ntriples" || grep -qF "\"Refused file\"" <<< "$item_ntriples"; then + echo "DEBUG: The refused upload was appended!" + exit 1 +fi + +# and the file was not stored + +status=$(curl -k -s -o /dev/null -w "%{http_code}" \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + "${END_USER_BASE_URL}uploads/${sha1sum}") + +echo "DEBUG: Refused file status: $status" +if [ "$status" = "$STATUS_OK" ]; then + echo "DEBUG: The refused file was stored!" + exit 1 +fi diff --git a/tests/http/document-hierarchy/PUT-empty.sh b/tests/http/document-hierarchy/PUT-empty-422.sh similarity index 84% rename from tests/http/document-hierarchy/PUT-empty.sh rename to tests/http/document-hierarchy/PUT-empty-422.sh index d4bd1b2bd..0a3806e0a 100755 --- a/tests/http/document-hierarchy/PUT-empty.sh +++ b/tests/http/document-hierarchy/PUT-empty-422.sh @@ -17,7 +17,7 @@ ldh admin add agent \ --agent "$AGENT_URI" \ "${ADMIN_BASE_URL}acl/groups/writers/" -# check that graph without parent is forbidden +# check that an empty document is refused: the server types it dh:Item, which requires a dct:title ( curl -k -w "%{http_code}\n" -o /dev/null -s \ @@ -28,4 +28,4 @@ curl -k -w "%{http_code}\n" -o /dev/null -s \ "${END_USER_BASE_URL}non-existing/" < "Test item" . <${item}> "named object PUT" . EOF ) \ diff --git a/tests/http/document-hierarchy/PUT-location.sh b/tests/http/document-hierarchy/PUT-location.sh index 9e8d11ed5..6745de89e 100755 --- a/tests/http/document-hierarchy/PUT-location.sh +++ b/tests/http/document-hierarchy/PUT-location.sh @@ -29,6 +29,7 @@ response=$(curl -k -s \ -o /dev/null \ --data-binary @- \ "$new_doc_uri" < "New document" . EOF ) diff --git a/tests/http/document-hierarchy/PUT-missing-title-422-body.sh b/tests/http/document-hierarchy/PUT-missing-title-422-body.sh new file mode 100755 index 000000000..f175a99ae --- /dev/null +++ b/tests/http/document-hierarchy/PUT-missing-title-422-body.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# A PUT whose body says nothing about the document itself - an ontology file pushed as a document, say - +# was validated while the document was untyped, then typed dh:Item by the server and written without a +# title. The document the server types is now held to dh:Item's constraints: def:MissingTitle trips with +# the document as spin:violationRoot, nothing is written, and the 422 body describes the document but not +# the other resources in the body. + +item="${END_USER_BASE_URL}$(uuidgen | tr '[:upper:]' '[:lower:]')/" + +response=$(curl -k -w "%{http_code}\n" -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Accept: application/n-triples" \ + -H "Content-Type: application/n-triples" \ + --data-binary @- \ + "$item" < . +<${item}#ontology> "Untitled document's ontology" . +EOF +) + +status=$(echo "$response" | tail -n 1) +body=$(echo "$response" | sed '$d') + +echo "DEBUG: Expected status: $STATUS_UNPROCESSABLE_ENTITY" +echo "DEBUG: Got status: $status" +if [ "$status" != "$STATUS_UNPROCESSABLE_ENTITY" ]; then + echo "DEBUG: Status mismatch!" + exit 1 +fi + +ntriples=$(echo "$body" | rapper -q --input ntriples --output ntriples /dev/stdin -) +echo "DEBUG: Response body as N-Triples:" +echo "$ntriples" + +# the violation rooted in the document + +expected=" <${item}>" +echo "DEBUG: Expected present: $expected" +if ! echo "$ntriples" | grep -qF "$expected"; then + echo "DEBUG: Violation root missing!" + exit 1 +fi + +# the document as the server typed it + +expected="<${item}> " +echo "DEBUG: Expected present: $expected" +if ! echo "$ntriples" | grep -qF "$expected"; then + echo "DEBUG: Violating document description missing!" + exit 1 +fi + +# the rest of the body is scoped out + +unexpected="^<${item}#ontology> " +echo "DEBUG: Expected absent as subject: <${item}#ontology>" +if echo "$ntriples" | grep -q "$unexpected"; then + echo "DEBUG: Non-violating resource leaked into the 422 body!" + exit 1 +fi + +# nothing was written: asked of the store rather than the document URL, which answers 403 for a document +# that does not exist (a typeless URL matches no authorization) + +written=$(curl -s -G \ + -H "Accept: application/sparql-results+xml" \ + --data-urlencode "query=ASK { GRAPH <${item}> { ?s ?p ?o } }" \ + "$END_USER_ENDPOINT_URL" \ +| xmllint --xpath "string(//*[local-name() = 'boolean'])" -) + +echo "DEBUG: Expected graph written: false" +echo "DEBUG: Got graph written: $written" +if [ "$written" != "false" ]; then + echo "DEBUG: The refused document was written!" + exit 1 +fi diff --git a/tests/http/document-hierarchy/PUT-multipart-412.sh b/tests/http/document-hierarchy/PUT-multipart-412.sh new file mode 100755 index 000000000..8e5d5447b --- /dev/null +++ b/tests/http/document-hierarchy/PUT-multipart-412.sh @@ -0,0 +1,76 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# create an item with random slug + +slug=$(uuidgen | tr '[:upper:]' '[:lower:]') + +item=$(ldh create item \ + -c "$AGENT_CERT_KEYSTORE" \ + -p "$AGENT_CERT_PWD" \ + -b "$END_USER_BASE_URL" \ + --title "Test item" \ + --slug "$slug" \ + --container "$END_USER_BASE_URL") + +# A multipart PUT - the document form's - answers to If-Match like any other write: one made against a state +# of the document that has since changed is refused, and the change made in between survives. + +stale_etag=$(etag "$item" "$AGENT_CERT_FILE" "$AGENT_CERT_PWD" "application/n-triples") + +# someone else's change, made after that state was read + +curl -k -f -s -o /dev/null \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PATCH \ + -H "If-Match: $stale_etag" \ + -H "Accept: application/n-triples" \ + -H "Content-Type: application/sparql-update" \ + --data-binary "INSERT { <${item}> \"Changed in between\" } WHERE { }" \ + "$item" + +status=$(curl -k -w "%{http_code}\n" -o /dev/null -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Accept: application/n-triples" \ + -H "If-Match: $stale_etag" \ + -F "rdf=" \ + -F "su=${item}" \ + -F "pu=http://purl.org/dc/terms/title" \ + -F "ol=Overwriting title" \ + "$item") + +echo "DEBUG: Expected status: $STATUS_PRECONDITION_FAILED Got: $status" +if [ "$status" != "$STATUS_PRECONDITION_FAILED" ]; then + exit 1 +fi + +item_ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item") + +if ! grep -qF "<${item}> \"Changed in between\"" <<< "$item_ntriples"; then + echo "DEBUG: The change made in between was overwritten!" + exit 1 +fi +if grep -qF "\"Overwriting title\"" <<< "$item_ntriples"; then + echo "DEBUG: The refused write landed!" + exit 1 +fi diff --git a/tests/http/document-hierarchy/PUT-multipart-created.sh b/tests/http/document-hierarchy/PUT-multipart-created.sh new file mode 100755 index 000000000..a505cdeb1 --- /dev/null +++ b/tests/http/document-hierarchy/PUT-multipart-created.sh @@ -0,0 +1,65 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# A multipart PUT to a document that does not exist yet creates it the way a PUT with an RDF body does: 201 +# with a Location, and the type, container and creation metadata the server assigns. It used to write the +# form's triples as the graph and nothing else. + +item="${END_USER_BASE_URL}$(uuidgen | tr '[:upper:]' '[:lower:]')/" + +response=$(curl -k -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Accept: application/n-triples" \ + -D - \ + -o /dev/null \ + -F "rdf=" \ + -F "su=${item}" \ + -F "pu=http://purl.org/dc/terms/title" \ + -F "ol=Created through a form" \ + "$item") + +http_code=$(echo "$response" | grep -m1 "^HTTP" | awk '{print $2}') +location=$(echo "$response" | grep -i "^location:" | tr -d '\r' | awk '{print $2}') + +echo "DEBUG: Expected status: $STATUS_CREATED Got: $http_code" +echo "DEBUG: Expected Location: $item Got: $location" +[ "$http_code" = "$STATUS_CREATED" ] +[ "$location" = "$item" ] + +item_ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item") +echo "DEBUG: Created document:" +echo "$item_ntriples" + +for triple in \ + "<${item}> \"Created through a form\"" \ + "<${item}> " \ + "<${item}> <${END_USER_BASE_URL}>" \ + "<${item}> \"" \ + "<${item}> <" \ + "<${item}> <" +do + if ! grep -qF "$triple" <<< "$item_ntriples"; then + echo "DEBUG: Missing: $triple" + exit 1 + fi +done diff --git a/tests/http/document-hierarchy/PUT-multipart-metadata.sh b/tests/http/document-hierarchy/PUT-multipart-metadata.sh new file mode 100755 index 000000000..290bec7f2 --- /dev/null +++ b/tests/http/document-hierarchy/PUT-multipart-metadata.sh @@ -0,0 +1,95 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# create an item with random slug + +slug=$(uuidgen | tr '[:upper:]' '[:lower:]') + +item=$(ldh create item \ + -c "$AGENT_CERT_KEYSTORE" \ + -p "$AGENT_CERT_PWD" \ + -b "$END_USER_BASE_URL" \ + --title "Test item" \ + --slug "$slug" \ + --container "$END_USER_BASE_URL") + +created=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item" \ +| grep "^<${item}> ") + +# A multipart PUT - the document form's, whose files have to be written before the RDF - is the same PUT as +# one with an RDF body: it wrote the form's triples as the graph and nothing else, so a document saved +# through the form lost its created/creator/owner metadata and container, and its If-Match was never checked. +# The body here says only what the form would: the document's new title. + +function put_multipart() +{ + curl -k -w "%{http_code}\n" -o /dev/null -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Accept: application/n-triples" \ + "$@" \ + -F "rdf=" \ + -F "su=${item}" \ + -F "pu=http://purl.org/dc/terms/title" \ + -F "ol=Renamed item" \ + "$item" +} + +# without If-Match: the document exists, so the write has to say which state it was made against + +status=$(put_multipart) +echo "DEBUG: [no If-Match] Expected status: $STATUS_PRECONDITION_REQUIRED Got: $status" +if [ "$status" != "$STATUS_PRECONDITION_REQUIRED" ]; then + exit 1 +fi + +# with If-Match: the write lands + +status=$(put_multipart -H "If-Match: $(etag "$item" "$AGENT_CERT_FILE" "$AGENT_CERT_PWD" "application/n-triples")") +echo "DEBUG: [If-Match] Expected status: $STATUS_OK Got: $status" +if [ "$status" != "$STATUS_OK" ]; then + exit 1 +fi + +item_ntriples=$(curl -k -f -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -H "Accept: application/n-triples" \ + "$item") +echo "DEBUG: Document after the multipart PUT:" +echo "$item_ntriples" + +# the new title, and the metadata the server keeps or assigns + +for triple in \ + "<${item}> \"Renamed item\"" \ + "$created" \ + "<${item}> <" \ + "<${item}> <" \ + "<${item}> \"" \ + "<${item}> <${END_USER_BASE_URL}>" \ + "<${item}> " +do + if ! grep -qF "$triple" <<< "$item_ntriples"; then + echo "DEBUG: Missing: $triple" + exit 1 + fi +done diff --git a/tests/http/document-hierarchy/PUT-multipart-missing-title-422.sh b/tests/http/document-hierarchy/PUT-multipart-missing-title-422.sh new file mode 100755 index 000000000..9ca6ab680 --- /dev/null +++ b/tests/http/document-hierarchy/PUT-multipart-missing-title-422.sh @@ -0,0 +1,51 @@ +#!/usr/bin/env bash +set -euo pipefail + +initialize_dataset "$END_USER_BASE_URL" "$TMP_END_USER_DATASET" "$END_USER_ENDPOINT_URL" +initialize_dataset "$ADMIN_BASE_URL" "$TMP_ADMIN_DATASET" "$ADMIN_ENDPOINT_URL" +purge_cache "$END_USER_VARNISH_SERVICE" +purge_cache "$ADMIN_VARNISH_SERVICE" +purge_cache "$FRONTEND_VARNISH_SERVICE" +reset_packages +clear_ontology + +# add agent to the writers group + +ldh admin add agent \ + -c "$OWNER_CERT_KEYSTORE" \ + -p "$OWNER_CERT_PWD" \ + --agent "$AGENT_URI" \ + "${ADMIN_BASE_URL}acl/groups/writers/" + +# A multipart PUT that says nothing about the document it creates gets the document typed dh:Item by the +# server, like a PUT with an RDF body, and is held to dh:Item's constraints the same way: def:MissingTitle +# refuses it with 422, and nothing is written. + +item="${END_USER_BASE_URL}$(uuidgen | tr '[:upper:]' '[:lower:]')/" + +status=$(curl -k -w "%{http_code}\n" -o /dev/null -s \ + -E "$AGENT_CERT_FILE":"$AGENT_CERT_PWD" \ + -X PUT \ + -H "Accept: application/n-triples" \ + -F "rdf=" \ + -F "su=${item}#thing" \ + -F "pu=http://www.w3.org/2000/01/rdf-schema#label" \ + -F "ol=Not the document" \ + "$item") + +echo "DEBUG: Expected status: $STATUS_UNPROCESSABLE_ENTITY Got: $status" +if [ "$status" != "$STATUS_UNPROCESSABLE_ENTITY" ]; then + exit 1 +fi + +written=$(curl -s -G \ + -H "Accept: application/sparql-results+xml" \ + --data-urlencode "query=ASK { GRAPH <${item}> { ?s ?p ?o } }" \ + "$END_USER_ENDPOINT_URL" \ +| xmllint --xpath "string(//*[local-name() = 'boolean'])" -) + +echo "DEBUG: Expected graph written: false Got: $written" +if [ "$written" != "false" ]; then + echo "DEBUG: The refused document was written!" + exit 1 +fi diff --git a/tests/http/document-hierarchy/PUT-no-slash-308.sh b/tests/http/document-hierarchy/PUT-no-slash-308.sh index 77535cc6a..026493e3b 100755 --- a/tests/http/document-hierarchy/PUT-no-slash-308.sh +++ b/tests/http/document-hierarchy/PUT-no-slash-308.sh @@ -41,6 +41,7 @@ curl -k -w "%{http_code}\n" -o /dev/null -s \ -H "Content-Type: application/n-triples" \ --data-binary @- \ "$invalid_item" < "No slash" . <${invalid_item}> "named object PUT" . EOF ) \ @@ -56,6 +57,7 @@ curl -k -L -w "%{http_code}\n" -o /dev/null -s \ -H "Content-Type: application/n-triples" \ --data-binary @- \ "$invalid_item" < "No slash" . <${invalid_item}> "named object PUT" . EOF ) \ diff --git a/tests/http/document-hierarchy/PUT-relative-uri-turtle.sh b/tests/http/document-hierarchy/PUT-relative-uri-turtle.sh index 8e39cbbec..00ff7ef76 100755 --- a/tests/http/document-hierarchy/PUT-relative-uri-turtle.sh +++ b/tests/http/document-hierarchy/PUT-relative-uri-turtle.sh @@ -28,6 +28,7 @@ status=$(curl -k -w "%{http_code}" -o /dev/null -s \ -H "Content-Type: text/turtle" \ --data-binary @- \ "$item" < "New item" . "named object PUT" . "another named object PUT" . EOF diff --git a/tests/http/proxy/PUT-proxied-location.sh b/tests/http/proxy/PUT-proxied-location.sh index 7c4a4ce3f..4a689cff2 100755 --- a/tests/http/proxy/PUT-proxied-location.sh +++ b/tests/http/proxy/PUT-proxied-location.sh @@ -33,6 +33,7 @@ response=$(curl -k -s \ --data-binary @- \ --url-query "uri=${new_doc_uri}" \ "$END_USER_BASE_URL" < "New document" . EOF ) From a52b32970e5088c806f6e662475b92e9ff695407 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Tue, 29 Sep 2026 23:51:40 +0200 Subject: [PATCH 15/16] Extract the vocabularies and document shapes into linkeddatahub-rdf (#398) * Extract the vocabularies and document shapes into linkeddatahub-rdf The RDF a client has to send to create a container, add a chart or grant an authorization was described in static buildModel methods hanging off picocli @Command classes, reachable only by depending on the CLI - so Web-Algebra's ldh-* operations, which build the same documents through REST-VKG's execution context rather than through Jersey, had no way to share them and would have had to restate every shape. A shape restated is a shape that drifts. rdf/ now builds com.atomgraph:linkeddatahub-rdf with Jena as its only dependency: what is shared is the shape of the request body, not how it is sent, so neither consumer inherits the other's transport. The eighteen builders move off the command classes into classes named for what they build - Documents (container, item), Blocks, Views, Queries, Services, Imports, Acl and Ontologies - with BaseCommand.createSubject becoming Subjects.of, since fifteen of them needed it. The nine vocabulary classes and Slugs, SequenceNumbers, Digests and Updates move across unchanged. Commands are now argument parsing and a call. URIRewriter splits rather than moving whole: childURI, encodeSlug and adminBase are conventions the platform itself applies and belong in the library as URIs, while rewrite is the --proxy option sending a request somewhere other than where its URI says, which stays a CLI concern with origin now private. The platform keeps its own com.atomgraph.linkeddatahub.vocabulary, still duplicated with the library's. Collapsing them would make the platform depend on rdf/, and the Dockerfile builds the webapp from COPY src and COPY pom.xml alone, so the dependency has to point away from the platform rather than at it. cli/pom.xml resolves the library by ${project.version}; make cli installs it first, and make cli-version and release.sh's sync_cli_version now version both poms together, since the two must move as one or the CLI stops resolving. Co-Authored-By: Claude Opus 5 (1M context) * Publish linkeddatahub-rdf to Maven Central alongside the platform The library had no distributionManagement and no publishing plugin, so a client outside this machine could not resolve it at all - REST-VKG's ldh-* operations would have had only a local install to depend on. rdf/pom.xml now declares the central-portal-snapshots repository and the central-publishing plugin, the licenses, developers and scm blocks Central rejects a release without, and a release profile attaching sources, javadoc and GPG signatures, mirroring what release activates for the platform. release.sh deploys it immediately after release:perform, which carries reactor modules only. It builds from the release tag via git archive rather than from the working tree, because sync_cli_version has moved rdf/pom.xml on to the next development version by that point. Inside the irreversible zone on purpose: a platform release whose pinned library is absent is a release whose clients cannot build. Building the release profile also caught a javadoc reference to SP#Describe, which the trimmed vocabulary does not carry - it holds Construct and Select. Co-Authored-By: Claude Opus 5 (1M context) * Declare the snapshot repository the CLI resolves its library from cli/ is its own build, so the platform pom's do not reach it, and it had never needed any of its own: everything it depended on is released to Maven Central. linkeddatahub-rdf between releases is not - Central serves no snapshots - so resolution succeeded only where the library happened to be in the local repository, which is to say on the machine that had just installed it. CI resolved nothing and failed on the dependency. Verified against a local repository that never had the install, which is the state CI starts from: the CLI builds and its 86 tests pass. Co-Authored-By: Claude Opus 5 (1M context) --------- Co-authored-by: Claude Opus 5 (1M context) --- .gitignore | 1 + CLAUDE.md | 40 +++- cli/README.md | 6 + cli/pom.xml | 25 +++ .../linkeddatahub/cli/BaseCommand.java | 15 -- .../cli/command/AddConstruct.java | 37 +--- .../linkeddatahub/cli/command/AddFile.java | 4 +- .../cli/command/AddGenericService.java | 41 +--- .../cli/command/AddPackageImport.java | 2 +- .../cli/command/AddResultSetChart.java | 39 +--- .../linkeddatahub/cli/command/AddSelect.java | 5 +- .../linkeddatahub/cli/command/AddView.java | 36 +--- .../cli/command/CreateContainer.java | 57 +---- .../linkeddatahub/cli/command/CreateItem.java | 42 +--- .../cli/command/ListPackages.java | 2 +- .../cli/command/RemovePackageImport.java | 2 +- .../cli/command/admin/AddOntologyImport.java | 2 +- .../command/admin/acl/AddAgentToGroup.java | 2 +- .../admin/acl/CreateAuthorization.java | 65 +----- .../cli/command/admin/acl/CreateGroup.java | 44 +--- .../cli/command/admin/acl/MakePublic.java | 8 +- .../command/admin/ontologies/AddClass.java | 37 +--- .../admin/ontologies/AddConstructor.java | 38 +--- .../ontologies/AddPropertyConstraint.java | 33 +-- .../admin/ontologies/AddRestriction.java | 35 +-- .../command/admin/ontologies/AddSelect.java | 5 +- .../admin/ontologies/CreateOntology.java | 44 +--- .../admin/ontologies/ImportOntology.java | 25 +-- .../cli/command/content/AddObjectBlock.java | 38 +--- .../cli/command/content/AddXHTMLBlock.java | 35 +-- .../cli/command/content/RemoveBlock.java | 2 +- .../cli/command/imports/AddCSVImport.java | 37 +--- .../cli/command/imports/AddRDFImport.java | 37 +--- .../cli/command/imports/ImportCSV.java | 2 +- .../cli/command/imports/ImportRDF.java | 2 +- .../linkeddatahub/cli/http/LDHClient.java | 2 +- .../linkeddatahub/cli/util/PushPlan.java | 5 +- .../linkeddatahub/cli/util/URIRewriter.java | 58 +---- .../linkeddatahub/cli/PushOutputTest.java | 4 +- .../cli/util/URIRewriterTest.java | 25 --- make/config.mk | 2 +- make/local.mk | 20 +- rdf/pom.xml | 191 +++++++++++++++++ .../com/atomgraph/linkeddatahub/rdf/Acl.java | 120 +++++++++++ .../atomgraph/linkeddatahub/rdf/Blocks.java | 96 +++++++++ .../atomgraph/linkeddatahub/rdf}/Digests.java | 2 +- .../linkeddatahub/rdf/Documents.java | 105 +++++++++ .../atomgraph/linkeddatahub/rdf/Imports.java | 98 +++++++++ .../linkeddatahub/rdf/Ontologies.java | 199 ++++++++++++++++++ .../atomgraph/linkeddatahub/rdf/Queries.java | 65 ++++++ .../linkeddatahub/rdf}/SequenceNumbers.java | 2 +- .../atomgraph/linkeddatahub/rdf/Services.java | 69 ++++++ .../atomgraph/linkeddatahub/rdf}/Slugs.java | 2 +- .../atomgraph/linkeddatahub/rdf/Subjects.java | 51 +++++ .../com/atomgraph/linkeddatahub/rdf/URIs.java | 87 ++++++++ .../atomgraph/linkeddatahub/rdf}/Updates.java | 2 +- .../atomgraph/linkeddatahub/rdf/Views.java | 94 +++++++++ .../linkeddatahub/rdf/vocabulary}/A.java | 2 +- .../linkeddatahub/rdf/vocabulary}/AC.java | 2 +- .../linkeddatahub/rdf/vocabulary}/ACL.java | 2 +- .../linkeddatahub/rdf/vocabulary}/DH.java | 2 +- .../linkeddatahub/rdf/vocabulary}/LDH.java | 2 +- .../linkeddatahub/rdf/vocabulary}/NFO.java | 2 +- .../linkeddatahub/rdf/vocabulary}/SD.java | 2 +- .../linkeddatahub/rdf/vocabulary}/SP.java | 2 +- .../linkeddatahub/rdf/vocabulary}/SPIN.java | 2 +- .../linkeddatahub/rdf}/ModelBuildersTest.java | 70 +++--- .../rdf}/SequenceNumbersTest.java | 2 +- .../atomgraph/linkeddatahub/rdf/URIsTest.java | 54 +++++ .../linkeddatahub/rdf}/UpdatesTest.java | 2 +- release.sh | 39 +++- 71 files changed, 1476 insertions(+), 853 deletions(-) create mode 100644 rdf/pom.xml create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Acl.java create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Blocks.java rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/util => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf}/Digests.java (97%) create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Documents.java create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Imports.java create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Ontologies.java create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Queries.java rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/util => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf}/SequenceNumbers.java (98%) create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Services.java rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/util => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf}/Slugs.java (96%) create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Subjects.java create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/URIs.java rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/sparql => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf}/Updates.java (99%) create mode 100644 rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Views.java rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/A.java (96%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/AC.java (95%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/ACL.java (97%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/DH.java (96%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/LDH.java (98%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/NFO.java (96%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/SD.java (97%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/SP.java (96%) rename {cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab => rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary}/SPIN.java (96%) rename {cli/src/test/java/com/atomgraph/linkeddatahub/cli/command => rdf/src/test/java/com/atomgraph/linkeddatahub/rdf}/ModelBuildersTest.java (80%) rename {cli/src/test/java/com/atomgraph/linkeddatahub/cli/util => rdf/src/test/java/com/atomgraph/linkeddatahub/rdf}/SequenceNumbersTest.java (98%) create mode 100644 rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/URIsTest.java rename {cli/src/test/java/com/atomgraph/linkeddatahub/cli/sparql => rdf/src/test/java/com/atomgraph/linkeddatahub/rdf}/UpdatesTest.java (98%) diff --git a/.gitignore b/.gitignore index ac8300205..d8d44e3b7 100644 --- a/.gitignore +++ b/.gitignore @@ -25,3 +25,4 @@ /tests/ui/node_modules /tests/ui/out /tests/ui/test-results +/rdf/target/ diff --git a/CLAUDE.md b/CLAUDE.md index 723a28009..c2e17250f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -39,8 +39,9 @@ make down # Stop the services make down -- -v # Stop and remove volumes make drop # Complete reset (down -v, then wipe local dirs) -make cli # Build the ldh CLI, print the PATH export to run -make cli-version # Set cli/pom.xml to the platform version in pom.xml +make rdf # Install the linkeddatahub-rdf library +make cli # Build the ldh CLI (installs rdf first), print the PATH export +make cli-version # Set rdf/ and cli/ poms to the platform version in pom.xml ``` ### Testing @@ -155,15 +156,42 @@ into a shaded `cli/target/ldh.jar` that `cli/bin/ldh` launches. See `cli/README. script → command table and the behavioral differences from the scripts. ```bash -cd cli && mvn package && export PATH="$PWD/bin:$PATH" +make cli && export PATH="$PWD/cli/bin:$PATH" ldh create container --parent "$LDH_BASE" --title "Some" --slug some ldh admin add agent --agent "$AGENT_URI" "${ADMIN_BASE}acl/groups/writers/" ``` -`cli/` is not a module of the platform reactor (the root pom is the webapp artifact, so it cannot -carry ``), but it shares the platform's version: `release.sh` runs `versions:set` on it -around both release bumps, and `make cli-version` re-aligns it if it drifts. +## The RDF library + +`rdf/` builds `com.atomgraph:linkeddatahub-rdf` — the vocabularies (`com.atomgraph.linkeddatahub.rdf.vocabulary`) +and the document shapes the HTTP API accepts (`Documents`, `Blocks`, `Views`, `Queries`, `Services`, +`Imports`, `Acl`, `Ontologies`, plus the SPARQL `Updates`). Jena and nothing else: no Jersey, no +picocli. Every consumer talks to the platform over HTTP its own way, so what is shared is the shape +of the request body, not how it is sent. + +Two consumers today — the CLI, and Web-Algebra's `ldh-*` operations in `../REST-VKG` — which is why +the builders live here rather than on the picocli command classes that used to own them. A shape +described in one place cannot drift between them. + +The platform keeps its own `com.atomgraph.linkeddatahub.vocabulary`, still duplicated with this +library's. Collapsing them would make the platform depend on `rdf/`, and the Dockerfile builds the +webapp from `COPY src` + `COPY pom.xml` alone — so the dependency has to point away from the +platform, not at it. + +Neither `rdf/` nor `cli/` is a module of the platform reactor (the root pom is the webapp artifact, +so it cannot carry ``), but both share the platform's version: `release.sh` runs +`versions:set` on them around both release bumps, and `make cli-version` re-aligns them if they +drift. `cli/pom.xml` resolves the library by `${project.version}`, so the two move together. +`make cli` installs `rdf/` first; building `cli/` on its own needs `make rdf` to have run at least +once since the last version bump. + +`linkeddatahub-rdf` publishes to Maven Central on its own, since `release:perform` only carries +reactor modules: `release.sh` deploys it right after the platform, extracting `rdf/` from the release +tag with `git archive` because the working tree has already moved on to the next SNAPSHOT by then. +A snapshot can be published by hand with `cd rdf && mvn -Prelease clean deploy` — the `release` +profile attaches the sources and javadoc jars and signs them, which is what +`release` gets the platform. `LDH_CERT_FILE`, `LDH_CERT_PASSWORD`, `LDH_BASE` and `LDH_PROXY` supply defaults for `-c`, `-p`, `-b` and `--proxy`. `-c/--cert` takes either format the agent's credential comes in — a PKCS12 diff --git a/cli/README.md b/cli/README.md index cef2eb3a7..08d8bfbfb 100644 --- a/cli/README.md +++ b/cli/README.md @@ -46,11 +46,17 @@ make cli which prints the `export PATH=...` line to run afterwards. It is the equivalent of: ```bash +cd rdf && mvn install && cd .. cd cli mvn package export PATH="$PWD/bin:$PATH" ``` +The `rdf/` step installs `com.atomgraph:linkeddatahub-rdf`, which holds the vocabularies and the +document shapes the commands build — shared with Web-Algebra's `ldh-*` operations, so the two +cannot drift. It is resolved by `${project.version}`, so it has to be installed again after a +version bump; `make cli` does that for you. + This produces the self-contained `target/ldh.jar`, which the `cli/bin/ldh` launcher runs. The launcher prefers `LDH_JAR`, then a jar sitting beside it (the release archive layout), then `../target/ldh.jar` (the source checkout layout). diff --git a/cli/pom.xml b/cli/pom.xml index 4b7b1a567..72fc6d4e4 100644 --- a/cli/pom.xml +++ b/cli/pom.xml @@ -17,7 +17,32 @@ 3.1.11 + + + + central-portal-snapshots + https://central.sonatype.com/repository/maven-snapshots/ + + false + + + true + + + + + + + com.atomgraph + linkeddatahub-rdf + ${project.version} + info.picocli picocli diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/BaseCommand.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/BaseCommand.java index 972860578..19e4a0f50 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/BaseCommand.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/BaseCommand.java @@ -32,7 +32,6 @@ import java.util.concurrent.Callable; import org.apache.jena.rdf.model.Model; import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; import org.apache.jena.riot.Lang; import org.apache.jena.riot.RDFLanguages; import org.apache.jena.riot.RDFParser; @@ -124,20 +123,6 @@ protected static void put(LDHClient client, URI target, Model model) HttpException.check(target, client.put(target, Entity.entity(model, TEXT_TURTLE_TYPE), ACCEPT_TURTLE)).close(); } - /** - * Returns the subject resource for an appended description: the --uri value - * resolved against the target document URI, or a fresh blank node when not given. - * - * @param model model to create the resource in - * @param target target document URI - * @param uri --uri option value (absolute or relative, can be null) - * @return subject resource - */ - protected static Resource createSubject(Model model, URI target, String uri) - { - return uri != null ? model.createResource(target.resolve(uri).toString()) : model.createResource(); - } - /** * Parses an RDF stream into a model, resolving relative URIs against the base URI * (the equivalent of the scripts' turtle --base piping). diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddConstruct.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddConstruct.java index ec84ddecf..38f5f7eab 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddConstruct.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddConstruct.java @@ -19,16 +19,11 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.LDHClient; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Queries; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -88,33 +83,7 @@ public Integer call() throws Exception */ public static void core(LDHClient client, URI target, String uri, String title, String queryText, URI service, String description) { - post(client, target, buildModel(target, uri, SP.Construct, title, queryText, service, description)); - } - - /** - * Builds a SPIN query description. - * - * @param target target document URI - * @param uri query URI (optional) - * @param queryType SPIN query class (sp:Construct or sp:Select) - * @param title query title - * @param queryText query string - * @param service SPARQL service URI (optional) - * @param description query description (optional) - * @return query model - */ - public static Model buildModel(URI target, String uri, Resource queryType, String title, String queryText, URI service, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource query = createSubject(model, target, uri). - addProperty(RDF.type, queryType). - addProperty(DCTerms.title, title). - addProperty(SP.text, queryText); - if (service != null) query.addProperty(LDH.service, model.createResource(service.toString())); - if (description != null) query.addProperty(DCTerms.description, description); - - return model; + post(client, target, Queries.query(target, uri, SP.Construct, title, queryText, service, description)); } } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java index 7872d4529..a3f5df769 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddFile.java @@ -20,8 +20,8 @@ import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.http.LDHClient; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Digests; -import com.atomgraph.linkeddatahub.cli.vocab.NFO; +import com.atomgraph.linkeddatahub.rdf.Digests; +import com.atomgraph.linkeddatahub.rdf.vocabulary.NFO; import jakarta.ws.rs.client.Entity; import jakarta.ws.rs.core.MediaType; import java.io.IOException; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddGenericService.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddGenericService.java index 554b41f0c..7952b5307 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddGenericService.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddGenericService.java @@ -18,14 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.A; -import com.atomgraph.linkeddatahub.cli.vocab.SD; +import com.atomgraph.linkeddatahub.rdf.Services; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -72,41 +66,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, title, endpoint, graphStore, authUser, authPwd, description)); + post(getClient(), target, Services.service(target, uri, title, endpoint, graphStore, authUser, authPwd, description)); print(target); return 0; } - /** - * Builds the service description. - * - * @param target target document URI - * @param uri service URI (optional) - * @param title service title - * @param endpoint SPARQL endpoint URI - * @param graphStore Graph Store Protocol endpoint URI (optional) - * @param authUser HTTP Basic auth username (optional) - * @param authPwd HTTP Basic auth password (optional) - * @param description service description (optional) - * @return service model - */ - public static Model buildModel(URI target, String uri, String title, URI endpoint, URI graphStore, String authUser, String authPwd, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource service = createSubject(model, target, uri). - addProperty(RDF.type, SD.Service). - addProperty(DCTerms.title, title). - addProperty(SD.endpoint, model.createResource(endpoint.toString())). - addProperty(SD.supportedLanguage, SD.SPARQL11Query). - addProperty(SD.supportedLanguage, SD.SPARQL11Update); - if (graphStore != null) service.addProperty(A.graphStore, model.createResource(graphStore.toString())); - if (authUser != null) service.addProperty(A.authUser, authUser); - if (authPwd != null) service.addProperty(A.authPwd, authPwd); - if (description != null) service.addProperty(DCTerms.description, description); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddPackageImport.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddPackageImport.java index 698c3652c..189b72270 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddPackageImport.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddPackageImport.java @@ -19,7 +19,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddResultSetChart.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddResultSetChart.java index ee9ebeb9c..cb3eefb0a 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddResultSetChart.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddResultSetChart.java @@ -18,14 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Views; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -72,39 +66,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, title, query, chartType, categoryVarName, seriesVarName, description)); + post(getClient(), target, Views.resultSetChart(target, uri, title, query, chartType, categoryVarName, seriesVarName, description)); print(target); return 0; } - /** - * Builds the chart description. - * - * @param target target document URI - * @param uri chart URI (optional) - * @param title chart title - * @param query SELECT query URI - * @param chartType chart type URI - * @param categoryVarName category variable name - * @param seriesVarName series variable name - * @param description chart description (optional) - * @return chart model - */ - public static Model buildModel(URI target, String uri, String title, URI query, URI chartType, String categoryVarName, String seriesVarName, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource chart = createSubject(model, target, uri). - addProperty(RDF.type, LDH.ResultSetChart). - addProperty(DCTerms.title, title). - addProperty(SPIN.query, model.createResource(query.toString())). - addProperty(LDH.chartType, model.createResource(chartType.toString())). - addProperty(LDH.categoryVarName, categoryVarName). - addProperty(LDH.seriesVarName, seriesVarName); - if (description != null) chart.addProperty(DCTerms.description, description); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddSelect.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddSelect.java index cbcd857ca..0e3530f48 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddSelect.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddSelect.java @@ -18,7 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Queries; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; @@ -62,7 +63,7 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, AddConstruct.buildModel(target, uri, SP.Select, title, Files.readString(queryFile), service, description)); + post(getClient(), target, Queries.query(target, uri, SP.Select, title, Files.readString(queryFile), service, description)); print(target); return 0; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddView.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddView.java index ab21f69ce..d6c313370 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddView.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/AddView.java @@ -18,15 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.AC; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Views; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -67,35 +60,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, query, title, description, mode)); + post(getClient(), target, Views.view(target, uri, query, title, description, mode)); print(target); return 0; } - /** - * Builds the view description. - * - * @param target target document URI - * @param uri view URI (optional) - * @param query SELECT query URI - * @param title view title (optional) - * @param description view description (optional) - * @param mode layout mode URI (optional) - * @return view model - */ - public static Model buildModel(URI target, String uri, URI query, String title, String description, URI mode) - { - Model model = ModelFactory.createDefaultModel(); - - Resource view = createSubject(model, target, uri). - addProperty(RDF.type, LDH.View). - addProperty(SPIN.query, model.createResource(query.toString())); - if (title != null) view.addProperty(DCTerms.title, title); - if (description != null) view.addProperty(DCTerms.description, description); - if (mode != null) view.addProperty(AC.mode, model.createResource(mode.toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateContainer.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateContainer.java index 1a61d2014..b1ec21a46 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateContainer.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateContainer.java @@ -18,19 +18,10 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import com.atomgraph.linkeddatahub.cli.vocab.AC; -import com.atomgraph.linkeddatahub.cli.vocab.DH; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Documents; +import com.atomgraph.linkeddatahub.rdf.Slugs; +import com.atomgraph.linkeddatahub.rdf.URIs; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -73,49 +64,11 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - URI doc = URIRewriter.childURI(parent, slug != null ? slug : Slugs.defaultSlug()); - put(getClient(), doc, buildModel(doc, title, description, block, mode, primaryTopic)); + URI doc = URIs.childURI(parent, slug != null ? slug : Slugs.defaultSlug()); + put(getClient(), doc, Documents.container(doc, title, description, block, mode, primaryTopic)); print(doc); return 0; } - /** - * Builds the container document model with its first content block: the given block URI, - * a children view with an explicit mode, or the default children view. - * - * @param doc document URI - * @param title document title - * @param description document description (optional) - * @param block content block URI (optional) - * @param mode children view mode URI (optional, ignored when block is given) - * @param primaryTopic URI of the document's primary topic, relative or absolute (optional) - * @return document model - */ - public static Model buildModel(URI doc, String title, String description, URI block, URI mode, String primaryTopic) - { - Model model = ModelFactory.createDefaultModel(); - - Resource container = model.createResource(doc.toString()). - addProperty(RDF.type, DH.Container). - addProperty(DCTerms.title, title); - - if (block != null) container.addProperty(RDF.li(1), model.createResource(block.toString())); - else if (mode != null) container.addProperty(RDF.li(1), model.createResource(). - addProperty(RDF.type, LDH.Object). - addProperty(RDF.value, model.createResource(). - addProperty(RDF.type, LDH.View). - addProperty(SPIN.query, LDH.SelectChildren). - addProperty(AC.mode, model.createResource(mode.toString())))); - else container.addProperty(RDF.li(1), model.createResource(). - addProperty(RDF.type, LDH.Object). - addProperty(RDF.value, LDH.ChildrenView)); - - if (description != null) container.addProperty(DCTerms.description, description); - // See CreateItem.buildModel: resolved against the document, and singular by the vocabulary. - if (primaryTopic != null) container.addProperty(FOAF.primaryTopic, model.createResource(doc.resolve(primaryTopic).toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateItem.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateItem.java index 40b3dd3f7..526342dc3 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateItem.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/CreateItem.java @@ -18,16 +18,10 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import com.atomgraph.linkeddatahub.cli.vocab.DH; +import com.atomgraph.linkeddatahub.rdf.Documents; +import com.atomgraph.linkeddatahub.rdf.Slugs; +import com.atomgraph.linkeddatahub.rdf.URIs; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -64,37 +58,11 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - URI doc = URIRewriter.childURI(container, slug != null ? slug : Slugs.defaultSlug()); - put(getClient(), doc, buildModel(doc, title, description, primaryTopic)); + URI doc = URIs.childURI(container, slug != null ? slug : Slugs.defaultSlug()); + put(getClient(), doc, Documents.item(doc, title, description, primaryTopic)); print(doc); return 0; } - /** - * Builds the item document model. - * - * @param doc document URI - * @param title document title - * @param description document description (optional) - * @param primaryTopic URI of the document's primary topic, relative or absolute (optional) - * @return document model - */ - public static Model buildModel(URI doc, String title, String description, String primaryTopic) - { - Model model = ModelFactory.createDefaultModel(); - - Resource item = model.createResource(doc.toString()). - addProperty(RDF.type, DH.Item). - addProperty(DCTerms.title, title); - if (description != null) item.addProperty(DCTerms.description, description); - // Resolved against the document, so the conventional fragment topic is "#this" and a - // document about something described elsewhere takes that resource's absolute URI. - // Singular because foaf:primaryTopic is an owl:FunctionalProperty: a second value would - // not mean a second topic, it would entail the two topics are the same resource. - if (primaryTopic != null) item.addProperty(FOAF.primaryTopic, model.createResource(doc.resolve(primaryTopic).toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/ListPackages.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/ListPackages.java index 8823f8549..63713c656 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/ListPackages.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/ListPackages.java @@ -19,7 +19,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; import jakarta.ws.rs.core.MediaType; import jakarta.ws.rs.core.Response; import java.net.URI; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/RemovePackageImport.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/RemovePackageImport.java index 191f06b91..ab717ffb8 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/RemovePackageImport.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/RemovePackageImport.java @@ -19,7 +19,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/AddOntologyImport.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/AddOntologyImport.java index f54d246ec..15e8d939d 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/AddOntologyImport.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/AddOntologyImport.java @@ -18,7 +18,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Option; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/AddAgentToGroup.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/AddAgentToGroup.java index a2a59dce7..fbcbfaedd 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/AddAgentToGroup.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/AddAgentToGroup.java @@ -18,7 +18,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Option; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateAuthorization.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateAuthorization.java index 8fbc01c6d..1fabd986b 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateAuthorization.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateAuthorization.java @@ -18,21 +18,14 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import com.atomgraph.linkeddatahub.cli.vocab.ACL; -import com.atomgraph.linkeddatahub.cli.vocab.DH; +import com.atomgraph.linkeddatahub.rdf.Acl; +import com.atomgraph.linkeddatahub.rdf.Slugs; +import com.atomgraph.linkeddatahub.rdf.URIs; +import com.atomgraph.linkeddatahub.rdf.vocabulary.ACL; import java.net.URI; import java.util.ArrayList; import java.util.List; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -100,7 +93,7 @@ public Integer call() throws Exception if (!append && !control && !read && !write) throw new ParameterException(getSpec().commandLine(), "At least one of '--append', '--control', '--read', '--write' is required"); - URI doc = URIRewriter.childURI(URI.create(base + "acl/authorizations/"), slug != null ? slug : Slugs.defaultSlug()); + URI doc = URIs.childURI(URI.create(base + "acl/authorizations/"), slug != null ? slug : Slugs.defaultSlug()); List modes = new ArrayList<>(); if (append) modes.add(ACL.Append); @@ -108,56 +101,10 @@ public Integer call() throws Exception if (read) modes.add(ACL.Read); if (write) modes.add(ACL.Write); - put(getClient(), doc, buildModel(doc, uri, label, comment, agents, agentClasses, agentGroups, to, toAllIn, modes)); + put(getClient(), doc, Acl.authorization(doc, uri, label, comment, agents, agentClasses, agentGroups, to, toAllIn, modes)); print(doc); return 0; } - /** - * Builds the authorization document model. - * - * @param doc document URI - * @param uri authorization URI (optional, blank node if null) - * @param label authorization label - * @param comment authorization comment (optional) - * @param agents authorized agent URIs - * @param agentClasses authorized agent class URIs - * @param agentGroups authorized agent group URIs - * @param to accessed document URIs - * @param toAllIn accessed document class URIs - * @param modes granted access modes - * @return document model - */ - public static Model buildModel(URI doc, String uri, String label, String comment, - List agents, List agentClasses, List agentGroups, - List to, List toAllIn, List modes) - { - Model model = ModelFactory.createDefaultModel(); - - Resource auth = createSubject(model, doc, uri). - addProperty(RDF.type, ACL.Authorization). - addProperty(RDFS.label, label); - if (comment != null) auth.addProperty(RDFS.comment, comment); - - model.createResource(doc.toString()). - addProperty(RDF.type, DH.Item). - addProperty(FOAF.primaryTopic, auth). - addProperty(DCTerms.title, label); - - addResourceValues(auth, ACL.agent, agents); - addResourceValues(auth, ACL.agentClass, agentClasses); - addResourceValues(auth, ACL.agentGroup, agentGroups); - addResourceValues(auth, ACL.accessTo, to); - addResourceValues(auth, ACL.accessToClass, toAllIn); - modes.forEach(mode -> auth.addProperty(ACL.mode, mode)); - - return model; - } - - static void addResourceValues(Resource subject, Property property, List values) - { - values.forEach(value -> subject.addProperty(property, subject.getModel().createResource(value.toString()))); - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateGroup.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateGroup.java index ca35618fb..a6988a66a 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateGroup.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/CreateGroup.java @@ -18,17 +18,11 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import com.atomgraph.linkeddatahub.cli.vocab.DH; +import com.atomgraph.linkeddatahub.rdf.Acl; +import com.atomgraph.linkeddatahub.rdf.Slugs; +import com.atomgraph.linkeddatahub.rdf.URIs; import java.net.URI; import java.util.List; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -64,40 +58,12 @@ public class CreateGroup extends BaseCommand public Integer call() throws Exception { URI base = baseMixin.require(getSpec()); - URI doc = URIRewriter.childURI(URI.create(base + "acl/groups/"), slug != null ? slug : Slugs.defaultSlug()); + URI doc = URIs.childURI(URI.create(base + "acl/groups/"), slug != null ? slug : Slugs.defaultSlug()); - put(getClient(), doc, buildModel(doc, uri, name, description, members)); + put(getClient(), doc, Acl.group(doc, uri, name, description, members)); print(doc); return 0; } - /** - * Builds the group document model. - * - * @param doc document URI - * @param uri group URI (optional, blank node if null) - * @param name group name - * @param description group description (optional) - * @param members member agent URIs - * @return document model - */ - public static Model buildModel(URI doc, String uri, String name, String description, List members) - { - Model model = ModelFactory.createDefaultModel(); - - Resource group = createSubject(model, doc, uri). - addProperty(RDF.type, FOAF.Group). - addProperty(FOAF.name, name); - if (description != null) group.addProperty(DCTerms.description, description); - members.forEach(member -> group.addProperty(FOAF.member, model.createResource(member.toString()))); - - model.createResource(doc.toString()). - addProperty(RDF.type, DH.Item). - addProperty(FOAF.primaryTopic, group). - addProperty(DCTerms.title, name); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/MakePublic.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/MakePublic.java index a041e3521..86f76da27 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/MakePublic.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/acl/MakePublic.java @@ -19,8 +19,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; +import com.atomgraph.linkeddatahub.rdf.URIs; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; @@ -41,7 +41,7 @@ public class MakePublic extends BaseCommand public Integer call() throws Exception { URI base = baseMixin.require(getSpec()); - URI adminBase = URIRewriter.adminBase(base); + URI adminBase = URIs.adminBase(base); URI target = URI.create(adminBase + "acl/authorizations/public/"); HttpException.check(target, getClient().patch(target, Updates.makePublic(base, adminBase))).close(); @@ -53,7 +53,7 @@ public Integer call() throws Exception protected URI getEffectiveProxy() { // the request targets the admin app, so the proxy origin gets the admin subdomain too - return getProxyMixin().getProxy() != null ? URIRewriter.adminBase(getProxyMixin().getProxy()) : null; + return getProxyMixin().getProxy() != null ? URIs.adminBase(getProxyMixin().getProxy()) : null; } } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddClass.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddClass.java index 830330957..dc579add3 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddClass.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddClass.java @@ -18,16 +18,10 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Ontologies; import java.net.URI; import java.util.ArrayList; import java.util.List; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.OWL; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -71,37 +65,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, label, comment, constructor, constraint, superClasses)); + post(getClient(), target, Ontologies.owlClass(target, uri, label, comment, constructor, constraint, superClasses)); print(target); return 0; } - /** - * Builds the class description. - * - * @param target target document URI - * @param uri class URI (optional) - * @param label class label - * @param comment class comment (optional) - * @param constructor constructor query URI (optional) - * @param constraint constraint URI (optional) - * @param superClasses superclass URIs - * @return class model - */ - public static Model buildModel(URI target, String uri, String label, String comment, URI constructor, URI constraint, List superClasses) - { - Model model = ModelFactory.createDefaultModel(); - - Resource cls = createSubject(model, target, uri). - addProperty(RDF.type, OWL.Class). - addProperty(RDFS.label, label); - if (comment != null) cls.addProperty(RDFS.comment, comment); - if (constructor != null) cls.addProperty(SPIN.constructor, model.createResource(constructor.toString())); - if (constraint != null) cls.addProperty(SPIN.constraint, model.createResource(constraint.toString())); - superClasses.forEach(superClass -> cls.addProperty(RDFS.subClassOf, model.createResource(superClass.toString()))); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddConstructor.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddConstructor.java index ec1d1eb60..47c63cfc6 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddConstructor.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddConstructor.java @@ -18,16 +18,11 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Ontologies; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -65,37 +60,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, SP.Construct, label, Files.readString(queryFile), null, comment)); + post(getClient(), target, Ontologies.constructor(target, uri, SP.Construct, label, Files.readString(queryFile), null, comment)); print(target); return 0; } - /** - * Builds an ontology SPIN query description (labeled with rdfs:label, - * unlike the dct:title-based document queries). - * - * @param target target document URI - * @param uri query URI (optional) - * @param queryType SPIN query class (sp:Construct or sp:Select) - * @param label query label - * @param queryText query string - * @param service SPARQL service URI (optional) - * @param comment query comment (optional) - * @return query model - */ - public static Model buildModel(URI target, String uri, Resource queryType, String label, String queryText, URI service, String comment) - { - Model model = ModelFactory.createDefaultModel(); - - Resource query = createSubject(model, target, uri). - addProperty(RDF.type, queryType). - addProperty(RDFS.label, label). - addProperty(SP.text, queryText); - if (comment != null) query.addProperty(RDFS.comment, comment); - if (service != null) query.addProperty(LDH.service, model.createResource(service.toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddPropertyConstraint.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddPropertyConstraint.java index eafc655b2..f33f0608d 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddPropertyConstraint.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddPropertyConstraint.java @@ -18,14 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Ontologies; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -63,33 +57,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, label, property, comment)); + post(getClient(), target, Ontologies.propertyConstraint(target, uri, label, property, comment)); print(target); return 0; } - /** - * Builds the constraint description. - * - * @param target target document URI - * @param uri constraint URI (optional) - * @param label constraint label - * @param property required property URI - * @param comment constraint comment (optional) - * @return constraint model - */ - public static Model buildModel(URI target, String uri, String label, URI property, String comment) - { - Model model = ModelFactory.createDefaultModel(); - - Resource constraint = createSubject(model, target, uri). - addProperty(RDF.type, LDH.MissingPropertyValue). - addProperty(RDFS.label, label). - addProperty(SP.arg1, model.createResource(property.toString())); - if (comment != null) constraint.addProperty(RDFS.comment, comment); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddRestriction.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddRestriction.java index 9ac8ae79c..10cbd2613 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddRestriction.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddRestriction.java @@ -18,13 +18,9 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; +import com.atomgraph.linkeddatahub.rdf.Ontologies; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; import org.apache.jena.vocabulary.OWL; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -68,37 +64,10 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, buildModel(target, uri, label, comment, onProperty, allValuesFrom, hasValue)); + post(getClient(), target, Ontologies.restriction(target, uri, label, comment, onProperty, allValuesFrom, hasValue)); print(target); return 0; } - /** - * Builds the restriction description. - * - * @param target target document URI - * @param uri restriction URI (optional) - * @param label restriction label - * @param comment restriction comment (optional) - * @param onProperty restricted property URI (optional) - * @param allValuesFrom value class URI (optional) - * @param hasValue value resource URI (optional) - * @return restriction model - */ - public static Model buildModel(URI target, String uri, String label, String comment, URI onProperty, URI allValuesFrom, URI hasValue) - { - Model model = ModelFactory.createDefaultModel(); - - Resource restriction = createSubject(model, target, uri). - addProperty(RDF.type, OWL.Restriction). - addProperty(RDFS.label, label); - if (comment != null) restriction.addProperty(RDFS.comment, comment); - if (onProperty != null) restriction.addProperty(OWL.onProperty, model.createResource(onProperty.toString())); - if (allValuesFrom != null) restriction.addProperty(OWL.allValuesFrom, model.createResource(allValuesFrom.toString())); - if (hasValue != null) restriction.addProperty(OWL.hasValue, model.createResource(hasValue.toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddSelect.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddSelect.java index 323e9ab34..10467adfe 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddSelect.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/AddSelect.java @@ -18,7 +18,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Ontologies; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; @@ -62,7 +63,7 @@ public Integer call() throws Exception { baseMixin.require(getSpec()); // required by the script interface - post(getClient(), target, AddConstructor.buildModel(target, uri, SP.Select, label, Files.readString(queryFile), service, comment)); + post(getClient(), target, Ontologies.constructor(target, uri, SP.Select, label, Files.readString(queryFile), service, comment)); print(target); return 0; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/CreateOntology.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/CreateOntology.java index 9b4312e75..0431d906b 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/CreateOntology.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/CreateOntology.java @@ -18,18 +18,10 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; -import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import com.atomgraph.linkeddatahub.cli.vocab.DH; +import com.atomgraph.linkeddatahub.rdf.Ontologies; +import com.atomgraph.linkeddatahub.rdf.Slugs; +import com.atomgraph.linkeddatahub.rdf.URIs; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.OWL; -import org.apache.jena.vocabulary.RDF; -import org.apache.jena.vocabulary.RDFS; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -62,38 +54,12 @@ public class CreateOntology extends BaseCommand public Integer call() throws Exception { URI base = baseMixin.require(getSpec()); - URI doc = URIRewriter.childURI(URI.create(base + "ontologies/"), slug != null ? slug : Slugs.defaultSlug()); + URI doc = URIs.childURI(URI.create(base + "ontologies/"), slug != null ? slug : Slugs.defaultSlug()); - put(getClient(), doc, buildModel(doc, uri, label, comment)); + put(getClient(), doc, Ontologies.ontology(doc, uri, label, comment)); print(doc); return 0; } - /** - * Builds the ontology document model. - * - * @param doc document URI - * @param uri ontology URI (optional, blank node if null) - * @param label ontology label - * @param comment ontology comment (optional) - * @return document model - */ - public static Model buildModel(URI doc, String uri, String label, String comment) - { - Model model = ModelFactory.createDefaultModel(); - - Resource ontology = createSubject(model, doc, uri). - addProperty(RDF.type, OWL.Ontology). - addProperty(RDFS.label, label); - if (comment != null) ontology.addProperty(RDFS.comment, comment); - - model.createResource(doc.toString()). - addProperty(RDF.type, DH.Item). - addProperty(FOAF.primaryTopic, ontology). - addProperty(DCTerms.title, label); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/ImportOntology.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/ImportOntology.java index ebde26ec5..b187dda50 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/ImportOntology.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/admin/ontologies/ImportOntology.java @@ -19,7 +19,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +import com.atomgraph.linkeddatahub.rdf.Ontologies; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import jakarta.ws.rs.core.Form; import jakarta.ws.rs.core.MediaType; import jakarta.ws.rs.core.Response; @@ -27,9 +28,7 @@ import java.net.URLEncoder; import java.nio.charset.StandardCharsets; import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; import org.apache.jena.rdf.model.Resource; -import org.apache.jena.sparql.vocabulary.FOAF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -77,7 +76,7 @@ public Integer call() throws Exception // the vocabulary goes into the target first, so the CONSTRUCT has it to read and it stays afterwards post(getClient(), graph, vocabulary); post(getClient(), graph, construct(base, query, graph)); - post(getClient(), graph, buildAnnotationModel(graph, source)); + post(getClient(), graph, Ontologies.annotation(graph, source)); print(graph); @@ -141,22 +140,4 @@ protected Model construct(URI base, String query, URI graph) } } - /** - * Builds the arc saying what the document is about. The document is not the ontology, so it takes - * no owl:Ontology type of its own: the vocabulary stored alongside carries that. - * - * @param graph target document URI - * @param source imported ontology URI - * @return primary topic model - */ - public static Model buildAnnotationModel(URI graph, URI source) - { - Model model = ModelFactory.createDefaultModel(); - - model.createResource(graph.toString()). - addProperty(FOAF.primaryTopic, model.createResource(source.toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddObjectBlock.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddObjectBlock.java index 534554861..71a5a1a94 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddObjectBlock.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddObjectBlock.java @@ -19,18 +19,13 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.SequenceNumbers; -import com.atomgraph.linkeddatahub.cli.vocab.AC; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; +import com.atomgraph.linkeddatahub.rdf.Blocks; +import com.atomgraph.linkeddatahub.rdf.SequenceNumbers; import jakarta.ws.rs.core.Response; import java.net.URI; import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; import org.apache.jena.rdf.model.Property; -import org.apache.jena.rdf.model.Resource; import org.apache.jena.rdf.model.ResourceFactory; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -69,7 +64,7 @@ public class AddObjectBlock extends BaseCommand @Override public Integer call() throws Exception { - post(getClient(), target, buildModel(target, nextSequenceProperty(), uri, value, title, description, mode)); + post(getClient(), target, Blocks.object(target, nextSequenceProperty(), uri, value, title, description, mode)); print(target); return 0; @@ -91,31 +86,4 @@ protected Property nextSequenceProperty() return SequenceNumbers.nextSequenceProperty(current, ResourceFactory.createResource(target.toString())); } - /** - * Builds the object block description. - * - * @param target target document URI - * @param seq membership property (rdf:_N) - * @param uri block URI (optional) - * @param value object resource URI - * @param title block title (optional) - * @param description block description (optional) - * @param mode layout mode URI (optional) - * @return block model - */ - public static Model buildModel(URI target, Property seq, String uri, URI value, String title, String description, URI mode) - { - Model model = ModelFactory.createDefaultModel(); - - Resource block = createSubject(model, target, uri). - addProperty(RDF.type, LDH.Object). - addProperty(RDF.value, model.createResource(value.toString())); - model.createResource(target.toString()).addProperty(seq, block); - if (title != null) block.addProperty(DCTerms.title, title); - if (description != null) block.addProperty(DCTerms.description, description); - if (mode != null) block.addProperty(AC.mode, model.createResource(mode.toString())); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddXHTMLBlock.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddXHTMLBlock.java index 4237b7a98..ebb5c595c 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddXHTMLBlock.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/AddXHTMLBlock.java @@ -19,17 +19,13 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.SequenceNumbers; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; +import com.atomgraph.linkeddatahub.rdf.Blocks; +import com.atomgraph.linkeddatahub.rdf.SequenceNumbers; import jakarta.ws.rs.core.Response; import java.net.URI; import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; import org.apache.jena.rdf.model.Property; -import org.apache.jena.rdf.model.Resource; import org.apache.jena.rdf.model.ResourceFactory; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -72,35 +68,10 @@ public Integer call() throws Exception } Property seq = SequenceNumbers.nextSequenceProperty(current, ResourceFactory.createResource(target.toString())); - post(getClient(), target, buildModel(target, seq, uri, value, title, description)); + post(getClient(), target, Blocks.xhtml(target, seq, uri, value, title, description)); print(target); return 0; } - /** - * Builds the XHTML block description. - * - * @param target target document URI - * @param seq membership property (rdf:_N) - * @param uri block URI (optional) - * @param value XHTML content - * @param title block title (optional) - * @param description block description (optional) - * @return block model - */ - public static Model buildModel(URI target, Property seq, String uri, String value, String title, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource block = createSubject(model, target, uri). - addProperty(RDF.type, LDH.XHTML). - addProperty(RDF.value, model.createTypedLiteral(value, RDF.dtXMLLiteral)); - model.createResource(target.toString()).addProperty(seq, block); - if (title != null) block.addProperty(DCTerms.title, title); - if (description != null) block.addProperty(DCTerms.description, description); - - return model; - } - } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/RemoveBlock.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/RemoveBlock.java index ee61864de..8c8362491 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/RemoveBlock.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/content/RemoveBlock.java @@ -18,7 +18,7 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.HttpException; -import com.atomgraph.linkeddatahub.cli.sparql.Updates; +import com.atomgraph.linkeddatahub.rdf.Updates; import java.net.URI; import picocli.CommandLine.Command; import picocli.CommandLine.Option; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddCSVImport.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddCSVImport.java index 6a60f63e0..d9edc4547 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddCSVImport.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddCSVImport.java @@ -19,14 +19,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.LDHClient; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Imports; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; -import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; import picocli.CommandLine.Option; @@ -90,34 +84,7 @@ public Integer call() throws Exception */ public static void core(LDHClient client, URI target, String uri, String title, URI query, URI file, String delimiter, String description) { - post(client, target, buildModel(target, uri, title, query, file, delimiter, description)); - } - - /** - * Builds the CSV import description. - * - * @param target target document URI - * @param uri import URI (optional) - * @param title import title - * @param query transformation query URI - * @param file uploaded file URI - * @param delimiter CSV delimiter - * @param description import description (optional) - * @return import model - */ - public static Model buildModel(URI target, String uri, String title, URI query, URI file, String delimiter, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource csvImport = createSubject(model, target, uri). - addProperty(RDF.type, LDH.CSVImport). - addProperty(DCTerms.title, title). - addProperty(SPIN.query, model.createResource(query.toString())). - addProperty(LDH.file, model.createResource(file.toString())). - addProperty(LDH.delimiter, delimiter); - if (description != null) csvImport.addProperty(DCTerms.description, description); - - return model; + post(client, target, Imports.csv(target, uri, title, query, file, delimiter, description)); } } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddRDFImport.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddRDFImport.java index f7d825da2..c57807d6b 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddRDFImport.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/AddRDFImport.java @@ -19,14 +19,8 @@ import com.atomgraph.linkeddatahub.cli.BaseCommand; import com.atomgraph.linkeddatahub.cli.http.LDHClient; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.vocab.LDH; -import com.atomgraph.linkeddatahub.cli.vocab.SD; -import com.atomgraph.linkeddatahub.cli.vocab.SPIN; +import com.atomgraph.linkeddatahub.rdf.Imports; import java.net.URI; -import org.apache.jena.rdf.model.Model; -import org.apache.jena.rdf.model.ModelFactory; -import org.apache.jena.rdf.model.Resource; -import org.apache.jena.vocabulary.DCTerms; import org.apache.jena.vocabulary.RDF; import picocli.CommandLine.Command; import picocli.CommandLine.Mixin; @@ -91,34 +85,7 @@ public Integer call() throws Exception */ public static void core(LDHClient client, URI target, String uri, String title, URI file, URI query, URI graph, String description) { - post(client, target, buildModel(target, uri, title, file, query, graph, description)); - } - - /** - * Builds the RDF import description. - * - * @param target target document URI - * @param uri import URI (optional) - * @param title import title - * @param file uploaded file URI - * @param query transformation query URI (optional) - * @param graph target named graph URI (optional) - * @param description import description (optional) - * @return import model - */ - public static Model buildModel(URI target, String uri, String title, URI file, URI query, URI graph, String description) - { - Model model = ModelFactory.createDefaultModel(); - - Resource rdfImport = createSubject(model, target, uri). - addProperty(RDF.type, LDH.RDFImport). - addProperty(DCTerms.title, title). - addProperty(LDH.file, model.createResource(file.toString())); - if (graph != null) rdfImport.addProperty(SD.name, model.createResource(graph.toString())); - if (query != null) rdfImport.addProperty(SPIN.query, model.createResource(query.toString())); - if (description != null) rdfImport.addProperty(DCTerms.description, description); - - return model; + post(client, target, Imports.rdf(target, uri, title, file, query, graph, description)); } } diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportCSV.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportCSV.java index b40415c2f..86372a82a 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportCSV.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportCSV.java @@ -20,7 +20,7 @@ import com.atomgraph.linkeddatahub.cli.command.AddConstruct; import com.atomgraph.linkeddatahub.cli.command.AddFile; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; +import com.atomgraph.linkeddatahub.rdf.Slugs; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportRDF.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportRDF.java index 08088f41f..c93bde717 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportRDF.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/command/imports/ImportRDF.java @@ -20,7 +20,7 @@ import com.atomgraph.linkeddatahub.cli.command.AddConstruct; import com.atomgraph.linkeddatahub.cli.command.AddFile; import com.atomgraph.linkeddatahub.cli.mixin.BaseMixin; -import com.atomgraph.linkeddatahub.cli.util.Slugs; +import com.atomgraph.linkeddatahub.rdf.Slugs; import java.net.URI; import java.nio.file.Files; import java.nio.file.Path; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/http/LDHClient.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/http/LDHClient.java index 15c07feec..af71a62fb 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/http/LDHClient.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/http/LDHClient.java @@ -19,7 +19,6 @@ import com.atomgraph.core.MediaTypes; import com.atomgraph.core.client.GraphStoreClient; import com.atomgraph.linkeddatahub.cli.util.URIRewriter; -import org.apache.jena.rdf.model.Model; import jakarta.ws.rs.client.Client; import jakarta.ws.rs.client.Entity; import jakarta.ws.rs.client.WebTarget; @@ -31,6 +30,7 @@ import jakarta.ws.rs.core.MultivaluedMap; import jakarta.ws.rs.core.Response; import java.net.URI; +import org.apache.jena.rdf.model.Model; /** * Graph Store Protocol client for LinkedDataHub documents (direct graph identification), diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/PushPlan.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/PushPlan.java index a9c5eda4e..3469a1b6d 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/PushPlan.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/PushPlan.java @@ -16,6 +16,7 @@ package com.atomgraph.linkeddatahub.cli.util; +import com.atomgraph.linkeddatahub.rdf.URIs; import java.io.IOException; import java.net.URI; import java.nio.file.Files; @@ -127,7 +128,7 @@ private static void walk(Path dir, URI url, Path root, Deque ignores, else { String name = doc.getFileName().toString(); - URI docURI = isRootDocument(doc, root) ? url : URIRewriter.childURI(url, stem(name)); + URI docURI = isRootDocument(doc, root) ? url : URIs.childURI(url, stem(name)); steps.add(new Step(Kind.DOCUMENT, doc, relative(doc, root, false), docURI, documentLang(name).getContentType().getContentTypeStr())); } } @@ -138,7 +139,7 @@ private static void walk(Path dir, URI url, Path root, Deque ignores, for (Path subdir : dirs) { if (isIgnored(subdir, true, ignores)) steps.add(skip(subdir, root, true)); - else walk(subdir, URIRewriter.childURI(url, subdir.getFileName().toString()), root, ignores, steps); + else walk(subdir, URIs.childURI(url, subdir.getFileName().toString()), root, ignores, steps); } } finally diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/URIRewriter.java b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/URIRewriter.java index 593181c03..2bb2a603d 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/URIRewriter.java +++ b/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/URIRewriter.java @@ -17,10 +17,16 @@ package com.atomgraph.linkeddatahub.cli.util; import java.net.URI; -import java.nio.charset.StandardCharsets; /** - * URI manipulation helpers matching the conventions of the bin/ shell scripts. + * Sends a request somewhere other than where its URI says. + *

+ * This is the --proxy option: a document's URI identifies it, but the host that + * answers for it may be a different one - the client-certificate port, say, or a tunnel. The + * logical URI stays in the request; only the origin it is sent to changes. + *

+ * A CLI concern, not a document-shape one, which is why it stays here while the URI conventions + * the platform itself applies live in {@link com.atomgraph.linkeddatahub.rdf.URIs}. * * @author Martynas Jusevičius {@literal } */ @@ -48,57 +54,11 @@ public static URI rewrite(URI uri, URI proxy) * @param uri URI * @return origin string, e.g. https://localhost:4443 */ - public static String origin(URI uri) + private static String origin(URI uri) { if (uri.getScheme() == null || uri.getRawAuthority() == null) throw new IllegalArgumentException("URI '" + uri + "' is not absolute"); return uri.getScheme() + "://" + uri.getRawAuthority(); } - /** - * Converts an end-user application base URI to the base URI of its admin application - * by prefixing the host with the admin. subdomain. - * - * @param base end-user base URI - * @return admin base URI - */ - public static URI adminBase(URI base) - { - return URI.create(base.toString().replaceFirst("://", "://admin.")); - } - - /** - * Percent-encodes a string as a URI path segment. All characters except RFC 3986 - * unreserved ones are encoded, including /. - * - * @param slug path segment - * @return encoded path segment - */ - public static String encodeSlug(String slug) - { - StringBuilder sb = new StringBuilder(); - - for (byte b : slug.getBytes(StandardCharsets.UTF_8)) - { - char c = (char)(b & 0xFF); - if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || - c == '-' || c == '.' || c == '_' || c == '~') sb.append(c); - else sb.append('%').append(String.format("%02X", b & 0xFF)); - } - - return sb.toString(); - } - - /** - * Builds the URI of a child document from the parent container URI and a path segment slug. - * - * @param parent parent container URI (with trailing slash) - * @param slug path segment - * @return child document URI (with trailing slash) - */ - public static URI childURI(URI parent, String slug) - { - return URI.create(parent.toString() + encodeSlug(slug) + "/"); - } - } diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/PushOutputTest.java b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/PushOutputTest.java index e5ce8fa3b..30ec07e41 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/PushOutputTest.java +++ b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/PushOutputTest.java @@ -16,9 +16,9 @@ package com.atomgraph.linkeddatahub.cli; -import com.atomgraph.linkeddatahub.cli.http.StubServer; import com.atomgraph.linkeddatahub.cli.http.StubServer.Request; -import com.atomgraph.linkeddatahub.cli.util.Digests; +import com.atomgraph.linkeddatahub.cli.http.StubServer; +import com.atomgraph.linkeddatahub.rdf.Digests; import java.io.IOException; import java.io.StringWriter; import java.net.URI; diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/URIRewriterTest.java b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/URIRewriterTest.java index 222a90863..276d4b536 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/URIRewriterTest.java +++ b/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/URIRewriterTest.java @@ -48,29 +48,4 @@ public void rewriteRejectsRelativeURI() () -> URIRewriter.rewrite(URI.create("/relative/path"), URI.create("https://localhost:8443"))); } - @Test - public void adminBasePrefixesHostWithAdminSubdomain() - { - assertEquals(URI.create("https://admin.localhost:4443/"), URIRewriter.adminBase(URI.create("https://localhost:4443/"))); - } - - @Test - public void encodeSlugKeepsUnreservedCharacters() - { - assertEquals("abc-._~123", URIRewriter.encodeSlug("abc-._~123")); - } - - @Test - public void encodeSlugEncodesReservedAndNonASCII() - { - assertEquals("a%20b%2F%C4%87", URIRewriter.encodeSlug("a b/ć")); - } - - @Test - public void childURIAppendsEncodedSlugAndSlash() - { - assertEquals(URI.create("https://localhost:4443/some/my%20item/"), - URIRewriter.childURI(URI.create("https://localhost:4443/some/"), "my item")); - } - } diff --git a/make/config.mk b/make/config.mk index 3875143b9..dfa550006 100644 --- a/make/config.mk +++ b/make/config.mk @@ -4,7 +4,7 @@ # has no app to install and compiles its client stylesheet from source rather than from a # published image. Everything it adds lives in make/local.mk. -LOCAL_TARGETS := sef release cli cli-version tests ui-tests ui-tests-install load-tests +LOCAL_TARGETS := sef release rdf cli cli-version tests ui-tests ui-tests-install load-tests # only the deployment configuration is RDF worth parsing here; the rest of the tree is source VALIDATE_PATHS := config datasets diff --git a/make/local.mk b/make/local.mk index 54491bfab..c218bbfe3 100644 --- a/make/local.mk +++ b/make/local.mk @@ -15,17 +15,25 @@ sef: release: ./release.sh -# Set cli/pom.xml to the platform version in pom.xml. The CLI ships with the platform release, so -# the two versions are kept in step; release.sh runs this around the release version bumps, and this -# target is for drift and for manual SNAPSHOT bumps +# Set rdf/pom.xml and cli/pom.xml to the platform version in pom.xml. Both ship with the platform +# release, so all three versions are kept in step; release.sh runs this around the release version +# bumps, and this target is for drift and for manual SNAPSHOT bumps cli-version: @version=$$(mvn -q help:evaluate -Dexpression=project.version -DforceStdout); \ - cd cli && mvn -B -q versions:set -DnewVersion="$$version" -DgenerateBackupPoms=false && \ - echo "cli/pom.xml set to $$version" + for project in rdf cli; do \ + (cd $$project && mvn -B -q versions:set -DnewVersion="$$version" -DgenerateBackupPoms=false) && \ + echo "$$project/pom.xml set to $$version"; \ + done + +# Build the linkeddatahub-rdf library: the vocabularies and the document shapes the API accepts. +# Installed rather than packaged because the CLI - and Web-Algebra's ldh-* operations, in their own +# repository - resolve it as an ordinary Maven dependency. +rdf: + cd rdf && mvn -B install # Build the ldh CLI (requires Java 21 and Maven) and print the line that puts it on $PATH. # Released versions are also attached to the GitHub release, which needs neither. -cli: +cli: rdf cd cli && mvn -B package @echo @echo "Add the ldh launcher to your \$$PATH:" diff --git a/rdf/pom.xml b/rdf/pom.xml new file mode 100644 index 000000000..595a5c1f2 --- /dev/null +++ b/rdf/pom.xml @@ -0,0 +1,191 @@ + + + 4.0.0 + + com.atomgraph + linkeddatahub-rdf + 6.0.1-SNAPSHOT + jar + + LinkedDataHub RDF + The RDF shapes LinkedDataHub documents are made of, and the vocabularies they are written in + https://github.com/AtomGraph/LinkedDataHub + + + + + namedgraph + martynas@atomgraph.com + + Developer + Founder + + AtomGraph + https://atomgraph.com + + + + + + Apache License 2.0 + http://www.apache.org/licenses/LICENSE-2.0.html + repo + + + + + https://github.com/AtomGraph/LinkedDataHub + scm:git:git://github.com/AtomGraph/LinkedDataHub.git + scm:git:git@github.com:AtomGraph/LinkedDataHub.git + + + + + central-portal-snapshots + https://central.sonatype.com/repository/maven-snapshots/ + + + + + + UTF-8 + 21 + + + + + org.apache.jena + jena-arq + 6.1.0 + + + + org.apache.httpcomponents + httpcore + + + org.apache.httpcomponents + httpclient-cache + + + org.apache.httpcomponents + httpclient + + + org.apache.httpcomponents + httpclient-osgi + + + org.apache.httpcomponents + httpcore-osgi + + + + + org.junit.jupiter + junit-jupiter + 5.12.2 + test + + + + + + + org.apache.maven.plugins + maven-compiler-plugin + 3.14.1 + + 21 + + + + org.apache.maven.plugins + maven-surefire-plugin + 3.5.2 + + + + org.codehaus.mojo + versions-maven-plugin + 2.21.0 + + + org.sonatype.central + central-publishing-maven-plugin + 0.11.0 + true + + central-portal-snapshots + true + + + + + + + + + release + + + + org.apache.maven.plugins + maven-source-plugin + 3.4.0 + + + attach-sources + + jar-no-fork + + + + + + org.apache.maven.plugins + maven-javadoc-plugin + 3.12.0 + + + attach-javadocs + + jar + + + + + + org.apache.maven.plugins + maven-gpg-plugin + 3.2.8 + + + sign-artifacts + verify + + sign + + + + + --pinentry-mode + loopback + + + + + + + + + + + diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Acl.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Acl.java new file mode 100644 index 000000000..e95f18677 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Acl.java @@ -0,0 +1,120 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.ACL; +import com.atomgraph.linkeddatahub.rdf.vocabulary.DH; +import java.net.URI; +import java.util.List; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Property; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.sparql.vocabulary.FOAF; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; +import org.apache.jena.vocabulary.RDFS; + +/** + * Access control: authorizations and the groups they are granted to. + *

+ * Both live in their own document, so each model carries the dh:Item wrapper + * around the resource it is about - the document is not the resource. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Acl +{ + + private Acl() { } + + /** + * Builds an authorization document. Agents, agent classes and agent groups say who; the + * accessTo and accessToClass lists say what; the modes say how. + * + * @param doc document URI + * @param uri authorization URI, relative or absolute (optional, blank node when absent) + * @param label authorization label, also the document title + * @param comment authorization comment (optional) + * @param agents WebIDs the authorization is granted to + * @param agentClasses agent classes the authorization is granted to + * @param agentGroups agent groups the authorization is granted to + * @param to URIs of the documents access is granted on + * @param toAllIn URIs of the classes whose instances access is granted on + * @param modes access modes granted + * @return authorization model + */ + public static Model authorization(URI doc, String uri, String label, String comment, + List agents, List agentClasses, List agentGroups, + List to, List toAllIn, List modes) + { + Model model = ModelFactory.createDefaultModel(); + + Resource auth = Subjects.of(model, doc, uri). + addProperty(RDF.type, ACL.Authorization). + addProperty(RDFS.label, label); + if (comment != null) auth.addProperty(RDFS.comment, comment); + + model.createResource(doc.toString()). + addProperty(RDF.type, DH.Item). + addProperty(FOAF.primaryTopic, auth). + addProperty(DCTerms.title, label); + + addResourceValues(auth, ACL.agent, agents); + addResourceValues(auth, ACL.agentClass, agentClasses); + addResourceValues(auth, ACL.agentGroup, agentGroups); + addResourceValues(auth, ACL.accessTo, to); + addResourceValues(auth, ACL.accessToClass, toAllIn); + modes.forEach(mode -> auth.addProperty(ACL.mode, mode)); + + return model; + } + + /** + * Builds an agent group document. + * + * @param doc document URI + * @param uri group URI, relative or absolute (optional, blank node when absent) + * @param name group name, also the document title + * @param description group description (optional) + * @param members WebIDs of the group's members + * @return group model + */ + public static Model group(URI doc, String uri, String name, String description, List members) + { + Model model = ModelFactory.createDefaultModel(); + + Resource group = Subjects.of(model, doc, uri). + addProperty(RDF.type, FOAF.Group). + addProperty(FOAF.name, name); + if (description != null) group.addProperty(DCTerms.description, description); + members.forEach(member -> group.addProperty(FOAF.member, model.createResource(member.toString()))); + + model.createResource(doc.toString()). + addProperty(RDF.type, DH.Item). + addProperty(FOAF.primaryTopic, group). + addProperty(DCTerms.title, name); + + return model; + } + + private static void addResourceValues(Resource subject, Property property, List values) + { + values.forEach(value -> subject.addProperty(property, subject.getModel().createResource(value.toString()))); + } + +} diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Blocks.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Blocks.java new file mode 100644 index 000000000..9cf87a4a7 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Blocks.java @@ -0,0 +1,96 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.AC; +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Property; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * Content blocks, the ordered body of a document. + *

+ * A block is attached to its document by a container membership property + * (rdf:_1, rdf:_2, …) which fixes its position; see + * {@link SequenceNumbers} for reading the next free one off an existing document. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Blocks +{ + + private Blocks() { } + + /** + * Builds an object block: a block whose value is another resource, rendered by the mode given. + * + * @param target target document URI + * @param seq container membership property fixing the block's position + * @param uri block URI, relative or absolute (optional, blank node when absent) + * @param value URI of the resource the block shows + * @param title block title (optional) + * @param description block description (optional) + * @param mode layout mode URI (optional) + * @return block model + */ + public static Model object(URI target, Property seq, String uri, URI value, String title, String description, URI mode) + { + Model model = ModelFactory.createDefaultModel(); + + Resource block = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.Object). + addProperty(RDF.value, model.createResource(value.toString())); + model.createResource(target.toString()).addProperty(seq, block); + if (title != null) block.addProperty(DCTerms.title, title); + if (description != null) block.addProperty(DCTerms.description, description); + if (mode != null) block.addProperty(AC.mode, model.createResource(mode.toString())); + + return model; + } + + /** + * Builds an XHTML block: a block whose value is markup carried as an + * rdf:XMLLiteral. + * + * @param target target document URI + * @param seq container membership property fixing the block's position + * @param uri block URI, relative or absolute (optional, blank node when absent) + * @param value XHTML markup + * @param title block title (optional) + * @param description block description (optional) + * @return block model + */ + public static Model xhtml(URI target, Property seq, String uri, String value, String title, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource block = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.XHTML). + addProperty(RDF.value, model.createTypedLiteral(value, RDF.dtXMLLiteral)); + model.createResource(target.toString()).addProperty(seq, block); + if (title != null) block.addProperty(DCTerms.title, title); + if (description != null) block.addProperty(DCTerms.description, description); + + return model; + } + +} diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Digests.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Digests.java similarity index 97% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Digests.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Digests.java index dd1638251..e0d02435f 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Digests.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Digests.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.util; +package com.atomgraph.linkeddatahub.rdf; import java.io.IOException; import java.io.InputStream; diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Documents.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Documents.java new file mode 100644 index 000000000..5912db4f8 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Documents.java @@ -0,0 +1,105 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.AC; +import com.atomgraph.linkeddatahub.rdf.vocabulary.DH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SPIN; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.sparql.vocabulary.FOAF; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * The two document kinds of the hierarchy: containers and items. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Documents +{ + + private Documents() { } + + /** + * Builds the container document model with its first content block: the given block URI, + * a children view with an explicit mode, or the default children view. + * + * @param doc document URI + * @param title document title + * @param description document description (optional) + * @param block content block URI (optional) + * @param mode children view mode URI (optional, ignored when block is given) + * @param primaryTopic URI of the document's primary topic, relative or absolute (optional) + * @return document model + */ + public static Model container(URI doc, String title, String description, URI block, URI mode, String primaryTopic) + { + Model model = ModelFactory.createDefaultModel(); + + Resource container = model.createResource(doc.toString()). + addProperty(RDF.type, DH.Container). + addProperty(DCTerms.title, title); + + if (block != null) container.addProperty(RDF.li(1), model.createResource(block.toString())); + else if (mode != null) container.addProperty(RDF.li(1), model.createResource(). + addProperty(RDF.type, LDH.Object). + addProperty(RDF.value, model.createResource(). + addProperty(RDF.type, LDH.View). + addProperty(SPIN.query, LDH.SelectChildren). + addProperty(AC.mode, model.createResource(mode.toString())))); + else container.addProperty(RDF.li(1), model.createResource(). + addProperty(RDF.type, LDH.Object). + addProperty(RDF.value, LDH.ChildrenView)); + + if (description != null) container.addProperty(DCTerms.description, description); + // See item(): resolved against the document, and singular by the vocabulary. + if (primaryTopic != null) container.addProperty(FOAF.primaryTopic, model.createResource(doc.resolve(primaryTopic).toString())); + + return model; + } + + /** + * Builds the item document model. + * + * @param doc document URI + * @param title document title + * @param description document description (optional) + * @param primaryTopic URI of the document's primary topic, relative or absolute (optional) + * @return document model + */ + public static Model item(URI doc, String title, String description, String primaryTopic) + { + Model model = ModelFactory.createDefaultModel(); + + Resource item = model.createResource(doc.toString()). + addProperty(RDF.type, DH.Item). + addProperty(DCTerms.title, title); + if (description != null) item.addProperty(DCTerms.description, description); + // Resolved against the document, so the conventional fragment topic is "#this" and a + // document about something described elsewhere takes that resource's absolute URI. + // Singular because foaf:primaryTopic is an owl:FunctionalProperty: a second value would + // not mean a second topic, it would entail the two topics are the same resource. + if (primaryTopic != null) item.addProperty(FOAF.primaryTopic, model.createResource(doc.resolve(primaryTopic).toString())); + + return model; + } + +} diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Imports.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Imports.java new file mode 100644 index 000000000..5478b6cd9 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Imports.java @@ -0,0 +1,98 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SD; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SPIN; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * Import descriptions - a file plus the transformation that turns it into RDF. + *

+ * Creating one only describes the import; the platform runs it asynchronously once the + * description lands. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Imports +{ + + private Imports() { } + + /** + * Builds a CSV import description: a CSV file, the delimiter that parses it, and a + * CONSTRUCT query mapping each row to triples. + * + * @param target target document URI + * @param uri import URI, relative or absolute (optional, blank node when absent) + * @param title import title + * @param query URI of the query mapping rows to triples + * @param file URI of the CSV file + * @param delimiter CSV delimiter character + * @param description import description (optional) + * @return import model + */ + public static Model csv(URI target, String uri, String title, URI query, URI file, String delimiter, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource csvImport = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.CSVImport). + addProperty(DCTerms.title, title). + addProperty(SPIN.query, model.createResource(query.toString())). + addProperty(LDH.file, model.createResource(file.toString())). + addProperty(LDH.delimiter, delimiter); + if (description != null) csvImport.addProperty(DCTerms.description, description); + + return model; + } + + /** + * Builds an RDF import description: an RDF file, optionally transformed by a query and + * optionally loaded into a named graph. + * + * @param target target document URI + * @param uri import URI, relative or absolute (optional, blank node when absent) + * @param title import title + * @param file URI of the RDF file + * @param query URI of the query transforming the parsed graph (optional) + * @param graph name of the graph the triples load into (optional) + * @param description import description (optional) + * @return import model + */ + public static Model rdf(URI target, String uri, String title, URI file, URI query, URI graph, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource rdfImport = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.RDFImport). + addProperty(DCTerms.title, title). + addProperty(LDH.file, model.createResource(file.toString())); + if (graph != null) rdfImport.addProperty(SD.name, model.createResource(graph.toString())); + if (query != null) rdfImport.addProperty(SPIN.query, model.createResource(query.toString())); + if (description != null) rdfImport.addProperty(DCTerms.description, description); + + return model; + } + +} diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Ontologies.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Ontologies.java new file mode 100644 index 000000000..9f98faae9 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Ontologies.java @@ -0,0 +1,199 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.DH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SPIN; +import java.net.URI; +import java.util.List; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.sparql.vocabulary.FOAF; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.OWL; +import org.apache.jena.vocabulary.RDF; +import org.apache.jena.vocabulary.RDFS; + +/** + * The terms an application ontology is edited in: the ontology document itself, its classes, + * their restrictions, and the constructors and constraints attached to them. + *

+ * These carry rdfs:label/rdfs:comment rather than + * dct:title/dct:description - ontology terms are annotated the way + * RDFS annotates them, not the way documents are described. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Ontologies +{ + + private Ontologies() { } + + /** + * Builds an ontology document. + * + * @param doc document URI + * @param uri ontology URI, relative or absolute (optional, blank node when absent) + * @param label ontology label, also the document title + * @param comment ontology comment (optional) + * @return ontology model + */ + public static Model ontology(URI doc, String uri, String label, String comment) + { + Model model = ModelFactory.createDefaultModel(); + + Resource ontology = Subjects.of(model, doc, uri). + addProperty(RDF.type, OWL.Ontology). + addProperty(RDFS.label, label); + if (comment != null) ontology.addProperty(RDFS.comment, comment); + + model.createResource(doc.toString()). + addProperty(RDF.type, DH.Item). + addProperty(FOAF.primaryTopic, ontology). + addProperty(DCTerms.title, label); + + return model; + } + + /** + * Builds a class, with its constructor, constraint and superclasses. + * + * @param target target ontology document URI + * @param uri class URI, relative or absolute (optional, blank node when absent) + * @param label class label + * @param comment class comment (optional) + * @param constructor URI of the query constructing instances of the class (optional) + * @param constraint URI of the class's constraint (optional) + * @param superClasses URIs of the class's superclasses + * @return class model + */ + public static Model owlClass(URI target, String uri, String label, String comment, URI constructor, URI constraint, List superClasses) + { + Model model = ModelFactory.createDefaultModel(); + + Resource cls = Subjects.of(model, target, uri). + addProperty(RDF.type, OWL.Class). + addProperty(RDFS.label, label); + if (comment != null) cls.addProperty(RDFS.comment, comment); + if (constructor != null) cls.addProperty(SPIN.constructor, model.createResource(constructor.toString())); + if (constraint != null) cls.addProperty(SPIN.constraint, model.createResource(constraint.toString())); + superClasses.forEach(superClass -> cls.addProperty(RDFS.subClassOf, model.createResource(superClass.toString()))); + + return model; + } + + /** + * Builds a property restriction. + * + * @param target target ontology document URI + * @param uri restriction URI, relative or absolute (optional, blank node when absent) + * @param label restriction label + * @param comment restriction comment (optional) + * @param onProperty URI of the restricted property (optional) + * @param allValuesFrom URI of the class the property's values must come from (optional) + * @param hasValue URI of the value the property must have (optional) + * @return restriction model + */ + public static Model restriction(URI target, String uri, String label, String comment, URI onProperty, URI allValuesFrom, URI hasValue) + { + Model model = ModelFactory.createDefaultModel(); + + Resource restriction = Subjects.of(model, target, uri). + addProperty(RDF.type, OWL.Restriction). + addProperty(RDFS.label, label); + if (comment != null) restriction.addProperty(RDFS.comment, comment); + if (onProperty != null) restriction.addProperty(OWL.onProperty, model.createResource(onProperty.toString())); + if (allValuesFrom != null) restriction.addProperty(OWL.allValuesFrom, model.createResource(allValuesFrom.toString())); + if (hasValue != null) restriction.addProperty(OWL.hasValue, model.createResource(hasValue.toString())); + + return model; + } + + /** + * Builds a SPIN constructor query - the query that builds a new instance of a class. + *

+ * The same shape as {@link Queries#query}, annotated for an ontology rather than described + * as a document. + * + * @param target target ontology document URI + * @param uri query URI, relative or absolute (optional, blank node when absent) + * @param queryType query form, e.g. {@link SP#Construct} + * @param label query label + * @param queryText the SPARQL query string + * @param service URI of the SPARQL service the query runs against (optional) + * @param comment query comment (optional) + * @return constructor model + */ + public static Model constructor(URI target, String uri, Resource queryType, String label, String queryText, URI service, String comment) + { + Model model = ModelFactory.createDefaultModel(); + + Resource query = Subjects.of(model, target, uri). + addProperty(RDF.type, queryType). + addProperty(RDFS.label, label). + addProperty(SP.text, queryText); + if (comment != null) query.addProperty(RDFS.comment, comment); + if (service != null) query.addProperty(LDH.service, model.createResource(service.toString())); + + return model; + } + + /** + * Builds a constraint requiring a property to have a value. + * + * @param target target ontology document URI + * @param uri constraint URI, relative or absolute (optional, blank node when absent) + * @param label constraint label + * @param property URI of the property that must have a value + * @param comment constraint comment (optional) + * @return constraint model + */ + public static Model propertyConstraint(URI target, String uri, String label, URI property, String comment) + { + Model model = ModelFactory.createDefaultModel(); + + Resource constraint = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.MissingPropertyValue). + addProperty(RDFS.label, label). + addProperty(SP.arg1, model.createResource(property.toString())); + if (comment != null) constraint.addProperty(RDFS.comment, comment); + + return model; + } + + /** + * Builds the annotation naming an imported ontology's source, written into the graph the + * import landed in. + * + * @param graph URI of the graph the ontology was imported into + * @param source URI the ontology was imported from + * @return annotation model + */ + public static Model annotation(URI graph, URI source) + { + Model model = ModelFactory.createDefaultModel(); + + model.createResource(graph.toString()). + addProperty(FOAF.primaryTopic, model.createResource(source.toString())); + + return model; + } + +} diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Queries.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Queries.java new file mode 100644 index 000000000..60da2763d --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Queries.java @@ -0,0 +1,65 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * Stored SPARQL queries. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Queries +{ + + private Queries() { } + + /** + * Builds a stored query. The query form is the caller's to state - {@link SP#Select} and + * {@link SP#Construct} are stored the same way, and only the type says which one this is. + * + * @param target target document URI + * @param uri query URI, relative or absolute (optional, blank node when absent) + * @param queryType query form, e.g. {@link SP#Select} + * @param title query title + * @param queryText the SPARQL query string + * @param service URI of the SPARQL service the query runs against (optional) + * @param description query description (optional) + * @return query model + */ + public static Model query(URI target, String uri, Resource queryType, String title, String queryText, URI service, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource query = Subjects.of(model, target, uri). + addProperty(RDF.type, queryType). + addProperty(DCTerms.title, title). + addProperty(SP.text, queryText); + if (service != null) query.addProperty(LDH.service, model.createResource(service.toString())); + if (description != null) query.addProperty(DCTerms.description, description); + + return model; + } + +} diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbers.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbers.java similarity index 98% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbers.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbers.java index 22fcc3738..8da89aed4 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbers.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbers.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.util; +package com.atomgraph.linkeddatahub.rdf; import org.apache.jena.rdf.model.Model; import org.apache.jena.rdf.model.Property; diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Services.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Services.java new file mode 100644 index 000000000..f7f496a2d --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Services.java @@ -0,0 +1,69 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.A; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SD; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * SPARQL service descriptions - the endpoints a dataspace can query and write. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Services +{ + + private Services() { } + + /** + * Builds a generic SPARQL service description. + * + * @param target target document URI + * @param uri service URI, relative or absolute (optional, blank node when absent) + * @param title service title + * @param endpoint SPARQL endpoint URI + * @param graphStore Graph Store Protocol endpoint URI (optional) + * @param authUser HTTP Basic auth user name (optional) + * @param authPwd HTTP Basic auth password (optional) + * @param description service description (optional) + * @return service model + */ + public static Model service(URI target, String uri, String title, URI endpoint, URI graphStore, String authUser, String authPwd, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource service = Subjects.of(model, target, uri). + addProperty(RDF.type, SD.Service). + addProperty(DCTerms.title, title). + addProperty(SD.endpoint, model.createResource(endpoint.toString())). + addProperty(SD.supportedLanguage, SD.SPARQL11Query). + addProperty(SD.supportedLanguage, SD.SPARQL11Update); + if (graphStore != null) service.addProperty(A.graphStore, model.createResource(graphStore.toString())); + if (authUser != null) service.addProperty(A.authUser, authUser); + if (authPwd != null) service.addProperty(A.authPwd, authPwd); + if (description != null) service.addProperty(DCTerms.description, description); + + return model; + } + +} diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Slugs.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Slugs.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Slugs.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Slugs.java index 6d77b04ed..3f5b49d56 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/util/Slugs.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Slugs.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.util; +package com.atomgraph.linkeddatahub.rdf; import java.util.Locale; import java.util.UUID; diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Subjects.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Subjects.java new file mode 100644 index 000000000..fd62b6686 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Subjects.java @@ -0,0 +1,51 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.Resource; + +/** + * The subject a description appended to a document is about. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Subjects +{ + + private Subjects() { } + + /** + * Returns the subject resource for an appended description: the given URI resolved against + * the target document URI, or a fresh blank node when no URI is given. + *

+ * A blank node is the right default because most appended descriptions - a content block, a + * query, a chart - are only ever referred to from within the document that carries them, so + * minting a URI for them would add an identifier nothing dereferences. + * + * @param model model to create the resource in + * @param target target document URI + * @param uri subject URI, absolute or relative to the target, or null for a blank node + * @return subject resource + */ + public static Resource of(Model model, URI target, String uri) + { + return uri != null ? model.createResource(target.resolve(uri).toString()) : model.createResource(); + } + +} diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/URIs.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/URIs.java new file mode 100644 index 000000000..915828cc1 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/URIs.java @@ -0,0 +1,87 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import java.net.URI; +import java.nio.charset.StandardCharsets; + +/** + * Where LinkedDataHub puts things: how a child document's URI is derived from its parent and a + * slug, and how an end-user application's base URI relates to its admin application's. + *

+ * Conventions, not manipulation - every method here encodes a rule the platform also applies, so + * a client that follows them addresses documents the platform will agree exist. + * + * @author Martynas Jusevičius {@literal } + */ +public final class URIs +{ + + private URIs() { } + + /** + * Builds the URI of a child document from the parent container URI and a path segment slug. + *

+ * The trailing slash is what makes it a container-addressable document rather than a fragment + * of its parent, so it is added here rather than left to the caller. + * + * @param parent parent container URI (with trailing slash) + * @param slug path segment + * @return child document URI (with trailing slash) + */ + public static URI childURI(URI parent, String slug) + { + return URI.create(parent.toString() + encodeSlug(slug) + "/"); + } + + /** + * Percent-encodes a string as a URI path segment. All characters except RFC 3986 + * unreserved ones are encoded, including / - a slug names one segment, so a + * slash in it is data rather than a further step in the path. + * + * @param slug path segment + * @return encoded path segment + */ + public static String encodeSlug(String slug) + { + StringBuilder sb = new StringBuilder(); + + for (byte b : slug.getBytes(StandardCharsets.UTF_8)) + { + char c = (char)(b & 0xFF); + if ((c >= 'A' && c <= 'Z') || (c >= 'a' && c <= 'z') || (c >= '0' && c <= '9') || + c == '-' || c == '.' || c == '_' || c == '~') sb.append(c); + else sb.append('%').append(String.format("%02X", b & 0xFF)); + } + + return sb.toString(); + } + + /** + * Converts an end-user application base URI to the base URI of its admin application + * by prefixing the host with the admin. subdomain, which is how nginx's + * wildcard routing tells the two apart. + * + * @param base end-user base URI + * @return admin base URI + */ + public static URI adminBase(URI base) + { + return URI.create(base.toString().replaceFirst("://", "://admin.")); + } + +} diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/sparql/Updates.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Updates.java similarity index 99% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/sparql/Updates.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Updates.java index 160cb6bb6..02cbe6251 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/sparql/Updates.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Updates.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.sparql; +package com.atomgraph.linkeddatahub.rdf; import java.net.URI; import org.apache.jena.query.ParameterizedSparqlString; diff --git a/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Views.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Views.java new file mode 100644 index 000000000..74fa2b5c1 --- /dev/null +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/Views.java @@ -0,0 +1,94 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.AC; +import com.atomgraph.linkeddatahub.rdf.vocabulary.LDH; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SPIN; +import java.net.URI; +import org.apache.jena.rdf.model.Model; +import org.apache.jena.rdf.model.ModelFactory; +import org.apache.jena.rdf.model.Resource; +import org.apache.jena.vocabulary.DCTerms; +import org.apache.jena.vocabulary.RDF; + +/** + * Views: a query plus how its results are shown. + * + * @author Martynas Jusevičius {@literal } + */ +public final class Views +{ + + private Views() { } + + /** + * Builds a view of a query's results. + * + * @param target target document URI + * @param uri view URI, relative or absolute (optional, blank node when absent) + * @param query URI of the query the view runs + * @param title view title (optional) + * @param description view description (optional) + * @param mode layout mode URI (optional) + * @return view model + */ + public static Model view(URI target, String uri, URI query, String title, String description, URI mode) + { + Model model = ModelFactory.createDefaultModel(); + + Resource view = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.View). + addProperty(SPIN.query, model.createResource(query.toString())); + if (title != null) view.addProperty(DCTerms.title, title); + if (description != null) view.addProperty(DCTerms.description, description); + if (mode != null) view.addProperty(AC.mode, model.createResource(mode.toString())); + + return model; + } + + /** + * Builds a chart of a SELECT query's result set. The category and series variable names + * pick the result-set columns the chart's axes read. + * + * @param target target document URI + * @param uri chart URI, relative or absolute (optional, blank node when absent) + * @param title chart title + * @param query URI of the query the chart runs + * @param chartType chart type URI + * @param categoryVarName result-set variable name providing the categories + * @param seriesVarName result-set variable name providing the series + * @param description chart description (optional) + * @return chart model + */ + public static Model resultSetChart(URI target, String uri, String title, URI query, URI chartType, String categoryVarName, String seriesVarName, String description) + { + Model model = ModelFactory.createDefaultModel(); + + Resource chart = Subjects.of(model, target, uri). + addProperty(RDF.type, LDH.ResultSetChart). + addProperty(DCTerms.title, title). + addProperty(SPIN.query, model.createResource(query.toString())). + addProperty(LDH.chartType, model.createResource(chartType.toString())). + addProperty(LDH.categoryVarName, categoryVarName). + addProperty(LDH.seriesVarName, seriesVarName); + if (description != null) chart.addProperty(DCTerms.description, description); + + return model; + } + +} diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/A.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/A.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/A.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/A.java index 73dd5abd3..c37362ccb 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/A.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/A.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.ResourceFactory; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/AC.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/AC.java similarity index 95% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/AC.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/AC.java index 9de56512e..d04870afe 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/AC.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/AC.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.ResourceFactory; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/ACL.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/ACL.java similarity index 97% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/ACL.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/ACL.java index e7f3ff709..b193ce1ba 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/ACL.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/ACL.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/DH.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/DH.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/DH.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/DH.java index dcc80f803..92517f2b5 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/DH.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/DH.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Resource; import org.apache.jena.rdf.model.ResourceFactory; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/LDH.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/LDH.java similarity index 98% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/LDH.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/LDH.java index 397f57f70..c4f2e0637 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/LDH.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/LDH.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/NFO.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/NFO.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/NFO.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/NFO.java index de6a53c65..b388abc2f 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/NFO.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/NFO.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SD.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SD.java similarity index 97% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SD.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SD.java index 4bec9d26f..47eaf2184 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SD.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SD.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SP.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SP.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SP.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SP.java index 6634441f9..8217816c8 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SP.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SP.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.Resource; diff --git a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SPIN.java b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SPIN.java similarity index 96% rename from cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SPIN.java rename to rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SPIN.java index a4cbec33d..9c183d737 100644 --- a/cli/src/main/java/com/atomgraph/linkeddatahub/cli/vocab/SPIN.java +++ b/rdf/src/main/java/com/atomgraph/linkeddatahub/rdf/vocabulary/SPIN.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.vocab; +package com.atomgraph.linkeddatahub.rdf.vocabulary; import org.apache.jena.rdf.model.Property; import org.apache.jena.rdf.model.ResourceFactory; diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/ModelBuildersTest.java b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/ModelBuildersTest.java similarity index 80% rename from cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/ModelBuildersTest.java rename to rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/ModelBuildersTest.java index 770dd454c..88686d467 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/command/ModelBuildersTest.java +++ b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/ModelBuildersTest.java @@ -14,22 +14,10 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.command; - -import com.atomgraph.linkeddatahub.cli.command.admin.acl.CreateAuthorization; -import com.atomgraph.linkeddatahub.cli.command.admin.acl.CreateGroup; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.AddClass; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.AddConstructor; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.AddPropertyConstraint; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.AddRestriction; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.CreateOntology; -import com.atomgraph.linkeddatahub.cli.command.admin.ontologies.ImportOntology; -import com.atomgraph.linkeddatahub.cli.command.content.AddObjectBlock; -import com.atomgraph.linkeddatahub.cli.command.content.AddXHTMLBlock; -import com.atomgraph.linkeddatahub.cli.command.imports.AddCSVImport; -import com.atomgraph.linkeddatahub.cli.command.imports.AddRDFImport; -import com.atomgraph.linkeddatahub.cli.vocab.ACL; -import com.atomgraph.linkeddatahub.cli.vocab.SP; +package com.atomgraph.linkeddatahub.rdf; + +import com.atomgraph.linkeddatahub.rdf.vocabulary.ACL; +import com.atomgraph.linkeddatahub.rdf.vocabulary.SP; import java.io.StringReader; import java.net.URI; import java.util.List; @@ -88,7 +76,7 @@ public void createItem() dct:title "My item" ; dct:description "Desc" . """), - CreateItem.buildModel(doc, "My item", "Desc", null)); + Documents.item(doc, "My item", "Desc", null)); } @Test @@ -103,7 +91,7 @@ public void createItemWithFragmentPrimaryTopic() dct:title "My item" ; foaf:primaryTopic . """), - CreateItem.buildModel(doc, "My item", null, "#this")); + Documents.item(doc, "My item", null, "#this")); } @Test @@ -118,7 +106,7 @@ public void createItemWithAbsolutePrimaryTopic() dct:title "About Bob" ; foaf:primaryTopic . """), - CreateItem.buildModel(doc, "About Bob", null, "https://example.org/bob#me")); + Documents.item(doc, "About Bob", null, "https://example.org/bob#me")); } @Test @@ -129,7 +117,7 @@ public void createContainerDefaultChildrenView() dct:title "Some" ; rdf:_1 [ a ldh:Object ; rdf:value ldh:ChildrenView ] . """), - CreateContainer.buildModel(TARGET, "Some", null, null, null, null)); + Documents.container(TARGET, "Some", null, null, null, null)); } @Test @@ -140,7 +128,7 @@ public void createContainerWithMode() dct:title "Some" ; rdf:_1 [ a ldh:Object ; rdf:value [ a ldh:View ; spin:query ldh:SelectChildren ; ac:mode ] ] . """), - CreateContainer.buildModel(TARGET, "Some", null, null, URI.create("https://w3id.org/atomgraph/client#GridMode"), null)); + Documents.container(TARGET, "Some", null, null, URI.create("https://w3id.org/atomgraph/client#GridMode"), null)); } @Test @@ -151,7 +139,7 @@ public void createContainerWithBlock() dct:title "Some" ; rdf:_1 . """), - CreateContainer.buildModel(TARGET, "Some", null, URI.create("https://localhost:4443/some/#block"), null, null)); + Documents.container(TARGET, "Some", null, URI.create("https://localhost:4443/some/#block"), null, null)); } @Test @@ -165,7 +153,7 @@ public void createContainerWithPrimaryTopic() rdf:_1 [ a ldh:Object ; rdf:value ldh:ChildrenView ] ; foaf:primaryTopic <#this> . """), - CreateContainer.buildModel(TARGET, "Some", null, null, null, "#this")); + Documents.container(TARGET, "Some", null, null, null, "#this")); } @Test @@ -177,7 +165,7 @@ public void addViewWithURIAndMode() dct:title "View" ; ac:mode . """), - AddView.buildModel(TARGET, "#view", URI.create("https://localhost:4443/queries/q/#this"), "View", null, + Views.view(TARGET, "#view", URI.create("https://localhost:4443/queries/q/#this"), "View", null, URI.create("https://w3id.org/atomgraph/client#GridMode"))); } @@ -190,7 +178,7 @@ public void addConstructQuery() sp:text \"""CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }\""" ; ldh:service . """), - AddConstruct.buildModel(TARGET, null, SP.Construct, "Query", "CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }", + Queries.query(TARGET, null, SP.Construct, "Query", "CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }", URI.create("https://localhost:4443/services/s/#this"), null)); } @@ -208,7 +196,7 @@ public void createOntology() foaf:primaryTopic _:ontology ; dct:title "My ontology" . """), - CreateOntology.buildModel(doc, null, "My ontology", "Comment")); + Ontologies.ontology(doc, null, "My ontology", "Comment")); } @Test @@ -225,7 +213,7 @@ public void createGroup() foaf:primaryTopic _:group ; dct:title "Editors" . """), - CreateGroup.buildModel(doc, null, "Editors", null, + Acl.group(doc, null, "Editors", null, List.of(URI.create("https://localhost:4443/acl/agents/a/#this"), URI.create("https://localhost:4443/acl/agents/b/#this")))); } @@ -245,7 +233,7 @@ public void createAuthorization() foaf:primaryTopic _:auth ; dct:title "Auth" . """), - CreateAuthorization.buildModel(doc, null, "Auth", null, + Acl.authorization(doc, null, "Auth", null, List.of(URI.create("https://localhost:4443/acl/agents/a/#this")), List.of(), List.of(), List.of(URI.create("https://localhost:4443/some/")), List.of(), List.of(ACL.Read, ACL.Write))); @@ -261,7 +249,7 @@ public void addCSVImport() ldh:file ; ldh:delimiter "," . """), - AddCSVImport.buildModel(TARGET, null, "Cities", URI.create("https://localhost:4443/some/#query"), + Imports.csv(TARGET, null, "Cities", URI.create("https://localhost:4443/some/#query"), URI.create("https://localhost:4443/uploads/abc"), ",", null)); } @@ -274,7 +262,7 @@ public void addRDFImportWithGraph() ldh:file ; sd:name . """), - AddRDFImport.buildModel(TARGET, "#import", "Data", URI.create("https://localhost:4443/uploads/abc"), + Imports.rdf(TARGET, "#import", "Data", URI.create("https://localhost:4443/uploads/abc"), null, URI.create("https://localhost:4443/graphs/g/"), null)); } @@ -286,7 +274,7 @@ public void addXHTMLBlock() _:block a ldh:XHTML ; rdf:value "

Hello

"^^rdf:XMLLiteral . """), - AddXHTMLBlock.buildModel(TARGET, RDF.li(4), null, "

Hello

", null, null)); + Blocks.xhtml(TARGET, RDF.li(4), null, "

Hello

", null, null)); } @Test @@ -301,7 +289,7 @@ public void addGenericServiceWithGraphStoreAndAuth() a:authUser "user" ; a:authPwd "pwd" . """), - AddGenericService.buildModel(TARGET, "#service", "Remote", URI.create("https://remote.example/sparql"), + Services.service(TARGET, "#service", "Remote", URI.create("https://remote.example/sparql"), URI.create("https://remote.example/service"), "user", "pwd", null)); } @@ -314,7 +302,7 @@ public void addGenericServiceMinimal() sd:endpoint ; sd:supportedLanguage sd:SPARQL11Query, sd:SPARQL11Update . """), - AddGenericService.buildModel(TARGET, "#service", "Remote", URI.create("https://remote.example/sparql"), + Services.service(TARGET, "#service", "Remote", URI.create("https://remote.example/sparql"), null, null, null, null)); } @@ -330,7 +318,7 @@ public void addResultSetChart() ldh:categoryVarName "category" ; ldh:seriesVarName "series" . """), - AddResultSetChart.buildModel(TARGET, "#chart", "Chart", URI.create("https://localhost:4443/queries/select/#this"), + Views.resultSetChart(TARGET, "#chart", "Chart", URI.create("https://localhost:4443/queries/select/#this"), URI.create("https://w3id.org/atomgraph/client#BarChart"), "category", "series", "Desc")); } @@ -345,7 +333,7 @@ public void addClassWithSuperClasses() spin:constraint ; rdfs:subClassOf , . """), - AddClass.buildModel(TARGET, "#Concept", "Concept", "A concept", + Ontologies.owlClass(TARGET, "#Concept", "Concept", "A concept", URI.create("https://localhost:4443/queries/construct/#this"), URI.create("https://localhost:4443/constraints/#this"), List.of(URI.create("https://localhost:4443/ns#Thing"), URI.create("https://localhost:4443/ns#Other")))); @@ -358,7 +346,7 @@ public void addClassMinimal() <#Concept> a owl:Class ; rdfs:label "Concept" . """), - AddClass.buildModel(TARGET, "#Concept", "Concept", null, null, null, List.of())); + Ontologies.owlClass(TARGET, "#Concept", "Concept", null, null, null, List.of())); } @Test @@ -371,7 +359,7 @@ public void addConstructor() sp:text "CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }" ; ldh:service . """), - AddConstructor.buildModel(TARGET, "#constructor", SP.Construct, "Constructor", + Ontologies.constructor(TARGET, "#constructor", SP.Construct, "Constructor", "CONSTRUCT { ?s ?p ?o } WHERE { ?s ?p ?o }", URI.create("https://localhost:4443/services/remote/#this"), "Builds a Concept")); } @@ -384,7 +372,7 @@ public void addPropertyConstraint() rdfs:label "Title required" ; sp:arg1 dct:title . """), - AddPropertyConstraint.buildModel(TARGET, "#constraint", "Title required", + Ontologies.propertyConstraint(TARGET, "#constraint", "Title required", URI.create("http://purl.org/dc/terms/title"), null)); } @@ -399,7 +387,7 @@ public void addRestriction() owl:allValuesFrom rdfs:Literal ; owl:hasValue . """), - AddRestriction.buildModel(TARGET, "#restriction", "Has title", "Every instance carries a title", + Ontologies.restriction(TARGET, "#restriction", "Has title", "Every instance carries a title", URI.create("http://purl.org/dc/terms/title"), URI.create("http://www.w3.org/2000/01/rdf-schema#Literal"), URI.create("https://localhost:4443/values/default/"))); @@ -416,7 +404,7 @@ public void addObjectBlock() dct:description "Desc" ; ac:mode . """), - AddObjectBlock.buildModel(TARGET, RDF.li(2), null, URI.create("https://localhost:4443/other/"), + Blocks.object(TARGET, RDF.li(2), null, URI.create("https://localhost:4443/other/"), "Block", "Desc", URI.create("https://w3id.org/atomgraph/client#ReadMode"))); } @@ -430,7 +418,7 @@ public void importOntologyAnnotation() assertIsomorphic(parse(""" <> foaf:primaryTopic . """), - ImportOntology.buildAnnotationModel(TARGET, URI.create("http://www.w3.org/2004/02/skos/core#"))); + Ontologies.annotation(TARGET, URI.create("http://www.w3.org/2004/02/skos/core#"))); } } diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbersTest.java b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbersTest.java similarity index 98% rename from cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbersTest.java rename to rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbersTest.java index 275030c71..2f50d5994 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/util/SequenceNumbersTest.java +++ b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/SequenceNumbersTest.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.util; +package com.atomgraph.linkeddatahub.rdf; import org.apache.jena.rdf.model.Model; import org.apache.jena.rdf.model.ModelFactory; diff --git a/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/URIsTest.java b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/URIsTest.java new file mode 100644 index 000000000..bd908e58f --- /dev/null +++ b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/URIsTest.java @@ -0,0 +1,54 @@ +/* + * Copyright 2026 Martynas Jusevičius . + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package com.atomgraph.linkeddatahub.rdf; + +import java.net.URI; +import org.junit.jupiter.api.Test; +import static org.junit.jupiter.api.Assertions.assertEquals; + +/** + * Tests for {@link URIs}. + */ +public class URIsTest +{ + + @Test + public void adminBasePrefixesHostWithAdminSubdomain() + { + assertEquals(URI.create("https://admin.localhost:4443/"), URIs.adminBase(URI.create("https://localhost:4443/"))); + } + + @Test + public void encodeSlugKeepsUnreservedCharacters() + { + assertEquals("abc-._~123", URIs.encodeSlug("abc-._~123")); + } + + @Test + public void encodeSlugEncodesReservedAndNonASCII() + { + assertEquals("a%20b%2F%C4%87", URIs.encodeSlug("a b/ć")); + } + + @Test + public void childURIAppendsEncodedSlugAndSlash() + { + assertEquals(URI.create("https://localhost:4443/some/my%20item/"), + URIs.childURI(URI.create("https://localhost:4443/some/"), "my item")); + } + +} diff --git a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/sparql/UpdatesTest.java b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/UpdatesTest.java similarity index 98% rename from cli/src/test/java/com/atomgraph/linkeddatahub/cli/sparql/UpdatesTest.java rename to rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/UpdatesTest.java index 79981c84b..20451080c 100644 --- a/cli/src/test/java/com/atomgraph/linkeddatahub/cli/sparql/UpdatesTest.java +++ b/rdf/src/test/java/com/atomgraph/linkeddatahub/rdf/UpdatesTest.java @@ -14,7 +14,7 @@ * limitations under the License. */ -package com.atomgraph.linkeddatahub.cli.sparql; +package com.atomgraph.linkeddatahub.rdf; import java.net.URI; import org.apache.jena.query.Syntax; diff --git a/release.sh b/release.sh index addd5b7cf..b9f823a3b 100755 --- a/release.sh +++ b/release.sh @@ -127,20 +127,24 @@ fi print_status "GPG check passed" -# Set cli/pom.xml to the given version and commit it. The CLI is not a module of the platform -# reactor, so maven-release-plugin does not rewrite it - it is kept in step here instead, once for -# the release version and once for the next development version. +# Set rdf/pom.xml and cli/pom.xml to the given version and commit them. Neither is a module of the +# platform reactor, so maven-release-plugin does not rewrite them - they are kept in step here +# instead, once for the release version and once for the next development version. The CLI resolves +# the library by ${project.version}, so the two must move together or the CLI build breaks. sync_cli_version() { local version="$1" + local project - (cd cli && mvn -B -q versions:set -DnewVersion="$version" -DgenerateBackupPoms=false) + for project in rdf cli; do + (cd "$project" && mvn -B -q versions:set -DnewVersion="$version" -DgenerateBackupPoms=false) + done - if git diff --quiet -- cli/pom.xml; then - print_status "cli/pom.xml already at $version" + if git diff --quiet -- rdf/pom.xml cli/pom.xml; then + print_status "rdf/pom.xml and cli/pom.xml already at $version" else - git add cli/pom.xml - git commit -m "Set the CLI version to $version" - print_status "cli/pom.xml set to $version" + git add rdf/pom.xml cli/pom.xml + git commit -m "Set the RDF library and CLI versions to $version" + print_status "rdf/pom.xml and cli/pom.xml set to $version" fi } @@ -228,6 +232,23 @@ print_status "Performing Maven release (deploying to Sonatype)..." mvn release:perform -DlocalCheckout=true PUBLISHED=true +# Publish linkeddatahub-rdf at the same version. It is not a module of the reactor, so +# release:perform does not carry it - but the CLI resolves it by ${project.version}, and Web-Algebra's +# ldh-* operations resolve it from their own repository, so a platform release without it is a release +# whose clients cannot build. +# +# Deployed from the tag rather than the working tree: by this point sync_cli_version has already moved +# rdf/pom.xml on to the next development version, while the tag holds the release one. git archive +# extracts the subtree alone, so nothing here depends on the working tree's state. +# The staging directory is removed inline rather than by a trap: the script's only EXIT trap is +# cleanup_on_failure, and registering a second one would replace it. +print_status "Deploying linkeddatahub-rdf $RELEASE_VERSION..." +RDF_STAGING=$(mktemp -d) +git archive "$RELEASE_TAG" rdf | tar -x -C "$RDF_STAGING" +(cd "$RDF_STAGING/rdf" && mvn -B -Prelease clean deploy) +rm -rf "$RDF_STAGING" +print_status "linkeddatahub-rdf $RELEASE_VERSION deployed" + # Switch to master and merge only the release commit print_status "Merging release commit to master branch..." git checkout master From a936a278c81f7b3c275d76b18f6d3e028d92db68 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Martynas=20Jusevi=C4=8Dius?= Date: Wed, 30 Sep 2026 00:21:30 +0200 Subject: [PATCH 16/16] The next release is 6.0.1, and the changelog catches up with what has landed. Two of the nine entries were there; the rest are the RDF library's own publication, the write validation, the package ontology and registry fixes, the 429 retry, the chart token lookup, the CLI's stylesheet media type and the documentation links. The two that were there are cut to one line each, as the rest are. Omitted: the Taxonomy Editor renaming, the load and package test suites, the chart specs and the version bumps. Co-Authored-By: Claude Opus 5 (1M context) --- CHANGELOG.md | 17 ++++++++++++++--- 1 file changed, 14 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 52bf90167..f55545e80 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,7 +1,18 @@ -## [Unreleased] +## [6.0.1] - 2026-09-30 +### Added +- `linkeddatahub-rdf`, a separately published library of the vocabularies and the document shapes the HTTP API accepts, so the CLI and other clients build against one description of the request bodies + +### Changed +- Documentation links point at `docs.linkeddatahub.com` rather than the stale static mirror under `atomgraph.github.io` + ### Fixed -- A burst of renders could deadlock the platform against itself. Every server-side HTML render calls back into its own dataspace through the proxy for the page's labels (`ldh:send-request` to `/sparql` and `/ns`, and the stylesheet's `document()` loads), and each callback is answered by one of the same Tomcat's request threads while the render holds another. Once renders outnumbered the connector's threads - a scanner probing non-existent paths did it on linkeddatahub.com, every 403 and 404 page being a full render - every thread was a render waiting for a callback no thread was free to serve, and nothing completed until the client's read timeout, which is sized for a stalled backend. A request to this instance's own URL is now bounded on its own: `ClientUriRewriteFilter`, the one place that recognizes such a request and sends it to the proxy, gives it a connect and read timeout of `CLIENT_SELF_REQUEST_TIMEOUT` (default 5 s), after which the render gives its thread back. And the image's client pool is sized to the connector (`MAX_CONN_PER_ROUTE=200`, `MAX_TOTAL_CONN=400`, from 20 and 40): a self-call holds a pooled connection while it waits for another thread, so a pool smaller than the connector queued them behind each other and a burst drained one bounded wait at a time. The pool-wait (`CONNECTION_REQUEST_TIMEOUT`) also has a code default of 30 s now, matching the image's, where it used to be unbounded outside the image. `tests/load/render-burst-no-deadlock.sh` reproduces the wedge on a 16-thread stack and guards the fix. -- Proxied XHTML content left path-absolute `@href`/`@src` references unresolved, so authored media (`/uploads/{sha1}`) resolved against the proxying dataspace's origin and 404'd. The proxy-mode rewrite in `ldh:XHTMLContent` now resolves them against the content's base URI alongside the relative ones; only fragments stay untouched, because they address the rendering rather than the source +- A burst of server-side renders could deadlock the platform against itself, each render holding a request thread while it waited for a label callback that needed another; a request to this instance's own URL is now bounded by `CLIENT_SELF_REQUEST_TIMEOUT` (5 s) and the image's client pool is sized to the connector (`MAX_CONN_PER_ROUTE=200`, `MAX_TOTAL_CONN=400`) +- Every write is held to the constraints of the document as it will be written: a PUT that did not type the document was validated while the document was untyped, a POST validated only its body, and the document form's multipart PUT wrote the form's triples as the graph with no type, container, timestamps or `If-Match` +- A package ontology whose descriptor named its document rather than the ontology took its dataspace down, and bundled copies shadowed `packages.linkeddatahub.com`, so every page of the packages dataspace failed with an empty label +- Proxied XHTML content left path-absolute `@href`/`@src` references unresolved, so authored media (`/uploads/{sha1}`) resolved against the proxying dataspace's origin and 404'd +- A block retrying a `429` re-sent the first request of its chain instead of the one its step made, so a rate-limited chart drew the RDF/XML of its query's document as the results +- Chart colours came out empty in Chromium, which enumerates no custom properties: `ldh:css-token()` asks `getComputedStyle` for the property by name instead of reading the enumerated style map +- `ldh` uploads `.xsl` and `.xslt` as `text/xsl`; stored as `application/octet-stream`, a stylesheet answered the platform's request with `406` ## [6.0.0] - 2026-09-29 LinkedDataHub has a new interface. The app shell, content blocks, action bar, breadcrumbs, mode lists, type badges, property lists, tables, pager, modals and forms are drawn against a design system vendored into the platform — its tokens, components and typefaces ship with LDH, and `ldh.css` is the single app layer loaded over them, the one file a dataspace stylesheet has to reckon with. IXSL templates drive the dropdowns and modals, the `msi` font draws the icons and the RDFa editor edits `rdf:XMLLiteral`, so jQuery, `bootstrap.js`, WYMEditor and the sprite sheet are gone.