Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,52 @@
name: CI

on:
pull_request:
push:
branches: [main]
workflow_dispatch:

permissions:
contents: read

jobs:
test:
name: Build image and render a clip
runs-on: ubuntu-latest
steps:
- uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- name: Validate the SAM template
run: sam validate --lint
- name: Build the image
run: docker build -t spectrogram-renderer core
- name: Render a clip the way Lambda runs it
# Unprivileged, root-owned /tmp, and the CPU and memory of the 1024 MB tier.
run: |
docker run --rm --user 1000:1000 --tmpfs /tmp:uid=0,gid=0,mode=1777 \
--cpus 0.58 --memory 1024m --entrypoint python \
-v "$PWD/tests/smoke.py:/var/task/smoke.py" spectrogram-renderer smoke.py

deploy:
name: Deploy to AWS
needs: test
if: github.event_name != 'pull_request' && github.ref == 'refs/heads/main'
runs-on: ubuntu-latest
environment: production
concurrency: deploy
permissions:
id-token: write
contents: read
steps:
- uses: step-security/harden-runner@e14015d583714f6e62063499dc959a02595150a1 # v2.21.1
with:
egress-policy: audit
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: aws-actions/configure-aws-credentials@e1253824e5c10ff9df46874f81ed3ec929e19cfd # v6.3.0
with:
role-to-assume: ${{ vars.AWS_ROLE_ARN }}
aws-region: us-west-2
- run: sam build
- run: sam deploy --no-confirm-changeset --no-fail-on-empty-changeset
4 changes: 3 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -241,4 +241,6 @@ $RECYCLE.BIN/

*/build/*

# End of https://www.gitignore.io/api/osx,linux,python,windows,pycharm,visualstudiocode
# End of https://www.gitignore.io/api/osx,linux,python,windows,pycharm,visualstudiocode
# SAM build output
.aws-sam/
34 changes: 19 additions & 15 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,39 +1,43 @@
# audio_viz
# spectrogram-renderer

A Lambda function that renders one spectrogram PNG from one audio segment. Orcasite invokes it directly by name from [`Orcasite.Radio.AwsClient`](../lib/orcasite/radio/aws_client.ex), once per `AudioImage`, passing the S3 location of the segment and where to put the image. It has no other entry point.
A Lambda function that renders one spectrogram PNG from one hydrophone audio segment. [Orcasite](https://github.com/orcasound/orcasite) invokes it by name from [`Orcasite.Radio.AwsClient`](https://github.com/orcasound/orcasite/blob/main/server/lib/orcasite/radio/aws_client.ex), once per `AudioImage`, passing the S3 location of the segment and where to put the image. It has no other entry point.

- [`core/app.py`](core/app.py): the handler. Downloads the segment, decodes it with ffmpeg, renders with [`core/spectrogram_generator.py`](core/spectrogram_generator.py), uploads the PNG.
- [`core/Dockerfile`](core/Dockerfile): the image, built on the AWS Lambda Python base with a static ffmpeg.
- [`template.yaml`](template.yaml): the function's memory, timeout and IAM policy, as a SAM template. Stack name and region are in [`samconfig.toml`](samconfig.toml).
- [`template.yaml`](template.yaml): the function's memory, timeout and IAM policy, as a SAM template. Stack name (`audio-viz`, kept from when this lived in orcasite) and region are in [`samconfig.toml`](samconfig.toml).

## Cold starts

A new Lambda container starts with an empty `/tmp`, where numba, librosa and matplotlib keep their caches, so the first render in a container recompiles librosa's numba functions and rebuilds the font list. The Dockerfile runs [`warm.py`](core/warm.py) once at build time and keeps the caches it leaves in the image; [`app.py`](core/app.py) copies them into `/tmp` before importing those libraries. If you add a library that caches on first use, give it a directory under `/tmp` in the Dockerfile and add it to the same `mv`.

To check a deployed function, find `platform.report` records in its CloudWatch log (the template sets `LogFormat: JSON`). Cold invocations carry `initDurationMs`; their `durationMs + initDurationMs` should be within a couple of seconds of a warm invocation's `durationMs`. Lambda's `Duration` metric excludes init time, so it alone understates a cold start.

## Build, test and deploy
## Testing

Requires the [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html), Docker, and an AWS profile with access to the Orcasound account. Merging to `main` does not deploy; someone runs this:
[`tests/smoke.py`](tests/smoke.py) renders a synthetic MPEG-TS clip inside the built image, run as an unprivileged user with a root-owned empty `/tmp` and the CPU and memory of the deployed tier, which is how Lambda runs it. It fails if the first render is slow enough to suggest the caches were not restored. [CI](.github/workflows/ci.yml) runs it on every pull request; locally:

```bash
cd server/audio_viz
sam build
AWS_PROFILE=orcasound sam deploy
docker build -t spectrogram-renderer core
docker run --rm --user 1000:1000 --tmpfs /tmp:uid=0,gid=0,mode=1777 --cpus 0.58 --memory 1024m \
--entrypoint python -v "$PWD/tests/smoke.py:/var/task/smoke.py" spectrogram-renderer smoke.py
```

`sam build` builds the image and `sam deploy` pushes it to ECR and updates the CloudFormation stack, showing the change set first.
That rehearses the application, not the sandbox: Lambda's Runtime Interface Emulator in the base image reproduces the invoke API, not the filesystem rules. Anything that touches `/tmp`'s own metadata or relies on image contents under `/tmp` will pass locally on a plain tmpfs and fail deployed, hence the `--user` and `--tmpfs` flags above.

## Deploying

Merging to `main` deploys: the [CI workflow](.github/workflows/ci.yml) runs the smoke test, then `sam build && sam deploy` under the `production` environment, assuming the `spectrogram-renderer-deploy` IAM role through GitHub's OIDC provider. That role can change only this stack, its ECR repository and its function role.

To run the built function locally against a real event:
To deploy from a machine instead, with an AWS profile for the Orcasound account and the [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-sam-cli-install.html):

```bash
AWS_PROFILE=orcasound sam local invoke AudioVizFunction --event events/spectrogram_job.json
sam build
AWS_PROFILE=orcasound sam deploy
```

To tail its logs in AWS:
To invoke the deployed function on a real segment without writing anything (`image_key` null skips the upload):

```bash
sam logs -n AudioVizFunction --stack-name audio-viz --tail
aws lambda invoke --function-name "$(aws lambda list-functions --query "Functions[?contains(FunctionName,'AudioVizFunction')].FunctionName | [0]" --output text)" \
--cli-binary-format raw-in-base64-out --payload "$(cat events/spectrogram_job.json)" /dev/stdout
```

`tests/` holds SAM's generated unit tests; they do not yet exercise the handler.
Empty file removed __init__.py
Empty file.
63 changes: 0 additions & 63 deletions events/event.json

This file was deleted.

12 changes: 6 additions & 6 deletions events/spectrogram_job.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"id": "test_image",
"audio_bucket": "streaming-orcasound-net",
"audio_key": "rpi_sunset_bay/hls/1654758019/live004.ts",
"image_bucket": "dev-archive-orcasound-net",
"image_key": "spectrograms/live004_spect.png"
}
"id": "spectrogram_job_example",
"audio_bucket": "audio-orcasound-net",
"audio_key": "rpi_orcasound_lab/hls/1790665213/live5238.ts",
"image_bucket": null,
"image_key": null
}
Empty file removed tests/__init__.py
Empty file.
48 changes: 48 additions & 0 deletions tests/smoke.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
"""Renders one clip in a fresh container and reports how long each step took.

Run inside the built image, the way CI does, to rehearse a Lambda cold start:
an unprivileged user, a root-owned empty /tmp, and the CPU and memory of the
deployed tier. The clip is AAC in MPEG-TS like a real segment. The render
skips S3 both ways, as in warm.py.

Fails if the first render is slow enough to suggest the caches were not
restored, or if the two renders disagree about the image.
"""

import subprocess
import sys
import time

CLIP = "smoke.ts"
JOB = {
"id": CLIP,
"audio_bucket": "none",
"audio_key": "none",
"sample_rate": None,
"image_key": None,
"image_bucket": None,
}
# A cold render without the caches took over 30 s at this CPU share.
SLOW_FIRST_RENDER = 15.0

subprocess.run(
["ffmpeg", "-hide_banner", "-loglevel", "error", "-y", "-f", "lavfi", "-i",
"anoisesrc=d=10:c=pink:r=48000", "-ac", "1", "-c:a", "aac", "-f", "mpegts", f"/tmp/{CLIP}"],
check=True,
)

t0 = time.time()
import app # noqa: E402 (timed on purpose: this is where the caches are restored)

t1 = time.time()
first = app.make_spectrogram(JOB)
t2 = time.time()
second = app.make_spectrogram(JOB)
t3 = time.time()

print(f"import {t1 - t0:.1f}s | first render {t2 - t1:.1f}s | second render {t3 - t2:.1f}s | image {first['image_size']} bytes")

if first["image_size"] != second["image_size"]:
sys.exit(f"renders differ: {first['image_size']} vs {second['image_size']} bytes")
if t2 - t1 > SLOW_FIRST_RENDER:
sys.exit(f"first render took {t2 - t1:.1f}s; were the caches restored?")
Empty file removed tests/unit/__init__.py
Empty file.
72 changes: 0 additions & 72 deletions tests/unit/test_handler.py

This file was deleted.

Loading