Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion .env.test
Original file line number Diff line number Diff line change
Expand Up @@ -46,4 +46,11 @@ DISTANT_ALERT_API_LOGIN=''
DISTANT_ALERT_API_PASSWORD=''

KEYCLOAK_USERNAME=admin
KEYCLOAK_PASSWORD=admin_keycloak
KEYCLOAK_PASSWORD=admin_keycloak
# pyro-scripts service account (realm_pyronear.json): admin token for the init script and notebooks.
# The URL is the compose-network one; use http://localhost:9310/... when running scripts from the host.
KEYCLOAK_TOKEN_URL=http://keycloak:9310/realms/pyronear/protocol/openid-connect/token
KEYCLOAK_SCRIPTS_CLIENT_ID=pyro-scripts
KEYCLOAK_SCRIPTS_CLIENT_SECRET=pyro-scripts-dev-secret
# pyro-api client: the API introspects every Keycloak token with it
KEYCLOAK_API_CLIENT_SECRET=pyro-api-dev-secret
55 changes: 50 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,9 @@ make run
* **pyro-api**: Pyronear API (uvicorn)
* **db**: PostgreSQL database
* **rustfs**: S3-compatible storage (via RustFS)
* **keycloak**: identity provider for platform users and scripts (realm `pyronear`). The API image must
verify Keycloak tokens, which needs pyro-api's Keycloak support (build it with `make build-external`
until a release ships it).
* **frontend**: Web app (React, [pyro-platform](https://github.com/pyronear/pyro-platform))
* **pyro-engine**: Engine service (requires cameras, optional)
* **reolinkdev1 / reolinkdev2**: Fake Reolink cameras sending test images
Expand Down Expand Up @@ -70,7 +73,7 @@ docker logs engine

### Partial runs

* Backend only (API, DB, S3):
* Backend only (API, DB, S3, Keycloak):

```bash
make run-backend
Expand All @@ -97,6 +100,11 @@ docker logs engine
* Admin access currently does not display camera alerts, use user creds from `data/csv/users.csv`
* Configuration (API URL, etc.) lives in `containers/frontend/app-config.js`
* **Notebooks**: [http://localhost:8889](http://localhost:8889)
* **Keycloak admin console**: [http://localhost:9310/admin/](http://localhost:9310/admin/)

* Login: `KEYCLOAK_USERNAME` / `KEYCLOAK_PASSWORD` (set in `.env`)
* The realm is imported from `containers/keycloak/realm/realm_pyronear.json` only when the container is
created. After changing that file, recreate it: `docker compose up -d --force-recreate keycloak`
* **pgAdmin (db-ui)**: [http://localhost:8888/browser/](http://localhost:8888/browser/)

* Login: `DB_UI_MAIL` / `DB_UI_PWD` (set in `.env`)
Expand All @@ -115,10 +123,47 @@ docker logs engine

### Users Keycloak

| Username | Organization_id | Password |
|----------|-----------------|----------|
| test77 | 2 | test |
| test07 | 3 | test |
| Username | Organization_id | Realm role | Password |
|---------------|-----------------|------------|----------|
| test77 | 2 | operator | test |
| test07 | 3 | operator | test |
| test07-viewer | 3 | viewer | test |
| test-admin | 1 | admin | test |

Realm roles map to API roles: `admin` → `admin`, `operator` → `agent`, `viewer` → `user`.
The organization comes from the `pyro-organization` user attribute (an API organization id).

Access tokens for pyro-api carry:

* `aud`: includes `pyro-api` (`pyro-api` client scope)
* `roles`: the user's realm roles (`pyro-api` client scope)
* `organization_id`: the `pyro-organization` attribute (`pyro-organization` client scope)

### API client

pyro-api checks every Keycloak token twice: locally (signature, expiry, issuer, audience), then with
Keycloak's introspection endpoint, so a revoked session or a disabled user is rejected on its next request.
It authenticates those calls as the confidential `pyro-api` client (secret `KEYCLOAK_API_CLIENT_SECRET`,
dev-only value `pyro-api-dev-secret`).

### Scripts service account

The init script and notebooks call the API with an admin token from the `pyro-scripts` client
(client credentials grant, realm role `admin`, organization 1), configured by `KEYCLOAK_TOKEN_URL`,
`KEYCLOAK_SCRIPTS_CLIENT_ID` and `KEYCLOAK_SCRIPTS_CLIENT_SECRET` in `.env`. The secret is a dev-only value
committed in the realm export.

```python
from api import get_service_token
token = get_service_token(KEYCLOAK_TOKEN_URL, KEYCLOAK_SCRIPTS_CLIENT_ID, KEYCLOAK_SCRIPTS_CLIENT_SECRET)
```

Service-account tokens last 10 hours (`access.token.lifespan` on the `pyro-scripts` client, capped by the
realm's 10-hour session maximum) because the scripts fetch one token per run; user tokens keep the realm's
5 minutes.

From the host, use `http://localhost:9310/realms/pyronear/protocol/openid-connect/token` as the token URL.
Keycloak issues the same `iss` (`http://localhost:9310/realms/pyronear`) either way (`KC_HOSTNAME`).


### Add more images to Reolink Dev
Expand Down
12 changes: 8 additions & 4 deletions containers/init_script/init_script.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

# Import utility functions
from utils import (
get_token,
get_service_token,
api_request,
read_json_file,
write_json_file,
Expand All @@ -29,15 +29,19 @@
# API URL with /api/v1 for direct API requests
api_url = base_url + "/api/v1"

superuser_login = os.environ.get("SUPERADMIN_LOGIN")
superuser_pwd = os.environ.get("SUPERADMIN_PWD")
keycloak_token_url = os.environ.get("KEYCLOAK_TOKEN_URL")
keycloak_client_id = os.environ.get("KEYCLOAK_SCRIPTS_CLIENT_ID")
keycloak_client_secret = os.environ.get("KEYCLOAK_SCRIPTS_CLIENT_SECRET")
slack_hook = os.environ.get("SLACK_HOOK")
credentials_path = "data/credentials.json"
credentials_path_etl = "data/credentials-wildfire.json"


admin_token = get_service_token(
keycloak_token_url, keycloak_client_id, keycloak_client_secret
)
superuser_auth = {
"Authorization": f"Bearer {get_token(api_url, superuser_login, superuser_pwd)}",
"Authorization": f"Bearer {admin_token}",
"Content-Type": "application/json",
}

Expand Down
21 changes: 13 additions & 8 deletions containers/init_script/utils.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,25 +18,30 @@
import requests


def get_token(api_url: str, login: str, pwd: str) -> str:
def get_service_token(token_url: str, client_id: str, client_secret: str) -> str:
"""
Authenticate with API and return access token.
Get an access token from Keycloak for a service account (client credentials grant).

Args:
api_url: Base API URL (e.g., "http://api:5050/api/v1")
login: Username
pwd: Password
token_url: Realm token endpoint
(e.g., "http://keycloak:9310/realms/pyronear/protocol/openid-connect/token")
client_id: Confidential client with a service account (e.g., "pyro-scripts")
client_secret: Secret of that client

Returns:
Access token string
"""
response = requests.post(
f"{api_url}/login/creds",
data={"username": login, "password": pwd},
token_url,
data={
"grant_type": "client_credentials",
"client_id": client_id,
"client_secret": client_secret,
},
timeout=5,
)
if response.status_code != 200:
raise ValueError(response.json()["detail"])
raise ValueError(response.json().get("error_description", response.text))
return response.json()["access_token"]


Expand Down
142 changes: 138 additions & 4 deletions containers/keycloak/realm/realm_pyronear.json
Original file line number Diff line number Diff line change
Expand Up @@ -588,6 +588,47 @@
],
"notBefore": 0,
"groups": []
},
{
"username": "service-account-pyro-scripts",
"enabled": true,
"serviceAccountClientId": "pyro-scripts",
"attributes": {
"pyro-organization": [
"1"
]
},
"realmRoles": [
"default-roles-pyronear",
"admin"
],
"groups": []
},
{
"username": "test-admin",
"emailVerified": true,
"firstName": "Admin",
"lastName": "Pyronear",
"email": "admin@mail.com",
"attributes": {
"pyro-organization": [
"1"
]
},
"enabled": true,
"credentials": [
{
"type": "password",
"value": "test",
"temporary": false
}
],
"requiredActions": [],
"realmRoles": [
"default-roles-pyronear",
"admin"
],
"groups": []
}
],
"scopeMappings": [
Expand Down Expand Up @@ -853,7 +894,9 @@
"profile",
"roles",
"basic",
"email"
"email",
"pyro-api",
"pyro-organization"
],
"optionalClientScopes": [
"address",
Expand Down Expand Up @@ -974,6 +1017,56 @@
"organization",
"microprofile-jwt"
]
},
{
"clientId": "pyro-scripts",
"name": "Pyronear envdev scripts",
"description": "Service account used by the init script and notebooks (client credentials). Dev-only secret.",
"enabled": true,
"clientAuthenticatorType": "client-secret",
"secret": "pyro-scripts-dev-secret",
"publicClient": false,
"bearerOnly": false,
"standardFlowEnabled": false,
"implicitFlowEnabled": false,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": true,
"frontchannelLogout": false,
"protocol": "openid-connect",
"fullScopeAllowed": true,
"defaultClientScopes": [
"web-origins",
"acr",
"profile",
"roles",
"basic",
"email",
"pyro-api",
"pyro-organization"
],
"optionalClientScopes": [],
"attributes": {
"access.token.lifespan": "36000"
}
},
{
"clientId": "pyro-api",
"name": "Pyronear alert API",
"description": "Confidential client the API authenticates as to introspect access tokens. Dev-only secret.",
"enabled": true,
"clientAuthenticatorType": "client-secret",
"secret": "pyro-api-dev-secret",
"publicClient": false,
"bearerOnly": false,
"standardFlowEnabled": false,
"implicitFlowEnabled": false,
"directAccessGrantsEnabled": false,
"serviceAccountsEnabled": false,
"frontchannelLogout": false,
"protocol": "openid-connect",
"fullScopeAllowed": false,
"defaultClientScopes": [],
"optionalClientScopes": []
}
],
"clientScopes": [
Expand Down Expand Up @@ -1556,7 +1649,7 @@
"id.token.claim": "true",
"lightweight.claim": "true",
"access.token.claim": "true",
"claim.name": "organization",
"claim.name": "organization_id",
"jsonType.label": "int"
}
}
Expand Down Expand Up @@ -1695,6 +1788,46 @@
}
}
]
},
{
"name": "pyro-api",
"description": "Claims pyro-api relies on: its audience and the user's realm roles",
"protocol": "openid-connect",
"attributes": {
"include.in.token.scope": "false",
"display.on.consent.screen": "false",
"include.in.openid.provider.metadata": "false"
},
"protocolMappers": [
{
"name": "pyro-api audience",
"protocol": "openid-connect",
"protocolMapper": "oidc-audience-mapper",
"consentRequired": false,
"config": {
"included.custom.audience": "pyro-api",
"access.token.claim": "true",
"id.token.claim": "false",
"introspection.token.claim": "true",
"lightweight.claim": "false"
}
},
{
"name": "realm roles",
"protocol": "openid-connect",
"protocolMapper": "oidc-usermodel-realm-role-mapper",
"consentRequired": false,
"config": {
"claim.name": "roles",
"jsonType.label": "String",
"multivalued": "true",
"access.token.claim": "true",
"id.token.claim": "false",
"userinfo.token.claim": "false",
"introspection.token.claim": "true"
}
}
]
}
],
"defaultDefaultClientScopes": [
Expand All @@ -1706,7 +1839,8 @@
"web-origins",
"acr",
"basic",
"pyro-organization"
"pyro-organization",
"pyro-api"
],
"defaultOptionalClientScopes": [
"offline_access",
Expand Down Expand Up @@ -2801,4 +2935,4 @@
"clientPolicies": {
"policies": []
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,4 @@ parent=keycloak
import=common/keycloak
styles=css/login.css css/styles.css

# https://github.com/keycloak/keycloak/tree/main/themes/src/main/resources/theme/keycloak.v2/login
# https://github.com/keycloak/keycloak/tree/main/themes/src/main/resources/theme/keycloak.v2/login
22 changes: 20 additions & 2 deletions containers/notebooks/app/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,29 @@

import requests

__all__ = ["get_token", "get_camera_token"]
__all__ = ["get_service_token", "get_token", "get_camera_token"]


def get_service_token(token_url: str, client_id: str, client_secret: str) -> str:
"""Get a Keycloak access token for a service account (client credentials)"""
response = requests.post(
token_url,
data={
"grant_type": "client_credentials",
"client_id": client_id,
"client_secret": client_secret,
},
timeout=5,
)
response.raise_for_status()
return response.json()["access_token"]


def get_token(API_URL, login: str, passwrd: str) -> str:
"""Authenticate with API and return access token"""
"""Legacy: log in with API credentials.

Only works while pyro-api accepts API-issued user tokens.
"""
response = requests.post(
urljoin(API_URL, "/api/v1/login/creds"),
data={"username": login, "password": passwrd},
Expand Down
Loading
Loading