mcp: add lab (runtime) queries - #311
Conversation
Each CI matrix job installs one Python version, but invoking tox without an environment selector makes it evaluate every configured environment. The unavailable interpreters are then reported alongside the environment that actually ran:
py312: FAIL
py313: skipped because could not find python interpreter with spec(s): py313
py313: SKIP
py314: skipped because could not find python interpreter with spec(s): py314
py314: SKIP
evaluation failed :(
This output is visible in the Python 3.12 job from PR kernelci#311:
https://github.com/kernelci/kci-dev/actions/runs/33051625550/job/98448291347?pr=311
Add an explicit tox environment to each matrix entry and pass it to tox with -e. This preserves coverage for Python 3.12, 3.13, and 3.14 while ensuring every job runs only against the interpreter installed for it.
Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com>
65af198 to
40600b1
Compare
|
list_labs does not reliably discover labs accepted by list_builds Suggested fix:
|
|
labs_present becomes misleading when combined with status For example, when lava-collabora has only passing tests:
returns no results and omits lava-collabora from labs_present. This still makes a valid lab with no failures indistinguishable from a misspelled lab, which is precisely what the hint intends to solve. Capture the complete entry set before applying the status filter, or return separately named status-scoped and unfiltered lab lists. A regression test combining a valid lab, status="fail" and only passing entries should be added. |
|
Thanks @aliceinwire. I have fixed up, and also added the regression test you stated. However, on the /labs/ endpoint. Agreed in principle, but it isn't deployed yet, i.e. /labs/ 404'ing. I suggest we push this as a follow up PR when this goes through if you are okay with that? |
|
@bhcopeland please rebase |
|
|
||
|
|
||
| @tool_errors | ||
| def list_labs(days: int = 7): |
There was a problem hiding this comment.
It returns /metrics/’s lab_maps, which the backend derives from tests. Its builds count means distinct builds referenced by those tests, not builds produced by that lab. This can omit labs that only produce builds and makes the advertised discovery workflow for list_builds() unreliable.
Add list_labs, backed by the dashboard /metrics/ endpoint, returning the labs reporting to KernelCI with their build, boot and test counts for the last N days. Those names feed a new lab filter on list_builds, list_boots and list_tests, matching the top-level 'lab' field on boots and tests and 'misc.lab'/'misc.runtime' on builds. The dashboard has no server-side lab filter, so the filter narrows the fetched page rather than the request. Keep the 'labs' key in the compact get_summary response, which already aggregates per lab, and document data.runtime and data.platform in the list_nodes docstring. Reported-by: Yogesh Lal <yogesh.lal@oss.qualcomm.com> Signed-off-by: Denys Fedoryshchenko <denys.f@collabora.com> Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
days was passed through unchecked to a metrics aggregation that gets slower as its window grows. Measured against production: 3 days answers in under 10s cold, 7 days takes around 56s, and 14 and 30 days always exceed the 60s read timeout and come back as a bare "Dashboard metrics request failed" after a full minute. Cap the window at 7 days, so a caller cannot ask for one that cannot succeed. An unknown lab returned matched=0, which is indistinguishable from a lab that ran nothing on that commit. Return the labs the fetched entries actually report, so a mistyped name shows up without a second call. Validating up front is not possible: lab names are an open set and the only source is the metrics call itself. Drop 'maestro' as the example lab for builds. Every build from that origin reports it as misc.lab, so filtering by it matches everything; the runtime cluster is the value that discriminates. Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
labs_present was collected after the status filter, so a real lab with no entries in the requested status was left out of it: asking for status=fail on a lab whose tests all passed returned no results and no mention of the lab, which is exactly the mistyped-name case the hint exists to rule out. Collect it from the commit's entries before any status filter, so it answers "does this lab appear here at all" rather than "does it appear with this status". Signed-off-by: Ben Copeland <ben.copeland@linaro.org>
c131423 to
5d5fd44
Compare
|
Rebased (and added a to docs to your latest comment) |
3 participants
This is also based on #310 so that needs to be merged first.
Adds list_labs and a lab filter to the list tools, so "how is lab X doing" no longer means listing a whole tree and filtering client-side. list_labs is capped at 7 days because wider windows cannot succeed.
The feature commit is Denys's work, carried here with his authorship. The follow-up commit bounds list_labs and makes an unmatched lab self-explanatory; it also adjusts one docstring from the first commit, deliberately kept separate so the correction is attributed to me rather than folded into his commit.