Skip to content

new atime ci, no more old cml container #7901

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
tdhock wants to merge 7 commits into
base: master
Choose a base branch
from
+124 −7
Draft

new atime ci, no more old cml container #7901

Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
411b562
new atime ci, no more old cml container
tdhock Sep 18, 2026
b2f0d9d
run ci if yml config modified
tdhock Sep 18, 2026
3142777
rm workflow_dispatch which did not work anyway
tdhock Sep 18, 2026
e9abfaa
rm @main
tdhock Sep 18, 2026
feb22a3
Compile without -Werror=format-security
aitap Sep 18, 2026
e1fa82b
Allow manual dispatch
aitap Sep 18, 2026
4dc8c82
Revert "Allow manual dispatch"
aitap Sep 23, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
131 changes: 124 additions & 7 deletions .github/workflows/performance-tests.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,132 @@ on:
- 'R/**'
- 'src/**'
- '.ci/atime/**'
workflow_dispatch:
- '.github/workflows/performance-tests.yml'

jobs:
comment:
if: github.repository == 'Rdatatable/data.table'
runs-on: ubuntu-latest
container: ghcr.io/iterative/cml:0-dvc2-base1
env:
GITHUB_PAT: ${{ secrets.GITHUB_TOKEN }}
repo_token: ${{ secrets.GITHUB_TOKEN }}
steps:
- uses: Anirban166/Autocomment-atime-results@v1.4.3

@MichaelChirico MichaelChirico Sep 18, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC this is designed to supplant the old action which can't be updated? Is there anything stopping us from forking it instead (or, if the current PR is intended as such a fork, could we at least host it in a dedicated repo under Rdatatable or maybe under your tdhock?)

@tdhock tdhock Sep 22, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

right, the current PR is a fork. @Anirban166 would you consider putting this new code in your Autocomment-atime-results repo? (I was not sure if Ani is willing / able to continue maintenance, but that would be great)
The basic problem with the old action is that is uses an old cml container, old compilers, and this PR fixes that.

- name: Start timer
run: echo "START_SETUP_TIME=$(date +%s)" >> $GITHUB_ENV
shell: bash
- name: Checkout
uses: actions/checkout@v6
with:
fetch-depth: 0
- name: Retrieve branch references, necessary for perf testing past versions
shell: bash
run: |
git switch "${GITHUB_BASE_REF}"
git switch "${GITHUB_HEAD_REF}"
- uses: actions/setup-node@v6
name: setup-node https://github.com/actions/setup-node?tab=readme-ov-file#usage
- name: Safe directory exception
run: |
git config --global --add safe.directory '*'
shell: bash
- name: Setup
uses: eddelbuettel/github-actions/r-ci@master
- name: Dependencies
run: ./run.sh install_all
- name: Adjust compilation flags for old data.table versions
run: sudo sed -i 's|-Werror=format-security||' `Rscript -e 'writeLines(R.home("etc"))'`/Makeconf
Comment thread
tdhock marked this conversation as resolved.
- name: Install and run atime
run: |
echo "options(repos = c(CRAN = 'https://cloud.r-project.org'))" >> .Rprofile
Rscript -e 'install.packages(c("atime", "ggplot2", "directlabels"))'
echo "END_SETUP_START_TESTS_TIME=$(date +%s)" >> $GITHUB_ENV
Rscript -e 'atime::atime_pkg(Sys.getenv("GITHUB_WORKSPACE"), tests.dir = ".ci");'
echo "END_TESTS_TIME=$(date +%s)" >> $GITHUB_ENV
shell: bash
- name: Upload artifact
id: artifact-upload-step
uses: actions/upload-artifact@v4
with:
name: atime-results
path: .ci/atime/
- name: comments.json
if: ${{ github.event_name == 'pull_request' }}
env:
PR_NUMBER: ${{ github.event.number }}
TAG: atime
shell: bash
run: |
curl --no-progress-meter -L -H "Accept: application/vnd.github+json" -H "X-GitHub-Api-Version: 2026-03-10" https://api.github.com/repos/$GITHUB_REPOSITORY/issues/$PR_NUMBER/comments > comments.json
cat comments.json
echo > comment-number.txt
echo "<!-- automatic comment for $TAG -->" > comment.md
cat comment.md
- name: comment-number.txt
if: ${{ github.event_name == 'pull_request' }}
shell: node {0}
run: |
fs = require('fs');
tag = fs.readFileSync('comment.md', 'utf-8');
console.log(tag);
content = fs.readFileSync('comments.json', 'utf-8').replaceAll("\r", "");
console.log(content);
prc = JSON.parse(content);
flist = prc.filter((o) => { return o.body.startsWith(tag); });
console.log(flist);
if(flist.length == 0) process.exit(0);
comment_id = flist[0].id;
console.log(comment_id);
fs.writeFileSync('comment-number.txt', comment_id+"");
- name: add images
shell: bash
run: |
uploadImage() {
local png=$1
curl -F "file=@$png" -F "visibility=unlisted" https://picrd.com/api/upload | sed 's/.*image_url":"//' | sed 's/",".*//'
}
cat comment-number.txt
PREVIEW_URL=$(uploadImage .ci/atime/tests_preview_facet.png)
ALL_URL=$(uploadImage .ci/atime/tests_all_facet.png)
echo $PREVIEW_URL $ALL_URL
cat .ci/atime/HEAD_issues.md >> comment.md
echo -e "\n[![Comparison Plot]($PREVIEW_URL)]($ALL_URL)" >> comment.md
echo -e "\nGenerated via commit ${{ github.event.pull_request.head.sha }}" >> comment.md
echo -e "\nDownload link for the artifact containing the test results: [↓ atime-results.zip](${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}/artifacts/${{ steps.artifact-upload-step.outputs.artifact-id }})" >> comment.md
- name: add content to comment.md
run: |
R_SETUP_DURATION=$((END_SETUP_START_TESTS_TIME - START_SETUP_TIME))
VERSION_SETUP_DURATION=$(cut -d '.' -f 1 ./.ci/atime/install_seconds.txt)
ATIME_TESTS_DURATION=$((END_TESTS_TIME - END_SETUP_START_TESTS_TIME - VERSION_SETUP_DURATION))
formatTime() {
local totalSeconds=$1
if [ $totalSeconds -lt 0 ]; then
echo "Time calculation error: negative duration"
return
fi
if [ $totalSeconds -lt 60 ]; then
echo "${totalSeconds} seconds"
else
local minutes=$((totalSeconds / 60))
local seconds=$((totalSeconds % 60))
echo "${minutes} minutes and ${seconds} seconds"
fi
}
echo -e "\n| Task | Duration |" >> comment.md
echo -e "|---|---|" >> comment.md
echo -e "| R setup and installing dependencies | $(formatTime $R_SETUP_DURATION) |" >> comment.md
echo -e "| Installing different package versions | $(formatTime $VERSION_SETUP_DURATION) |" >> comment.md
echo -e "| Running and plotting the test cases | $(formatTime $ATIME_TESTS_DURATION) |" >> comment.md
- name: convert comment to json https://stackoverflow.com/questions/1251999/how-can-i-replace-each-newline-n-with-a-space-using-sed/1252191#1252191 then post
if: ${{ github.event_name == 'pull_request' }}
env:
PR_COMMENT_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR_NUMBER: ${{ github.event.number }}
shell: bash
run: |
ISSUES_API=https://api.github.com/repos/$GITHUB_REPOSITORY/issues
PR_API=$ISSUES_API/$PR_NUMBER/comments
VERSION=2026-03-10
ACCEPT=application/vnd.github+json
COMMENT_NUMBER=$(cat comment-number.txt)
if [ "$COMMENT_NUMBER" == "" ]; then CMD=$PR_API; else CMD=$ISSUES_API/comments/$COMMENT_NUMBER; fi; echo $CMD
cat comment.md
MD=$(sed 's/"/\\"/g' comment.md | sed ':a;N;$!ba;s/\n/\\n/g')
echo '{"body":"'$MD'"}' > pr-comment.json
cat pr-comment.json
curl --no-progress-meter -L -H "Accept: $ACCEPT" -H "X-GitHub-Api-Version: $VERSION" -X POST -H "Authorization: Bearer $PR_COMMENT_TOKEN" $CMD -d "@pr-comment.json"
Loading