Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ target
.idea
/docs/temp
/docs/build
/docs/ipython
.DS_Store
.vscode

Expand Down
17 changes: 17 additions & 0 deletions docs/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -34,9 +34,26 @@ fi

rm -rf build 2> /dev/null
rm -rf temp 2> /dev/null
rm -rf ipython 2> /dev/null
mkdir temp
cp -rf source/* temp/

# Give the notebook kernels a writable IPython profile of our own, and put
# `ipython_kernel_config.py` in it. Two reasons, both about build output:
#
# * Without a writable IPYTHONDIR, IPython warns "IPython parent '<home>' is
# not a writable location, using a temp directory" once per kernel start.
# * The config file filters ipykernel's unencrypted-TCP notice, which
# otherwise repeats once per executed page. See the comments in it.
#
# `IPKernelApp` reads the IPython profile directory rather than the Jupyter
# config path, so this has to be an IPYTHONDIR profile and not a
# `JUPYTER_CONFIG_PATH` entry. Kept out of temp/, which is the Sphinx source
# directory for this build.
export IPYTHONDIR="$script_dir/ipython"
mkdir -p "$IPYTHONDIR/profile_default"
cp ipython_kernel_config.py "$IPYTHONDIR/profile_default/"

# myst-nb executes each page as a notebook from the directory that page
# lives in, so the example data files must sit alongside every page that
# loads them by relative name (e.g. `ctx.read_csv("pokemon.csv")`). Symlink
Expand Down
62 changes: 62 additions & 0 deletions docs/ipython_kernel_config.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
# Licensed to the Apache Software Foundation (ASF) under one
# or more contributor license agreements. See the NOTICE file
# distributed with this work for additional information
# regarding copyright ownership. The ASF licenses this file
# to you under the Apache License, Version 2.0 (the
# "License"); you may not use this file except in compliance
# with the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing,
# software distributed under the License is distributed on an
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
# KIND, either express or implied. See the License for the
# specific language governing permissions and limitations
# under the License.

"""Config for the Jupyter kernel that executes the docs pages.

``build.sh`` copies this into a throwaway ``IPYTHONDIR`` profile, which is
where ``IPKernelApp`` looks for it. It is not on the Jupyter config path:
``IPKernelApp`` derives from ``BaseIPythonApplication`` and reads the IPython
profile directory, so ``JUPYTER_CONFIG_PATH`` has no effect on it.
"""

import logging

# ipykernel 7 warns at every kernel start that the ZeroMQ channels are
# unencrypted TCP. With `nb_execution_mode = "force"` the docs build starts a
# kernel per executed page, so the message repeats a dozen times per build and
# buries the output that matters.
#
# It does not describe a risk this build has. The kernel is a short-lived child
# process on the same machine, executing pages from this repository, reachable
# only over loopback. There is no second party to the conversation.
#
# Neither remedy the message itself suggests is usable here:
#
# - `transport="ipc"` does silence it, and it did work in isolation, but the
# real build then fails with "Kernel didn't respond in 60 seconds".
# - `KernelManager.transport_encryption = "auto"` fails at kernel start on
# jupyter_client 8.9.1, which hands the provisioned CurveZMQ key to the
# client as `str` where a `bytes` trait is expected.
#
# So the message is filtered rather than the condition removed. A filter is
# used in preference to `c.Application.log_level = "ERROR"` so that any *other*
# kernel warning still reaches the build log. If ipykernel rewords the message
# this filter stops matching and the warning comes back, which is the right way
# for it to fail.
_SUPPRESSED = "Kernel is running over TCP without encryption"


class _DropTransportEncryptionNotice(logging.Filter):
"""Drop ipykernel's unencrypted-TCP notice, and nothing else."""

def filter(self, record: logging.LogRecord) -> bool:
"""Return False only for the one message this build does not need."""
return _SUPPRESSED not in record.getMessage()


# Traitlets names an Application's logger after its class.
logging.getLogger("IPKernelApp").addFilter(_DropTransportEncryptionNotice())
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -178,6 +178,8 @@ extend-allowed-calls = ["datafusion.lit", "lit"]
]
"docs/*" = ["D"]
"docs/source/conf.py" = ["ANN001", "ERA001", "INP001"]
# A Jupyter kernel config file, executed by traitlets rather than imported.
"docs/ipython_kernel_config.py" = ["INP001"]

# CI and pre-commit invoke codespell with different paths, so we have a little
# redundancy here, and we intentionally drop python in the path.
Expand Down
Loading