diff --git a/CHANGELOG.md b/CHANGELOG.md index fdf70dc..8bc57d6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,24 @@ Every change a user would notice, newest first ([Keep a Changelog](https://keepa follow [Semantic Versioning](https://semver.org/); before 1.0, a minor version may change what a command does. ## [Unreleased] +### Changed + +- A function's box in the graph lists what it does in the order its code does it: a numbered row for each call to + its own functions, each AWS or library call and each HTTP request, indented inside loops and ifs, marked ↻ in a + loop and "if" when it runs only sometimes. Each call's arrow leaves from its row, and the boxes it calls are stacked + in the same order, so the graph reads top to bottom. A call not drawn is a dimmed row; click it, or **Open its + calls ›**, to draw the graph from that function. Calls in a row to one service the same way are one row + ("SQLite SELECT (+2)"), and a box lists at most 6 rows besides the ones its arrows leave from. +- Pointing at a box in the graph keeps every path through it in sight, not just the boxes next to it: what calls it + up to the entry point, and what it calls and reaches down to AWS, a database or the web. Pointing at a service + shows every path that reaches it. Clicking a box keeps its paths lit while its code is open. +- The graph draws 14 functions by default, down from 24, so a big codebase opens on a picture you can take in. + `--functions` still sets your own limit, and a box's calls not drawn still open as a graph of their own. +- A function that only passes on to one other no longer takes a box when the graph can't draw them all: the arrow + runs past it, dashed, and says which ("via load_config()"). +- On a graph of more than 10 functions, function boxes leave out their docstrings and when pills. Pointing at a box + shows all of it, and now also when it's called ("Called only when ..."). + ## [0.6.0] - 2026-10-20 ### Changed diff --git a/README.md b/README.md index 7eaffa2..8293e05 100644 --- a/README.md +++ b/README.md @@ -80,7 +80,7 @@ string is flagged. | `-e`, `--entry` | `module.function`, `module.Class.method`, `file.py[:function]`, or part of a name | | `-l`, `--list` | List entry points and stop | | `--plugins` | List platforms, libraries and deploy files and stop | -| `--all` | Draw every function (default: 24 functions, 5 calls deep) | +| `--all` | Draw every function (default: 14 functions, 5 calls deep) | | `--depth N`, `--functions N` | Your own limits | | `--serve`, `--port N` | Serve on localhost, redraw on change | | `--mermaid` | Print the graph as Mermaid | diff --git a/docs/images/graph-dark.webp b/docs/images/graph-dark.webp index 8881553..73c187b 100644 Binary files a/docs/images/graph-dark.webp and b/docs/images/graph-dark.webp differ diff --git a/docs/images/graph-light.webp b/docs/images/graph-light.webp index d446e05..0584ebe 100644 Binary files a/docs/images/graph-light.webp and b/docs/images/graph-light.webp differ diff --git a/docs/images/page-dark.webp b/docs/images/page-dark.webp index 9848c5d..f06f646 100644 Binary files a/docs/images/page-dark.webp and b/docs/images/page-dark.webp differ diff --git a/docs/images/page-light.webp b/docs/images/page-light.webp index f8181b6..f885978 100644 Binary files a/docs/images/page-light.webp and b/docs/images/page-light.webp differ diff --git a/docs/images/plugins-dark.webp b/docs/images/plugins-dark.webp index 8b421e1..84cdd95 100644 Binary files a/docs/images/plugins-dark.webp and b/docs/images/plugins-dark.webp differ diff --git a/docs/images/plugins-light.webp b/docs/images/plugins-light.webp index 3f168a5..40067f7 100644 Binary files a/docs/images/plugins-light.webp and b/docs/images/plugins-light.webp differ diff --git a/docs/reading.md b/docs/reading.md index 3db1c9f..2bd3921 100644 --- a/docs/reading.md +++ b/docs/reading.md @@ -24,6 +24,21 @@ Read it left to right. ![A code-node graph](images/graph-light.webp#only-light){ loading=lazy } ![A code-node graph](images/graph-dark.webp#only-dark){ loading=lazy } +### Inside a function's box { #rows } + +A function's box lists what it does, top to bottom in the order its code does it: each call to your own functions, +each AWS or library call, and each HTTP request, one numbered row each. The arrow for a call leaves from its row, +and the boxes it calls are stacked in the same order, so you can read down the box and across to the next column. + +| In a row | Means | +| --- | --- | +| Indented | Inside a loop or an `if` in this function | +| ↻ | In a loop: once per item | +| if | Only sometimes: point at the row for the test | +| Dimmed | A function not drawn here: click it to open the calls | +| `SQLite SELECT (+2)` | Three calls in a row to one service, the same way | +| +4 more | Calls left out of the box: **Open its calls ›** draws them all | + ### Arrow colours { #arrows } | Arrow | Means | @@ -37,7 +52,9 @@ Read it left to right. | ● amber | a warning is about it, such as an HTTP request with no timeout | !!! tip "Worth knowing" - Point at a box and everything it isn't connected to fades out. + Point at a box and everything not on a path through it fades out: what calls it, all the way up to the entry + point, and what it calls and reaches, all the way down. Point at S3 or a database to see every path that gets + there. Click a box and its paths stay lit while its code is open. ??? more "Zoom, full screen and big graphs" The graph sits in a box of one size, about the window's height, and is drawn at its real size. **Auto** shrinks @@ -57,11 +74,11 @@ Read it left to right. anywhere outside it, or the browser's Back button. Without JavaScript, × and Back still close it. ??? more "Open the calls a box leaves out" - A box whose calls aren't drawn says so in blue, "+12 calls not drawn". Click it and the graph starts from that - function, with every call it makes and whatever they reach. The line over the graph is the path down to it from + A box whose calls aren't drawn ends with **Open its calls ›** in blue, and its rows for them are dimmed. Click + either and the graph starts from that function, with every call it makes, in order, and whatever they reach. The line over the graph is the path down to it from the entry point: click a name in it to go back up, or the browser's Back button. **All** in the graph's bar draws - every function at once. This is the one part of the page that needs JavaScript; without it the line is plain - text, and the Step by step tab has every call. + every function at once. This is the one part of the page that needs JavaScript; without it the link isn't shown, + and the Step by step tab has every call. ## Step by step { #steps } diff --git a/docs/usage.md b/docs/usage.md index 79f9430..b5d8c55 100644 --- a/docs/usage.md +++ b/docs/usage.md @@ -65,10 +65,13 @@ with the folder's name, so you can tell them apart. ## Large projects { #limits } -The graph shows 24 functions, 5 calls deep: starting from the entry point, it keeps taking the function that leads to +The graph shows 14 functions, 5 calls deep: starting from the entry point, it keeps taking the function that leads to the most code (and to AWS, the web and environment variables), so the boxes are the ones doing the work, not the -nearest helpers. A box whose calls are left out says so ("+3 calls not drawn"): click that line and the graph starts -from that function instead, with every call it makes, and a path above it back up to the whole graph. **All** in +nearest helpers. A function that only passes on to another one doesn't take a box: the arrow goes past it, dashed, and +says so ("via load_config()"). Each function's box lists up to 6 of its calls in the order they run; past 10 +functions the boxes drop their docstrings, which are in the card that shows when you point at a box. A call not drawn +is dimmed in its box: click it, or **Open its calls ›**, and the graph starts from that function instead, with every +call it makes, and a path above it back up to the whole graph. **All** in the graph's bar draws every function at once, for up to 150 of them. The step-by-step list always has all of them. ```bash @@ -104,7 +107,7 @@ to see its commit. The gear in the panel's head turns that off. It's read with ` | `--plugins` | List the plugins | | `--serve`, `--port` | Live page (8765, or the next free port) | | `--all` | Draw every function | -| `--depth`, `--functions` | Graph limits (5, 24) | +| `--depth`, `--functions` | Graph limits (5, 14) | | `--mermaid` | Print Mermaid | | `-o`, `--out` | Where to save | | `--no-open` | Don't open a browser | diff --git a/src/code_node/analysis/graph.py b/src/code_node/analysis/graph.py index 58decff..0b3377f 100644 --- a/src/code_node/analysis/graph.py +++ b/src/code_node/analysis/graph.py @@ -33,6 +33,25 @@ class CodeSpan: lit: list[int] = field(default_factory=list) # the lines in it worth pointing at: where it calls, reads or sends +@dataclass +class GraphStep: + """ + A row of a function's box: one thing it does, in the order the code does it. A call to one of its own functions, or + an AWS call, a library's call or an HTTP request. + """ + + text: str # 'save()', 'S3 PutObject', 'Redis GET cart:{user}', 'POST https://...'; '+3 more' for the rows left out + line: int + kind: str = "call" # 'call', an AWS or library call's access ('reads', 'writes', 'sends', 'runs', 'signs'), + # 'http', or 'more' for the rows left out + target: str = "" # the box it leads to, by id: a function's box, or what it reaches; '' for a function not drawn + leads: list[str] = field(default_factory=list) # a function not drawn: the boxes reached through it + depth: int = 0 # the loops and ifs around it in its function + loop: bool = False # in a loop: once per item + when: str = "" # made only under an if: its test + name: str = "" # a call to one of its own functions: the function, as module.name + + @dataclass class GraphNode: """ @@ -57,6 +76,7 @@ class GraphNode: # point's logo, when its platform names one: 'fastapi-icon' name: str = "" # a function's box: the function, as module.name ('orders.repo.save') history: History | None = None # a function's box: what git says about its lines + steps: list[GraphStep] = field(default_factory=list) # a function's box: what it does, in order @dataclass @@ -161,7 +181,7 @@ def _aws_node(service: str) -> str: return "aws-" + (re.sub(r"[^a-z0-9]+", "-", service.lower()).strip("-") or "service") -_GRAPH_FUNCTIONS = 24 # functions code_graph() draws, at most (the ones that lead to the most code first) +_GRAPH_FUNCTIONS = 14 # functions code_graph() draws, at most (the ones that lead to the most code first) _REACH_WEIGHT = 30 # lines of code an AWS call, HTTP request or environment variable counts as, picking boxes @@ -169,6 +189,10 @@ def _aws_node(service: str) -> str: _GRAPH_LINES = 5 # lines a box lists before "+3 more" +_GRAPH_STEPS = 6 # rows of what a function does, at most, besides the ones that lead to a box (always shown) + +_STRUCTURE = ("loop", "if", "else", "except") # the steps around a line that make it run more than once, or not always" + _SPAN_BEFORE, _SPAN_AFTER = 2, 3 # lines shown around each place a box is about @@ -260,12 +284,16 @@ def _pick( room: int, max_depth: int, first: bool = False, + reaching: frozenset[str] = frozenset(), ) -> list[str]: """ The functions the graph draws besides the entry point, at most `room` of them and max_depth calls deep: grown out from the entry point (and what runs at import), each time taking the one the drawn boxes call that leads to the most code, so the boxes are the ones that do the work and each is joined to one above it. first takes every function the entry point calls before the rest. + When they don't all fit, a function that only passes on to one other (it calls one of its own functions and + reaches nothing itself: not in `reaching`) is passed through instead of taking a box: the one it calls is offered + in its place, and drawn joined to the box above by an arrow through it. """ def depth(name: str) -> int: @@ -288,6 +316,17 @@ def offer(name: str) -> None: if name not in level: offer(name) picked: list[str] = [] + passed: set[str] = set() # functions passed through + + def passes_on(name: str) -> str: # the one function it calls, when it only passes on to it + if name in reaching: + return "" + called = {child for child, _ in children.get(name, []) if child in by_name and child != name} + if len(called) != 1: + return "" + (child,) = called + return child if child not in picked and child not in passed and depth(child) <= max_depth else "" + if first: for child, _ in children.get(entry, []): if child in by_name and child != entry and child not in picked and depth(child) <= max_depth: @@ -298,7 +337,12 @@ def offer(name: str) -> None: offer(child) while queue and len(picked) < room: name = heapq.heappop(queue)[3] - if name in picked: + if name in picked or name in passed: + continue + child = passes_on(name) + if child: + passed.add(name) + offer(child) continue picked.append(name) for child, _ in children.get(name, []): @@ -344,6 +388,187 @@ def _focused(cmap: CodeMap, focus: str) -> CodeMap: ) +def _bodies(cmap: CodeMap) -> dict[str, CodeStep]: + """ + Each function's 'def' step with its lines under it, from the outline, by module.name. + """ + found: dict[str, CodeStep] = {} + for root in cmap.steps: + for step in root.walk(): + if step.kind == "def" and step.name and not step.seen and step.name not in found: + found[step.name] = step + return found + + +def _frames(body: CodeStep | None) -> list[tuple[int, int, str, str]]: + """ + The loops and branches in a function's own lines (not the functions it calls): (first line, last line, kind, code). + """ + out: list[tuple[int, int, str, str]] = [] + + def last(step: CodeStep) -> int: + return max([step.line] + [last(s) for s in step.steps]) + + def visit(steps: list[CodeStep]) -> None: + for step in steps: + if step.kind in _STRUCTURE: + out.append((step.line, last(step), step.kind, step.code)) + visit(step.steps) + + if body is not None: + visit(body.steps) + return out + + +def _steps( + f: CodeFunction, + cmap: CodeMap, + children: dict[str, list[tuple[str, str]]], + ids: dict[str, str], + by_name: dict[str, CodeFunction], + frames: list[tuple[int, int, str, str]], +) -> list[GraphStep]: + """ + What a function does, in the order its lines do it: its calls to its own functions, its AWS and library calls and + its HTTP requests, each with the box it leads to, how deep in loops and ifs it is, and whether it's in a loop or + under an if. A line in a loop's or an if's own header runs as often as the line itself, so isn't counted inside it. + Its loops are the ones in its own lines: a function called in a loop isn't in one itself. Calls in a row to one + service, the same way, are one row ('SQLite SELECT (+2)'). Past _GRAPH_STEPS rows, the rest are left out, counted + in a last '+3 more' row, but never a call to a function drawn, the first row to each service, nor the first call + through which each other box is reached. + """ + reached: dict[str, set[str]] = {} # a function not drawn -> the boxes reached through it + + def through(name: str) -> set[str]: + if name not in reached: + reached[name] = set() + boxes: set[str] = set() + for c in [c for c in cmap.aws if c.function == name and not c.startup]: + boxes.add(_aws_node(c.service)) + for lc in [c for c in cmap.library_calls if c.function == name and not c.startup]: + boxes.add(_service_node(lc.library, lc.service)) + if any(s.kind == "http" and s.function == name for s in cmap.spots): + boxes.add("http") + for child, _ in children.get(name, []): + if child in ids: + boxes.add(ids[child]) + elif child in by_name: + boxes |= through(child) + reached[name] = boxes + return reached[name] + + def around(line: int) -> list[tuple[int, int, str, str]]: + return [fr for fr in frames if fr[0] < line <= fr[1]] + + def test(line: int) -> str: + tests = [fr for fr in around(line) if fr[2] != "loop"] + if not tests: + return "" + code = tests[-1][3].rstrip(":") + return code[3:] if code.startswith("if ") else code + + rows: list[GraphStep] = [] + for child, line in children.get(f.name, []): + if child not in by_name or not line.isdigit(): + continue + at = int(line) + rows.append( + GraphStep( + f"{_short_name(child)}()", + at, + target=ids.get(child, ""), + leads=[] if child in ids else sorted(through(child)), + depth=len(around(at)), + loop=any(fr[2] == "loop" for fr in around(at)), + when=by_name[child].when.get(f"{f.name} line {line}", "") or test(at), + name=child, + ) + ) + for c in cmap.aws: + if c.function == f.name and not c.startup: + rows.append( + GraphStep( + f"{_service_name(c.service)} {c.operation}", + c.line, + c.access, + _aws_node(c.service), + depth=len(around(c.line)), + loop=any(fr[2] == "loop" for fr in around(c.line)), + when=test(c.line), + ) + ) + for lc in cmap.library_calls: + if lc.function == f.name and not lc.startup: + rows.append( + GraphStep( + f"{lc.service} {lc.row}", + lc.line, + lc.access, + _service_node(lc.library, lc.service), + depth=len(around(lc.line)), + loop=any(fr[2] == "loop" for fr in around(lc.line)), + when=test(lc.line), + ) + ) + for spot in cmap.spots: + if spot.kind == "http" and spot.function == f.name and not spot.startup: + rows.append( + GraphStep( + spot.detail, + spot.line, + "http", + "http", + depth=len(around(spot.line)), + loop=any(fr[2] == "loop" for fr in around(spot.line)), + when=test(spot.line), + ) + ) + seen: set[tuple[str, int, str]] = set() + ordered: list[GraphStep] = [] + runs: list[int] = [] # how many calls each row stands for + for row in sorted(rows, key=lambda r: r.line): # stable: calls on one line stay in the order they're read + if (row.text, row.line, row.kind) in seen: + continue + seen.add((row.text, row.line, row.kind)) + last = ordered[-1] if ordered else None + if last and row.kind != "call" and (last.kind, last.target) == (row.kind, row.target): + runs[-1] += 1 # calls in a row to one service, the same way: one row + last.loop |= row.loop + last.when = last.when if last.when == row.when else "" + continue + ordered.append(row) + runs.append(1) + for row, run in zip(ordered, runs): + if run > 1: + row.text += f" (+{run - 1})" + if len(ordered) <= _GRAPH_STEPS: + return ordered + needed: set[int] = set() # every call to a function drawn, the first way to each service, and the first call + # through which each other box is reached + firsts: set[tuple[str, str]] = set() + for k, row in enumerate(ordered): + if row.kind == "call" and row.target: + needed.add(k) + elif row.kind != "call" and (row.kind, row.target) not in firsts: + firsts.add((row.kind, row.target)) + needed.add(k) + led_to = {row.target for k, row in enumerate(ordered) if k in needed} + for k, row in enumerate(ordered): + if row.kind == "call" and not row.target and set(row.leads) - led_to: + led_to |= set(row.leads) + needed.add(k) + room = max(_GRAPH_STEPS - 1 - len(needed), 0) + kept = [] + for k, row in enumerate(ordered): + if k in needed: + kept.append(row) + elif room: + kept.append(row) + room -= 1 + left = len(ordered) - len(kept) + return kept + ([GraphStep(f"+{left} more", 0, "more")] if left else []) + + def code_graph( cmap: CodeMap, *, max_functions: int = _GRAPH_FUNCTIONS, max_depth: int = _GRAPH_DEPTH, focus: str = "" ) -> CodeGraph: @@ -403,18 +628,54 @@ def code_graph( room = max_functions - entry_is_function if focus: # a function opened on its own shows every call it makes room = max(room, len({child for child, _ in children.get(entry, [])})) - picked = _pick(cmap, entry, start_key, children, level, weight, order, room, max_depth, first=bool(focus)) + reaching = frozenset( + [c.function for c in cmap.aws] + + [c.function for c in cmap.library_calls] + + [e.function for e in cmap.env] + + [s.function for s in cmap.spots if s.kind == "http"] + + [key for _, _, key in cmap.routes] + ) + picked = _pick( + cmap, entry, start_key, children, level, weight, order, room, max_depth, first=bool(focus), reaching=reaching + ) shown = ([by_name[entry]] if entry_is_function else []) + sorted( (by_name[name] for name in picked), key=lambda f: (level.get(f.name, f.depth + 1), order[f.name]) ) graph.hidden = len(cmap.functions) - len(shown) graph.too_deep = sum(1 for f in cmap.functions if level.get(f.name, f.depth + 1) > max_depth) ids = {f.name: f"f{i}" for i, f in enumerate(shown)} + + # a box none of whose callers is drawn (they were passed through): an arrow from the nearest boxes above, through + # the functions between + through: dict[str, list[tuple[str, list[str]]]] = {} # a function -> (a caller drawn, the functions between) + for f in shown: + if f.name == entry or any(caller in ids or caller == start_key for caller, _ in parents.get(f.name, [])): + continue + found: list[tuple[str, list[str]]] = [] + seen, queue = {f.name}, [(f.name, list[str]())] + while queue: + key, between = queue.pop(0) + for caller, _ in parents.get(key, []): + if caller in seen: + continue + seen.add(caller) + if caller in ids or caller == start_key: + found.append((caller, between)) + elif caller in by_name: + queue.append((caller, [caller, *between])) + if found: + through[f.name] = found + passed = {name for found in through.values() for _, between in found for name in between} + via_children: dict[str, list[str]] = defaultdict(list) + for name, found in through.items(): + for caller, _ in found: + via_children[caller].append(name) + drawn_level = {entry: 0, start_key: 0} # calls from the entry point through the boxes drawn: each box's column todo = [entry, start_key] while todo: key = todo.pop(0) - for child, _ in children.get(key, []): + for child in [child for child, _ in children.get(key, [])] + via_children.get(key, []): if child in ids and child not in drawn_level: drawn_level[child] = drawn_level[key] + 1 todo.append(child) @@ -519,6 +780,13 @@ def warned_tip(tip: str, why: list[str]) -> str: when=_when(f, [where for where in f.called_from if where.rpartition(" line ")[0] == caller]), ) ) + for name, found in through.items(): + for caller, between in found: + source = "start" if caller == start_key else ids[caller] + if (source, ids[name]) not in seen_calls: + seen_calls.add((source, ids[name])) + said = ", ".join(f"{_short_name(k)}()" for k in between) + edges.append(GraphEdge(source, ids[name], "call", f"via {said}", indirect=True)) has_input = not focus and (bool(cmap.event) or platform.draws_input) if has_input: edges.insert(0, GraphEdge("event", entry_id, "event", "its input")) @@ -801,7 +1069,7 @@ def warned_tip(tip: str, why: list[str]) -> str: dict.fromkeys( f"{_short_name(child)}()" for child, _ in children.get(f.name, []) - if child in by_name and child not in ids + if child in by_name and child not in ids and child not in passed ) ) graph.nodes.append( @@ -815,6 +1083,7 @@ def warned_tip(tip: str, why: list[str]) -> str: warned_tip( f"{f.name}({f.args}) in {f.file}, {_plural(f.lines, 'line')}" + (f": {f.doc}" if f.doc else "") + + (f"\nCalled only when {when}" if (when := _when(f, f.called_from)) else "") + ( f"\nIt calls {_plural(len(more), 'function')} the graph leaves out: {', '.join(more)}" if more @@ -838,6 +1107,10 @@ def warned_tip(tip: str, why: list[str]) -> str: history=f.history, ) ) + bodies = _bodies(cmap) + nodes_by_id = {n.id: n for n in graph.nodes} + for f in shown: + nodes_by_id[ids[f.name]].steps = _steps(f, cmap, children, ids, by_name, _frames(bodies.get(f.name))) out_column = 1 + max([n.column for n in graph.nodes if n.kind in ("handler", "function", "start")] + [1]) for node in reach: node.column = out_column diff --git a/src/code_node/cli.py b/src/code_node/cli.py index 4543404..fb0141e 100644 --- a/src/code_node/cli.py +++ b/src/code_node/cli.py @@ -70,7 +70,10 @@ def _parse_args(argv: list[str] | None) -> argparse.Namespace: parser.add_argument("--all", action="store_true", help="draw every function, however deep (big graphs)") parser.add_argument("--depth", type=int, default=GRAPH_DEPTH, help="calls deep the graph draws (default 5)") parser.add_argument( - "--functions", type=int, default=GRAPH_FUNCTIONS, help="functions the graph draws at most (default 24)" + "--functions", + type=int, + default=GRAPH_FUNCTIONS, + help=f"functions the graph draws at most (default {GRAPH_FUNCTIONS})", ) parser.add_argument("--mermaid", action="store_true", help="print the graph as Mermaid instead of a page") parser.add_argument( diff --git a/src/code_node/pages.py b/src/code_node/pages.py index 0e5c61f..547cdf8 100644 --- a/src/code_node/pages.py +++ b/src/code_node/pages.py @@ -9,7 +9,7 @@ from dataclasses import dataclass, replace from pathlib import Path -from .analysis.graph import code_graph +from .analysis.graph import _GRAPH_DEPTH, _GRAPH_FUNCTIONS, code_graph from .analysis.mapping import entry_file, map_code from .analysis.model import CodeMap from .deploys import find_deployment @@ -19,9 +19,9 @@ from .report.editor import editor_link from .report.style import render_page -GRAPH_FUNCTIONS = 24 # functions the graph draws by default, the ones that lead to the most code first +GRAPH_FUNCTIONS = _GRAPH_FUNCTIONS # functions the graph draws by default, the ones that lead to the most code first -GRAPH_DEPTH = 5 # ...and how many calls deep +GRAPH_DEPTH = _GRAPH_DEPTH # ...and how many calls deep _ALL = 10_000 # --all: no limit that a real codebase reaches diff --git a/src/code_node/report/graph.py b/src/code_node/report/graph.py index cb204e0..24f56bb 100644 --- a/src/code_node/report/graph.py +++ b/src/code_node/report/graph.py @@ -18,7 +18,7 @@ from ..kit.text import _esc from ..analysis.findings import _short_name -from ..analysis.graph import CodeGraph, GraphEdge, GraphNode, GraphView +from ..analysis.graph import CodeGraph, GraphEdge, GraphNode, GraphStep, GraphView from .changes import blame_keys, blame_mark, blame_row, history_html from .brands import AWS_LOGOS, SERVICE_LOGOS, fills, logo, small_logo from .editor import editor_name, open_at @@ -86,6 +86,9 @@ class _Graph: _GRAPH_ROUTE = 28 # a web route's pill +_GRAPH_DENSE = 10 # function boxes past which a graph's boxes are compact: a line of their docstring, no when pill, +# and their calls not drawn counted, not named (all of it is in the box's card on hover) + _GRAPH_SWEEPS = 16 # passes that reorder each column by where its neighbours are, so fewer arrows cross; the order # with the fewest crossings is kept @@ -262,6 +265,8 @@ class _Graph: _G_LOOP = "M12.4 9.3A4.6 4.6 0 1 1 11.3 4.7M12.1 2.3v2.8H9.3" +_G_LOOP_MARK = "\u21bb" # a row in a loop + _G_ACCESS = ("reads", "writes", "sends", "runs", "signs") _G_LEGEND = ( @@ -422,6 +427,13 @@ def __init__(self, node: GraphNode) -> None: self.prose: list[str] = [] self.label: list[str] = [] # its name, in one line or two self.arrows = 0 # the most arrows on one of its sides: a box with many is tall enough to space them + self.compact = False # a function's box in a dense graph (_GRAPH_DENSE) + self.steps: list[GraphStep] = [] # a function's box: what it does, in order, one row each (rows: their keys) + self.opens = False # its calls not drawn open a graph of its own (with JavaScript) + + @property + def when(self) -> str: # its when pill's test, if it has one + return "" if self.compact else self.node.when def row_y(self, text: str) -> float | None: if text in self.rows: @@ -448,6 +460,8 @@ def _g_need(card: _GCard) -> float: need = max(need, whole) if card.route: need = max(need, 16 + _g_mono(_g_route(card)[0] + _g_route(card)[2], 12.5) + 36) + for step in card.steps: + need = max(need, _g_step_x(step) + _g_step_width(step) + _g_tag_width(step) + 14) if n.kind in _GRAPH_LISTS: need = max( [need] @@ -478,11 +492,16 @@ def _g_cards(graph: CodeGraph) -> dict[str, _GCard]: if node_id in cards: cards[node_id].arrows = max(cards[node_id].arrows, count) for e in graph.edges: # a web app's route goes on the function it leads to - if e.kind == "call" and e.label and not e.label.startswith("line ") and e.target in cards: + if e.kind == "call" and e.label and not e.label.startswith("line ") and not e.indirect and e.target in cards: cards[e.target].route = e.label + dense = sum(n.kind in ("handler", "function") for n in graph.nodes) > _GRAPH_DENSE for card in cards.values(): if card.node.kind in _GRAPH_LISTS: card.rows = list(card.node.lines) + elif card.node.steps: + card.steps = list(card.node.steps) + card.rows = [f"\0{k}" for k in range(len(card.steps))] + card.compact = dense and card.node.kind in ("handler", "function") return cards @@ -507,10 +526,22 @@ def _g_size(cards: dict[str, _GCard], widths: dict[int, float], column_of: dict[ else: card.label = _g_wrap(n.label, room, lambda t: _g_measure_label(n, t)) card.head = (_GRAPH_HEAD if n.sub else _GRAPH_HEAD_BARE) + _GRAPH_LABEL_LINE * (len(card.label) - 1) - top = card.head + (_GRAPH_ROUTE if card.route else 0) + (_GRAPH_ROUTE if n.when else 0) + top = card.head + (_GRAPH_ROUTE if card.route else 0) + (_GRAPH_ROUTE if card.when else 0) card.body = top if n.kind in _GRAPH_LISTS and card.rows: card.height = top + 6 + _GRAPH_ROW * len(card.rows) + 8 + elif card.steps: # what it does, in order, under a line or two of its docstring (none in a dense graph) + wrapped = [part for line in n.lines for part in _g_wrap(line, card.width - 30, lambda t: _g_sans(t, 12.5))] + card.prose = [] if card.compact else wrapped[:2] + card.body = top + (_GRAPH_PROSE * len(card.prose) + 6 if card.prose else 0) + card.height = card.body + 6 + _GRAPH_ROW * len(card.rows) + 6 + (_GRAPH_PROSE + 4 if card.opens else 0) + card.height = max(card.height, 24 + 12 * card.arrows) + elif card.compact: # its docstring's first line, cut to fit + card.prose = [_g_fit(n.lines[0], card.width - 30, lambda t: _g_sans(t, 12.5))] if n.lines else [] + card.height = top + (_GRAPH_PROSE * len(card.prose) + 10 if card.prose else 4) + if n.more: + card.height += _GRAPH_PROSE + (0 if card.prose else 8) + card.height = max(card.height, 24 + 12 * card.arrows) else: card.prose = [ part for line in n.lines for part in _g_wrap(line, card.width - 30, lambda t: _g_sans(t, 12.5)) @@ -593,8 +624,12 @@ def _graph_layout( fan_in[e.target] += 1 near: dict[str, list[str]] = defaultdict(list) hops: dict[int, list[str]] = {} + below: dict[tuple[str, str], float] = {} # (a function's box, what its arrow goes to first) -> how far below the + # box's middle the arrow's row is: what a function calls is put in the order of its rows, the order its code runs + share: dict[tuple[str, str], float] = {} # ...and how far down the box, from 0 to 1, for counting crossings for i, e in enumerate(graph.edges): a, b = column_of[e.source], column_of[e.target] + rows = [y for r in _g_rows(cards[e.source], e) if (y := cards[e.source].row_y(r)) is not None] chain = [e.source] if abs(b - a) > 1: # a waypoint in each column it crosses, either way, shared with the arrows it bundles with step = 1 if b > a else -1 @@ -612,6 +647,9 @@ def _graph_layout( height[hop] = 4.0 chain += hops[i] chain.append(e.target) + if rows and cards[e.source].steps: + below[(e.source, chain[1])] = sum(rows) / len(rows) - height[e.source] / 2 + share[(e.source, chain[1])] = (sum(rows) / len(rows)) / (height[e.source] + 1) for x, y in zip(chain, chain[1:]): if x != y: near[x].append(y) @@ -655,8 +693,8 @@ def crossings() -> int: """ rank = {item: j for items in columns.values() for j, item in enumerate(items)} count = 0 - for pairs in by_gap.values(): - ends = sorted((rank[x], rank[y]) for x, y in pairs) + for pairs in by_gap.values(): # an arrow leaving a function's row crosses the ones from the rows below it + ends = sorted((rank[x] + share.get((x, y), 0.0), rank[y] + share.get((y, x), 0.0)) for x, y in pairs) for j, (a1, b1) in enumerate(ends): count += sum(1 for a2, b2 in ends[j + 1 :] if a2 > a1 and b2 < b1) return count @@ -668,7 +706,11 @@ def crossings() -> int: items = columns[k] def pull(item: str) -> float: - ys = [center[m] for m in near[item] if column_of[m] != k] + ys = [ + center[m] + below.get((m, item), 0.0) - below.get((item, m), 0.0) + for m in near[item] + if column_of[m] != k + ] return sum(ys) / len(ys) if ys else center[item] items.sort(key=pull) @@ -679,6 +721,25 @@ def pull(item: str) -> float: columns.clear() columns.update(best[1]) center = best[2] + first_row: dict[str, tuple[str, float]] = {} # a function's box -> the box that calls it first, and the row + for e in graph.edges: + if e.kind == "call" and cards[e.source].steps and e.target not in first_row: + ys = [y for r in _g_rows(cards[e.source], e) if (y := cards[e.source].row_y(r)) is not None] + if ys and column_of[e.source] < column_of[e.target]: + first_row[e.target] = (e.source, min(ys)) + for k, items in columns.items(): # boxes one box calls, in the order of its rows: in the slots they're in + by_caller: dict[str, list[int]] = defaultdict(list) + for j, item in enumerate(items): + if item in first_row: + by_caller[first_row[item][0]].append(j) + moved = False + for slots in by_caller.values(): + ordered = sorted((items[j] for j in slots), key=lambda item: first_row[item][1]) + moved |= ordered != [items[j] for j in slots] + for j, item in zip(slots, ordered): + items[j] = item + if moved: + settle(items, sorted(center[item] for item in items)) top = min(center[item] - height[item] / 2 for item in center) last = len(order) - 1 # a call within the last column curls round into the margin; the others, into a gap through: dict[int, int] = defaultdict( @@ -721,6 +782,17 @@ def _g_rows(card: _GCard, e: GraphEdge) -> list[str]: The rows of its box an arrow leaves from or lands on: the variables it carries, the operations it calls (the '+3 more' row for one that isn't listed); none for the event as a whole or a client. """ + if card.steps: # a function's: the rows of the calls it leaves from (through a function not drawn, that one's) + if card.node.id != e.source or e.kind in ("event", "env", "client"): + return [] + hit = [ + key + for key, step in zip(card.rows, card.steps) + if (step.target == e.target and step.kind == e.kind) or (e.indirect and e.target in step.leads) + ] + if not hit and e.indirect and card.steps[-1].kind == "more": + hit = [card.rows[-1]] + return hit if card.node.kind not in _GRAPH_LISTS or e.kind in ("event", "client"): return [] if e.label in card.rows: @@ -751,6 +823,8 @@ def middle(node_id: str) -> float: for row in rows_for(cards[here], e): joined[here][row].append(middle(there)) for node_id, by_row in joined.items(): + if cards[node_id].steps: # a function's rows are in the order its code runs: never moved + continue rank = {row: sum(ys) / len(ys) for row, ys in by_row.items()} cards[node_id].rows.sort(key=lambda r: (r.startswith("+"), rank.get(r, 1e9))) @@ -868,7 +942,7 @@ def _g_edge_tip(e: GraphEdge, graph: CodeGraph) -> str: said = f"{src} {_EDGE_WORDS.get(e.kind, e.kind)} {dst}: {e.label}" said += "; in a loop, once per item" if e.loop else "" said += "; known only by the methods' names (their client couldn't be followed)" if e.guessed else "" - said += "; in functions further down that the graph leaves out" if e.indirect else "" + said += "; in functions further down that the graph leaves out" if e.indirect and e.kind != "call" else "" return said + ("; a warning is about it" if e.tone == "warn" else "") @@ -886,12 +960,14 @@ def _edge_svg( hot: set[tuple[int, int]] | None = None, taken: list[tuple[float, float]] | None = None, lane: int = 0, + lit: list[str] | None = None, ) -> tuple[str, str]: """ An arrow: its lines (one from each row it leaves to each row it reaches), its head, its loop mark; and, drawn over the boxes, the dots where it leaves them. A warned arrow's glow and ! go on the lines in `hot` (by the index of the row each leaves and reaches), or on all of them. Its marks go on its line, clear of the marks in `taken` (other arrows'), which they're added to; `lane` puts a call within a column outside the ones drawn before it. + `lit` names the boxes that, pointed at, leave it in sight (from _g_lineage()). """ sx, _, sw, _ = boxes[e.source] tx, _, tw, _ = boxes[e.target] @@ -935,6 +1011,7 @@ def _edge_svg( + (" warn" if e.tone == "warn" else "") + (" guess" if e.guessed else "") + (" via" if e.indirect else "") + + "".join(f" l-{_esc(m)}" for m in lit or [e.source, e.target]) ) taken = [] if taken is None else taken @@ -1008,16 +1085,100 @@ def _g_prose_top(card: _GCard) -> float: """ Where a box's prose starts: under its header, its route's pill and its when pill. """ - return card.head + (_GRAPH_ROUTE if card.route else 0) + (_GRAPH_ROUTE if card.node.when else 0) + return card.head + (_GRAPH_ROUTE if card.route else 0) + (_GRAPH_ROUTE if card.when else 0) def _g_more_y(card: _GCard) -> float: """ The baseline of a box's line of calls not drawn. """ + if card.steps: + return card.body + 6 + _GRAPH_ROW * len(card.rows) + 17 return _g_prose_top(card) + 13 + _GRAPH_PROSE * len(card.prose) +def _g_step_x(step: GraphStep) -> float: + """ + Where a row's text starts: past its number, indented for the loops and ifs around it, and past a dot for what it + reaches. + """ + return 34 + 12 * min(step.depth, 4) + (10 if step.kind not in ("call", "more") else 0) + + +def _g_step_width(step: GraphStep) -> float: + return _g_mono(step.text, 12.5) if step.kind != "more" else _g_sans(step.text, 12.5) + + +def _g_tag(step: GraphStep) -> str: # said at a row's right: in a loop, under an if + return " ".join(([_G_LOOP_MARK] if step.loop else []) + (["if"] if step.when else [])) + + +def _g_tag_width(step: GraphStep) -> float: + tag = _g_tag(step) + return _g_sans(tag, 11) + 10 if tag else 0 + + +def _g_step_tip(step: GraphStep) -> str: + """ + What a row says on hover: what it does and where, and why it may run more than once or not at all. + """ + said = f"{step.text}, line {step.line}" + if step.when: + said += f", only when {step.when}" + if step.loop: + said += ", in a loop: once per item" + if step.kind == "call" and not step.target: + said += "; not drawn" + return said + + +def _g_steps_svg(card: _GCard, w: float) -> list[str]: + """ + A function's rows: what it does in the order its code does it, numbered, indented for the loops and ifs around + each, a dot in its arrow's colour for what it reaches, ↻ for a loop and 'if' for a call made only sometimes. A + call to a function not drawn is dimmed. + """ + parts = [f''] + number = 0 + for key, step in zip(card.rows, card.steps): + ry = card.row_y(key) + if ry is None: + continue + if step.kind == "more": + parts.append(f'{_esc(step.text)}') + continue + number += 1 + x = _g_step_x(step) + tag = _g_tag(step) + room = w - x - 14 - _g_tag_width(step) + row = [ + f"{_esc(_g_step_tip(step))}", + f'{number}', + ] + if step.kind != "call": + row.append(f'') + row.append( + f'{_esc(_g_fit(step.text, room, lambda t: _g_mono(t, 12.5)))}' + "" + ) + if tag: + row.append(f'{_esc(tag)}') + drawn = step.kind != "call" or bool(step.target) + parts.append(f'{"".join(row)}') + return parts + + +def _g_off_rows(card: _GCard) -> list[float]: + """ + The baselines of a function's rows of calls to functions not drawn. + """ + return [ + y + 4.3 + for key, step in zip(card.rows, card.steps) + if step.kind == "call" and not step.target and (y := card.row_y(key)) is not None + ] + + def _node_svg( card: _GCard, box: tuple[float, float, float, float], @@ -1058,7 +1219,7 @@ def _node_svg( f"{_esc(more)}" ) top += _GRAPH_ROUTE - if n.when: # called only under an if: its test, in a pill like a route's + if card.when: # called only under an if: its test, in a pill like a route's test = _g_fit(n.when, w - 28 - 20 - _g_mono("when ", 12.5), lambda t: _g_mono(t, 12.5)) pill_w = _g_mono("when " + test, 12.5) + 20 parts.append( @@ -1089,9 +1250,13 @@ def _node_svg( ) for k, line in enumerate(card.prose): parts.append(f'{_esc(line)}') - if n.more: + if card.steps: + parts += _g_steps_svg(card, w) + if card.opens: # with JavaScript, a link to the graph of every call it makes + parts.append(f'Open its calls \u203a') + elif n.more: said = f"+{_g_calls(len(n.more))} not drawn" - for k in range(len(n.more), 0, -1): # as many of their names as fit whole, the rest as … + for k in range(0 if card.compact else len(n.more), 0, -1): # as many of their names as fit whole, the rest … names = ", ".join(n.more[:k]) + (", …" if k < len(n.more) else "") if _g_sans(f"{said}: {names}", 12.5) <= w - 28: said = f"{said}: {names}" @@ -1612,10 +1777,51 @@ def _g_hidden(graph: CodeGraph, opens: bool) -> str: head = f"+{graph.hidden}{'more function' if graph.hidden == 1 else 'more functions'} not drawn ({why})" pill = f'{head}: the Step by step tab has them all' if opens: - pill += f'{head}: click a box\'s blue line of calls to open them' + pill += f'{head}: click a dimmed call in a box, or Open its calls, to draw them' return pill +def _g_lineage(graph: CodeGraph) -> tuple[dict[str, set[str]], list[set[str]]]: + """ + What stays in sight when a box is pointed at: the boxes joined to it, and every box on a path through it, from + the entry point down to what it reaches; and for each arrow, the boxes it's on such a path of. Paths follow calls + and what the code reaches, never the input or the environment: those light only the boxes they go into. + """ + into: dict[str, set[str]] = defaultdict(set) + out: dict[str, set[str]] = defaultdict(set) + for e in graph.edges: + if e.kind not in ("event", "env"): + into[e.target].add(e.source) + out[e.source].add(e.target) + + def closure(start: str, step: dict[str, set[str]]) -> set[str]: + seen, todo = set(), [start] + while todo: + for other in step.get(todo.pop(), ()): + if other not in seen: + seen.add(other) + todo.append(other) + seen.discard(start) + return seen + + up = {n.id: closure(n.id, into) for n in graph.nodes} + down = {n.id: closure(n.id, out) for n in graph.nodes} + boxes = {n.id: up[n.id] | down[n.id] for n in graph.nodes} + for e in graph.edges: + boxes[e.source].add(e.target) + boxes[e.target].add(e.source) + arrows = [] + for e in graph.edges: + lit = {e.source, e.target} + if e.kind not in ("event", "env"): + for n in graph.nodes: + below, above = up[n.id] | {n.id}, down[n.id] | {n.id} + if (e.target in below and e.source in up[n.id]) or (e.source in above and e.target in down[n.id]): + lit.add(n.id) + arrows.append(lit) + return boxes, arrows + + def _g_canvas( graph: CodeGraph, block: _Graph, @@ -1632,13 +1838,13 @@ def _g_canvas( link}) has its line of calls not drawn made that link. """ cards = _g_cards(graph) + for n in graph.nodes: + cards[n.id].opens = bool(n.more and n.name in opens) boxes, routes, width, tall, xs, widths, column_of = _graph_layout(graph, cards) ends = _g_anchors(graph, cards, boxes) - near: dict[str, list[str]] = defaultdict(list) + near, lit = _g_lineage(graph) access: dict[str, dict[str, str]] = defaultdict(dict) # an AWS box's operation -> what it does for e in graph.edges: - near[e.source].append(e.target) - near[e.target].append(e.source) if e.kind in _G_ACCESS: for name in _g_names(e): access[e.target].setdefault(name, e.kind) @@ -1661,9 +1867,15 @@ def _g_canvas( f".{root} .cgw:has(#{key}:target) .cg .p-{key} .fg" "{stroke:light-dark(#2563eb,#60a5fa);stroke-opacity:1;stroke-width:2}" ) - for n in graph.nodes: # pointing at a box leaves only it, its arrows and the boxes they join in sight - report.rules.add(f".{root} .cg:has(.n-{n.id}:hover) .ce:not(.s-{n.id}):not(.t-{n.id}){{opacity:.06}}") + for n in graph.nodes: # pointing at a box leaves only it and the paths through it in sight, its own arrows bold + report.rules.add(f".{root} .cg:has(.n-{n.id}:hover) .ce:not(.l-{n.id}){{opacity:.06}}") report.rules.add(f".{root} .cg:has(.n-{n.id}:hover) .cn:not(.n-{n.id}):not(.a-{n.id}){{opacity:.28}}") + if n.id in panel_of: # ...and while its code is open, until another box is pointed at + open_box = ( + f".{root} .cgw:has(#{panel_of[n.id]}:target) .cg:not(:has(.cn:hover)):has(.n-{n.id}.p-{panel_of[n.id]})" + ) + report.rules.add(f"{open_box} .ce:not(.l-{n.id}){{opacity:.06}}") + report.rules.add(f"{open_box} .cn:not(.n-{n.id}):not(.a-{n.id}){{opacity:.28}}") report.rules.add(f".{root} .cg:has(.n-{n.id}:hover) .tp-{n.id}{{display:inline}}") report.rules.add( f".{root} .cg:has(.n-{n.id}:hover) :is(.s-{n.id},.t-{n.id}) .ln{{stroke-width:2.4;stroke-opacity:1}}" @@ -1701,6 +1913,7 @@ def _g_canvas( hot, taken, lane, + sorted(lit[i]), ) ) if e.loop: @@ -1719,19 +1932,21 @@ def _g_canvas( _node_svg( cards[n.id], boxes[n.id], - list(dict.fromkeys(near[n.id])), + sorted(near[n.id]), access[n.id], panel_of.get(n.id, ""), n.id in opened, ) ] ) - links = "".join( # over each line of calls not drawn that opens a graph of its own: drawn with JavaScript only + links = "".join( # over each line of calls not drawn that opens a graph of its own, and over each row of a call + # not drawn: drawn with JavaScript only f'' - f'' for n in graph.nodes if (link := opened.get(n.id)) + for y in [_g_more_y(cards[n.id]), *_g_off_rows(cards[n.id])] ) tips = "".join(_g_tip_svg(n, boxes[n.id], width, tall, lines_of, n.id in panel_of) for n in graph.nodes) kinds = {e.kind for e in graph.edges} @@ -1753,7 +1968,7 @@ def _g_canvas( "in a loop: once per item" ) hint = ( - 'Point at a box to see only what it\'s joined to' + 'Point at a box to see only the paths through it' + ("; click it to read its code" if panel_of else "") + 'Drag to move about; Ctrl or ⌘ and scroll, or pinch, to zoom' ) @@ -1884,7 +2099,7 @@ def _graph_lines(block: _Graph, report: _Report) -> list[str]: [ ", in a loop" if e.loop else "", ", guessed from a method's name" if e.guessed else "", - ", through functions not drawn" if e.indirect else "", + ", through functions not drawn" if e.indirect and e.kind != "call" else "", ] ) out.append(f" {said}" + (" (!)" if e.tone == "warn" else "")) diff --git a/src/code_node/report/style.py b/src/code_node/report/style.py index f2581ae..51e882d 100644 --- a/src/code_node/report/style.py +++ b/src/code_node/report/style.py @@ -170,6 +170,10 @@ .cnd .cg .cn .sb .sep{fill-opacity:.4} .cnd .cg .cn .dv{stroke:rgba(127,127,127,.2);stroke-width:1;fill:none} .cnd .cg .cn .rw{font:12.5px var(--mono);fill-opacity:.88} +.cnd .cg .cn .rn{font-size:11px;fill-opacity:.38;font-variant-numeric:tabular-nums} +.cnd .cg .cn .tg{font-size:11px;font-weight:600;fill-opacity:.5} +.cnd .cg .cn .sr.off .rw{fill-opacity:.42} +.cnd .cgw:not(.js) .cg .opn{display:none} .cnd .cg .cn .rw.mo{font-family:inherit;font-style:italic;fill-opacity:.55} .cnd .cg .cn .pr{font-size:12.5px;fill-opacity:.7} .cnd .cg .cn .pr.mo{font-style:italic;fill-opacity:.5} diff --git a/tests/test_code_node.py b/tests/test_code_node.py index 06a4338..3dfb477 100644 --- a/tests/test_code_node.py +++ b/tests/test_code_node.py @@ -634,9 +634,86 @@ def test_the_graph_draws_the_functions_that_do_the_work(tmp_path: Path) -> None: project = read_project(write(tmp_path, {"app.py": ENGINE})) cmap = map_code(project.files, "app.main") graph = code_graph(cmap, max_functions=3) - assert [n.label for n in graph.nodes if n.kind in ("handler", "function")] == ["main()", "run()", "work()"] + # run() only passes on to work(): work() takes the box, joined to main() through it, and tidy() the one left + assert [n.label for n in graph.nodes if n.kind in ("handler", "function")] == ["main()", "tidy()", "work()"] + labels = {n.id: n.label for n in graph.nodes} + via = [(labels[e.source], labels[e.target], e.label) for e in graph.edges if e.indirect] + assert via == [("main()", "work()", "via run()")] main_box = next(n for n in graph.nodes if n.label == "main()") - assert main_box.more == ["tidy()", "note()"] # said in its box, rather than looking like it does nothing + assert main_box.more == ["note()"] # said in its box, rather than looking like it does nothing + everything = code_graph(cmap) # when they all fit, none is passed through + assert "run()" in [n.label for n in everything.nodes] and not any(e.indirect for e in everything.edges) + + +def test_pointing_at_a_box_keeps_the_paths_through_it(tmp_path: Path) -> None: + from code_node.report.graph import _g_lineage + + code = ( + "import os\nimport boto3\n\n\ndef main():\n load()\n other()\n\n\ndef load():\n os.environ['A']\n" + " save()\n\n\ndef save():\n boto3.client('s3').put_object(Bucket='b', Key='k', Body=b'')\n\n\n" + "def other():\n pass\n" + ) + project = read_project(write(tmp_path / "code", {"app.py": code})) + graph = code_graph(map_code(project.files, "app.main")) + ids = {n.label: n.id for n in graph.nodes} + boxes, arrows = _g_lineage(graph) + # S3: every box on the way down to it, but not other(), nor the environment past the box that reads it + assert boxes[ids["S3"]] == {ids["main()"], ids["load()"], ids["save()"]} + assert ids["Environment"] in boxes[ids["load()"]] and ids["Environment"] not in boxes[ids["save()"]] + on_path = {(e.source, e.target) for e, lit in zip(graph.edges, arrows) if ids["S3"] in lit} + assert on_path == {(ids["main()"], ids["load()"]), (ids["load()"], ids["save()"]), (ids["save()"], ids["S3"])} + + +RUNS = """ +import boto3 + +s3 = boto3.client("s3") + + +def main(items): + setup() + for x in items: + if x: + work(x) + tidy(x) + s3.put_object(Bucket="b", Key="k", Body=b"") + s3.put_object(Bucket="b", Key="j", Body=b"") + + +def setup(): + pass + + +def work(x): + pass + + +def tidy(x): + pass +""" + + +def test_a_function_box_lists_what_it_does_in_the_order_it_runs(tmp_path: Path) -> None: + from code_node.report.graph import _g_cards, _g_rows, _graph_layout + + project = read_project(write(tmp_path, {"app.py": RUNS})) + graph = code_graph(map_code(project.files, "app.main")) + box = next(n for n in graph.nodes if n.label == "main()") + rows = [(s.text, s.depth, s.loop, s.when) for s in box.steps] + assert rows == [ + ("setup()", 0, False, ""), + ("work()", 2, True, "x"), # in the loop, under its if + ("tidy()", 1, True, ""), + ("S3 PutObject (+1)", 0, False, ""), # two in a row to one service, one row + ] + ids = {n.label: n.id for n in graph.nodes} + assert [s.target for s in box.steps] == [ids["setup()"], ids["work()"], ids["tidy()"], ids["S3"]] + cards = _g_cards(graph) + boxes, *_ = _graph_layout(graph, cards) + called = sorted(["setup()", "work()", "tidy()"], key=lambda label: boxes[ids[label]][1]) + assert called == ["setup()", "work()", "tidy()"] # the boxes it calls, top to bottom in the order it calls them + to_work = next(e for e in graph.edges if e.target == ids["work()"]) + assert _g_rows(cards[box.id], to_work) == [cards[box.id].rows[1]] # its arrow leaves from its row def test_arrows_from_one_box_run_together_and_the_graph_zooms(tmp_path: Path) -> None: