Skip to content

Fix modern allow_unicode combining mark word split - #7

Open
lomeliDev wants to merge 1 commit into
masterfrom
issue/4-modern-allow-unicode-combining-mark-splits-a-word-is
Open

lomeliDev wants to merge 1 commit into
masterfrom
issue/4-modern-allow-unicode-combining-mark-splits-a-word-is

Conversation

@lomeliDev

Copy link
Copy Markdown
Owner

Summary

Fix the modern allow_unicode=True pipeline so combining marks produced by lowercasing remain attached to their Unicode word instead of becoming a separator. This addresses #4 without changing the frozen legacy algorithm.

Cause and change

İ.lower() produces i followed by COMBINING DOT ABOVE (Mn). The modern default Unicode disallowed-character pattern replaced that mark with -, producing i-stanbul. Modern filtering now preserves adjacent Mn/Mc marks, while explicit regex_pattern behavior remains unchanged. Lowercasing also applies NFC normalization where composition is defined.

Files changed:

  • slugify/slugify.py
  • tests/test_modern_unicode.py

Evidence

Red view, before the production fix:

$ python3 -m pytest -p no:cacheprovider tests/test_modern_unicode.py
1 failed
E AssertionError: assert '-' not in 'i-stanbul'

Green view, after the production fix:

$ python3 -m pytest -p no:cacheprovider tests/test_modern_unicode.py
1 passed

Additional available tests after the fix:

$ python3 -m pytest -p no:cacheprovider tests/test_modern_unicode.py tests/test_modern_word_boundary.py tests/test_add_uppercase_error.py tests/test_cli_interrupt.py
8 passed

The repository CI workflow matrix was read before validation. The required taller workflow runs Python 3.11 and 3.12 with flake8, pycodestyle, and the full pytest suite; the upstream workflows additionally declare Python 3.10–3.14 and PyPy 3.11. This workspace has Python 3.11.2 only. The focused tests pass; the full suite could not run because the available environment lacks the declared text-unidecode and optional Unidecode packages. flake8, pycodestyle, and mypy are also not installed locally, so those checks were not reproduced. No dependencies, lockfiles, workflows, or unrelated files were changed.

The diff is limited to the Issue's requested slugify/slugify.py and tests/test_modern_unicode.py files.

Co-authored-by: CommandCodeBot <noreply@commandcode.ai>
@lomeliDev

Copy link
Copy Markdown
Owner Author

Revisión adversarial independiente — CON REPAROS

Revisado por quien no escribió el código, en worktree propio (origin/pr/7), import resuelto al worktree, Python 3.11.2.

El arreglo es funcionalmente correcto (vista roja/verde reproducida):

# SIN el cambio de producción (revert de slugify.py, test presente):
FAILED test_combining_mark_does_not_split_word - AssertionError: assert '-' not in 'i-stanbul'
# CON el cambio:
1 passed

Suite completa con el cambio: python -m pytest tests -q → 134 passed, 5 skipped, 136 subtests passed, sin regresiones (base 133 + 1 nuevo). La normalización NFC tras lower() y el reemplazo que conserva marcas Mn/Mc adyacentes a alfanuméricos no alteran el resto de salidas modernas ni el legacy.

REPARO (bloqueante): el PR deja el CI del fork en ROJO. El check designado taller-ci falla en ambos jobs (3.11 y 3.12) en el paso de lint. Reproducido localmente con el comando exacto del workflow:

$ flake8 --ignore=E501,F403,F401,E241,E225,E128 --exclude=legacy_reference.py slugify setup.py tests tools
slugify/slugify.py:75:61: W504 line break after binary operator          # exit 1
$ pycodestyle --ignore=E128,E261,E225,E501,W605 --exclude=legacy_reference.py slugify tests setup.py tools
slugify/slugify.py:75:61: W504 line break after binary operator          # exit 1

W504 no está en la lista de --ignore de ninguno de los dos linters, así que el salto de línea tras or dentro de _replace_disallowed_unicode tumba el gate. (El cuerpo del PR declaraba flake8/pycodestyle «no reproducidos» — justo el hueco que esto delata.)

Arreglo trivial: reformatear la expresión de la línea 75 para que or no quede al final de línea (p. ej. colapsar la condición o poner el operador al inicio de la línea siguiente). Con eso el PR queda aprobable.

Alcance correcto (slugify/slugify.py + test nuevo; _legacy.py intacto).

— revisión adversarial automática, sesión Claude Code (independiente del autor).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant