RareLabelEncoder fails when replacement label already exists in categorical dtype #1095

Bug

RareLabelEncoder.transform unconditionally adds replace_with to a pandas Categorical column. If that label is already in the column's categories, pandas raises ValueError: new categories must not include old categories. This also prevents transforming an already transformed categorical dataframe a second time.

Reproduction

Current main d866312, Python 3.14 / pandas 3.0.5:

import pandas as pd
from feature_engine.encoding import RareLabelEncoder

X = pd.DataFrame({'cat': pd.Categorical(['a', 'a', 'Rare', 'b'],
                                      categories=['a', 'b', 'Rare'])})
enc = RareLabelEncoder(tol=.3, n_categories=1, variables=['cat']).fit(X)
print(enc.encoder_dict_)  # {'cat': ['a']}
enc.transform(X)  # ValueError: new categories must not include old categories: {'Rare'}

Expected / proposed fix

Return ['a', 'a', 'Rare', 'Rare'], retaining the categorical dtype, category order/ordered flag and the input dataframe. Add replace_with only if it is not already a category. The existing add_categories path remains for genuinely new labels.

I am working on a focused main-branch fix, independent expected-output regressions and validation on pandas 2.2 and 3.0. The merged RareLabelEncoder narwhals migration #1030 was inspected: its pandas path still calls add_categories unconditionally, so this guard is also relevant there; no migration implementation is included in this proposal.

This investigation and contribution are produced autonomously with Codex assistance for AHMETHAKANBEZIR1; no independent human review is claimed.