I have noticed that if I collect all the mfd used for the S-test and take the sum of the rates, and sum over the different cells I get a different total expected rate than the N-test. I have verified that by summing over all cell_id as follows

results['branch_id']['gem']['S_test']['val']['test_data']['cell_loglikes']['cell_id']['mod_mfd']

Source of the problem

I think the problem comes from how the mfd for each cell are calculated. They are calculated using

rate_mfd = get_model_mfd( rup_gdf, mag_bins, t_yrs=t_yrs, completeness_table=completeness_table )

and the get_model_mfd does

 if completeness_table is not None:
        model_mfd = {}
        for mag, rate in annual_mfd.items():
            duration = get_mag_duration_from_comp_table(
                completeness_table,
                mag,
                stop_date=stop_date,
            )
            model_mfd[mag] = rate * duration
 elif t_yrs is not None:
        model_mfd = {mag: rate * t_yrs for mag, rate in annual_mfd.items()}

So the t_yrs argument is ignored when providing a completeness table and the rates are computed for an incorrect time interval. Specifically, when there is no stop_date the function get_mag_duration_from_comp_table does this

    if stop_date is None:
        stop_date = datetime.datetime.now().date()

so this is set to today. In my case, I had specified a stop_date (which is ignored for the S-test) and [1970, 5] as completeness table. This leads to overestimation of the rates and therefore the S-test results is biased.

Solution

A simple solution is to use the stop_date instead of t_yrs as this is used to calculate the duration when a completeness table is provided, and we have the stop_date from the config file.