Comet Native Executors: run a query in Spark when Comet is turned off #7557

What is the problem the feature request solves?

With spark.comet.nativeExecutors.enabled on (#7289), a query run with spark.comet.enabled=false is still sent to the Comet Native Executor cluster, and the driver refuses it. CometNativeExecutors.applies checks spark.comet.nativeExecutors.enabled but not spark.comet.enabled. So CometRule offloads a plan that CometScanRule and CometExecRule left unconverted, because both return early when Comet is off (isCometLoaded). The stage-graph builder then fails on the plan's top operator with "Comet Native Executors cannot run this query, because of ...: it does not run natively".

Turning Comet off for a single query is the usual way to keep that query in Spark, for example a step that generates data with a Scala UDF before the queries Comet should run, or a test's withSQLConf. With the native executors on, that query fails instead.

Describe the potential solution

Have CometNativeExecutors.applies require spark.comet.enabled as well as spark.comet.nativeExecutors.enabled, so a query with Comet off runs in Spark, as it would without the cluster. Add a test in CometNativeExecutorsSuite that runs an aggregate with Comet off and checks that its plan has no CometNativeExecutorsExec.

Additional context

Part of #7288.

Willingness to contribute

Yes. I can contribute this fix independently.