[SPARK-52764][ML][CONNECT][TESTS] Restore test_parity_classification and test_parity_regression

[zhengruifeng]

What changes were proposed in this pull request?

Restore test_parity_classification and test_parity_regression

the changes for model summary offloading in fd74b5e#diff-50ad4e7f2b34a05cb38c56cd47491239b09002d1f95e59b6a1d0b257e9ec5b8e seems problematical, and cause dead lock in the python side.

  test_multinomial_logistic_regression_with_bound (pyspark.ml.tests.connect.test_parity_classification.ClassificationParityTests.test_multinomial_logistic_regression_with_bound) ... Exception ignored in: <function JavaWrapper.__del__ at 0x108b5e020>
Traceback (most recent call last):
  File "/Users/ruifeng.zheng/spark/python/pyspark/ml/util.py", line 379, in wrapped
    self._remote_model_obj.release_ref()
  File "/Users/ruifeng.zheng/spark/python/pyspark/ml/util.py", line 162, in release_ref
    del_remote_cache(self.ref_id)
  File "/Users/ruifeng.zheng/spark/python/pyspark/ml/util.py", line 358, in del_remote_cache
    session.client._delete_ml_cache([ref_id])
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/core.py", line 2133, in _delete_ml_cache
    (_, properties, _) = self.execute_command(command)
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/core.py", line 1158, in execute_command
    data, _, metrics, observed_metrics, properties = self._execute_and_fetch(
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/core.py", line 1660, in _execute_and_fetch
    for response in self._execute_and_fetch_as_iterator(
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/core.py", line 1635, in _execute_and_fetch_as_iterator
    raise kb
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/core.py", line 1617, in _execute_and_fetch_as_iterator
    generator = ExecutePlanResponseReattachableIterator(
  File "/Users/ruifeng.zheng/spark/python/pyspark/sql/connect/client/reattach.py", line 127, in __init__
    self._stub.ExecutePlan(self._initial_request, metadata=metadata)
  File "/Users/ruifeng.zheng/.dev/miniconda3/envs/spark_dev_313/lib/python3.13/site-packages/grpc/_channel.py", line 1396, in __call__
    call = self._managed_call(
  File "/Users/ruifeng.zheng/.dev/miniconda3/envs/spark_dev_313/lib/python3.13/site-packages/grpc/_channel.py", line 1784, in create
    with state.lock:
  File "/Users/ruifeng.zheng/spark/python/pyspark/core/context.py", line 409, in signal_handler
    raise KeyboardInterrupt()
KeyboardInterrupt:

I plan to add dedicated test for model summary offloading in separate PR, this PR is to restore the basic coverage

Why are the changes needed?

for test coverage

Does this PR introduce any user-facing change?

no,test-only

How was this patch tested?

Manually run the test in my local, the hanging issue doesn't occur in successive 10 runs

Was this patch authored or co-authored using generative AI tooling?

no

[zhengruifeng]

merged to master