Add scan concurrency config for datafusion source

[AdamGS]

Make the underlying scan concurrency configurable through datafusion

[a10y]

doesn't DF let you set this for an entire sessioncontext? do we want to override this on a per-source basis?

[AdamGS]

I think that's the copy of VortexOptions that is part of VortexFormat, which propagates it downstream from there through either VortexFormatFactory::create or VortexFormat::file_source

[a10y]

I was wondering if it makes sense to use the target_partitions config off the environment, but I realize that's different.

Maybe we can make it clear in the doc comment that this is the intra-partition concurrency

[AdamGS]

intra-partition was the term I was looking for!

[codecov]

Codecov Report

❌ Patch coverage is 60.71429% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.39%. Comparing base (d757be1) to head (f1a5852).
⚠️ Report is 3 commits behind head on develop.

Files with missing lines	Patch %	Lines
vortex-datafusion/src/persistent/source.rs	30.00%	7 Missing ⚠️
vortex-datafusion/src/persistent/format.rs	62.50%	3 Missing ⚠️
vortex-datafusion/src/persistent/opener.rs	90.00%	1 Missing ⚠️

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.