Skip to content

Update to Adaptive Sort and Distribution Changes#156

Merged
rlawrenc merged 9 commits into
mainfrom
investigate_order_by
Apr 28, 2026
Merged

Update to Adaptive Sort and Distribution Changes#156
rlawrenc merged 9 commits into
mainfrom
investigate_order_by

Conversation

@xelArga
Copy link
Copy Markdown
Collaborator

@xelArga xelArga commented Apr 27, 2026

This PR includes various fixes to adaptive sort and it's related methods.

File Management

  • fixed naming convention for posix systems on temporary files and includes for windows for posix.

Sort Benchmark

  • these are the tests run for benchmarks, optional comment for repeated values vs UWA dataset on projection followed by sort

PlatformIO

  • tests were compiled in parallel for desktop dist causing unity.h include compilation errors, now test framework is set and custom run makes tests go in sequential runs

Adaptive Sort

  • heap calculation in shift up wasn't including the key offset, this is now added which eliminates the heap disordering the data in certain pages when called
  • cpu penalty added to make minsort less attractive with larger runs
  • write to read ratio updated to be larger than current value
  • both values can be tuned over time to be more accurate - current formula calculation wasn't selecting minsort for large runs with the current average calculations and the cpu penalty can help minsort be less chosen with larger data where average distinct runs gets flattened. Global calculation can also address this, but storage and calculation becomes a consideration.
  • removed frozen heap promotion logic in favour of new sublist starts being marked with newsublist variable and started in the main records left loop iteration

Added 2 more tests to order by for repeated values and in order data.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

Makefile Ubuntu Unit Test Results

  1 files  ±0   18 suites  ±0   0s ⏱️ ±0s
117 tests +2  117 ✅ +2  0 💤 ±0  0 ❌ ±0 
143 runs  +2  143 ✅ +2  0 💤 ±0  0 ❌ ±0 

Results for commit d726729. ± Comparison against base commit 6c9b9bc.

♻️ This comment has been updated with latest results.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 27, 2026

PlatformIO Ubuntu Unit Test Results

  1 files  ±0  108 suites  ±0   17s ⏱️ -12s
116 tests +2  116 ✅ +2  0 💤 ±0  0 ❌ ±0 
143 runs  +2  143 ✅ +2  0 💤 ±0  0 ❌ ±0 

Results for commit d726729. ± Comparison against base commit 6c9b9bc.

♻️ This comment has been updated with latest results.

Copy link
Copy Markdown
Contributor

@rlawrenc rlawrenc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work finding the error

@rlawrenc rlawrenc merged commit 0e8cfdb into main Apr 28, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants