Add caching for MultiPodBuilder solutions#502
Conversation
robknight
changed the title
robknight
changed the title
| /// disk) is logged and falls back to a direct solve, so callers always | ||
| /// get either a valid solution or a typed solver error - never a panic | ||
| /// from the cache layer. | ||
| #[cfg(any(feature = "mem_cache", feature = "disk_cache"))] |
There was a problem hiding this comment.
The current code-base assumes one cache is enabled. I don't think the crate compiles with no cache enabled, so this line seems redundant.
| /// [`MultiPodBuilder::layout_key`]: super::MultiPodBuilder::layout_key | ||
| #[derive(Clone, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] | ||
| #[non_exhaustive] | ||
| pub struct LayoutKey { |
There was a problem hiding this comment.
From this PR we now have:
- SolverInput: this defines the shape of the problem and is passed to the solver to find a solution
- LayoutKey: this is the symbolic shape of the problem
LayoutKey is required because SolverInput has some values that are independent of the shape, like the hashes of input pods.
Then you have ways to move between the two.
The solver should be capable of finding a solution with a symbolic problem definition. So I would like to consider a slightly different implementation: instead of having LayoutKey and SolverInput, make SolverInput symbolic.
Then there's no need to make synthetic SolverInput from LayoutKey, and I think the implementation will be more straight forward.
It would look something like this:
- MultiPod builds up the problem into MultiPodInput
- From MultiPodInput we generate InputShape
- The solver works with InputShape and returns OutputShape. This is cached (key: InputShape, value: OutputShape)
- From OutputShape and MultiPodInput now we can build MultiPodOutput
There was a problem hiding this comment.
A different way of expressing my proposal is: instead of adding layout.rs with new types and methods, try to rework the implementation of the solver and SolvedMultiPod so that they naturally connect with types that encode the sape without data (so the input and output of the solver is purely symbolic)
2 participants
Closes #489
Uses the existing POD2 memory or disk cache to store solution layouts for MultiPodBuilder. This can avoid the overhead of repeatedly solving for the layout of PODs that only differ in their concrete inputs.
Many of the PODs we want to build in real-world applications have standard layouts: we know exactly which operations will be needed, in which order, and how many input PODs will be required. This means that we should be able to reuse multi-POD layout solutions in those cases.