Is your feature request related to a problem? Please describe.
Currently, when writing to DynamoDB, duplicate keys or existing items are always overwritten without any control.
This makes it difficult to handle scenarios where we want to update, skip, or explicitly error out.
Describe the solution you'd like
- Provide options for handling existing items:
- overwrite (default)
- update/merge
- skip
- error
- Provide options for handling duplicate keys within the same batch:
- last one wins
- first one wins
- skip
- error
- Allow users to configure this behavior in the step parameters.
Describe alternatives you've considered
- Preprocessing input files to remove duplicates before execution.
- Manually checking for existence with
query/scan before writing, but this increases complexity and cost.
Additional context
This was mentioned in the Additional Information of [PR #460]
Is your feature request related to a problem? Please describe.
Currently, when writing to DynamoDB, duplicate keys or existing items are always overwritten without any control.
This makes it difficult to handle scenarios where we want to update, skip, or explicitly error out.
Describe the solution you'd like
Describe alternatives you've considered
query/scanbefore writing, but this increases complexity and cost.Additional context
This was mentioned in the Additional Information of [PR #460]