Skip to content

Support configurable overwrite/update behavior for existing items in DynamoDBWrite #530

@nasudadada

Description

@nasudadada

Is your feature request related to a problem? Please describe.
Currently, when writing to DynamoDB, duplicate keys or existing items are always overwritten without any control.
This makes it difficult to handle scenarios where we want to update, skip, or explicitly error out.

Describe the solution you'd like

  • Provide options for handling existing items:
    • overwrite (default)
    • update/merge
    • skip
    • error
  • Provide options for handling duplicate keys within the same batch:
    • last one wins
    • first one wins
    • skip
    • error
  • Allow users to configure this behavior in the step parameters.

Describe alternatives you've considered

  • Preprocessing input files to remove duplicates before execution.
  • Manually checking for existence with query/scan before writing, but this increases complexity and cost.

Additional context
This was mentioned in the Additional Information of [PR #460]

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementImprovement to an existing feature

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions