Skip to content

Lambda function for out_of_vocabulary_token_option#24

Open
keshprad wants to merge 5 commits into
daltonfury42:masterfrom
keshprad:master
Open

Lambda function for out_of_vocabulary_token_option#24
keshprad wants to merge 5 commits into
daltonfury42:masterfrom
keshprad:master

Conversation

@keshprad

@keshprad keshprad commented Jul 3, 2021

Copy link
Copy Markdown
Contributor

This is a solution for lambda function feature request in #23

  1. Created out_of_vocabulary_handler
    • Added ability to handle lambda function
      • the lambda function gets token_og_case, which has the original casing
    • I realized that the current release is implicitly using .lower() if an invalid option for out_of_vocabulary_token_option is passed (eg: out_of_vocabulary_token_option = 'upper')
      • This is due to L128.
      • I'm now passing the token with original casing to the handler function. If an invalid option is passed, I will use .title()
  2. Added test case for lambda function.
  3. Updated README for out_of_vocab options

@keshprad

keshprad commented Jul 3, 2021

Copy link
Copy Markdown
Contributor Author

PR isn't ready to merge yet. I'm unsure how to address this, but would be a quick fix after we discuss

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant