Question: DATADIR usage and configuration

[Aaronzinhoo]

Checklist

• [ x ] I have included information about relevant versions
• I have verified that the issue persists when using the master branch of Faust. (N/A)

Expected behavior

Use of DATADIR should contain metadata related to a Table is being used. The datadir in created here

Actual behavior

DATADIR is created but the tables folder is always empty. I am just curious if this is the expected behavior for the data dir folder and if/when would it be populated with any data? Is this some older feature that is no longer needed? if its not needed, is there a way to configure it such that we dont create the datadir folder if not needed?

Versions

• Python version 3.11.5
• Faust version 0.11.2
• Operating system Ubuntu 20.04
• Kafka version 3.7.0
• RocksDB version N/A

[dada-engineer]

Do you have an example code? IMO the tables folder is populated if you define a Table app.Table("name", key_serializer="str") and also write to this table table["John"] = 100

I do use this in one service and the rocksDB is put into this directory.

[Aaronzinhoo]

@dada-engineer some of the services I use dont utilize the a table. Some services are simple and just perform a transformation on data from topic and send the transformed into another topic. Regardless the DATADIR is made though which is strange to me since if we are not using the Table functionality.

[dada-engineer]

It's initialized regardless. This is expected. Do you have a hard blocker due to this?