Guide for hardware generation

[mntr31]

• A program is needed that tells which hardware needs to be generated for a model.
• In Gati, we use the arch_param.vh file to define what hardware blocks need to be generated for running a specific model.
• We can dynamically generate Leaky ReLU, Global Avg. Pool, Maxpool, FC, Resize, Reshape, Concat, etc.
• A parser is needed that parses any onnx model and tells FPGA devs which options to enable in arch_param.vh.

@shashankbhosagi Also, let us know your approach.

[shashankbhosagi]

• From hardware team I would need the complete list of operators that can be generated and it's one to one mapping to arch_params.vh file.
• say if I encounter MaxPool then this line needs to be added in arch_params
• I plan on generating the arch params file given a onnx file.

Note: instead of doing this as temporary thing, I have decided to take this under dynamic hardware generation feature which currently only generates the best architecture for the given onnx model.

[shashankbhosagi]

Update: I have a basic structure of how this will be done once I will talk with @mntr31 and then update here once I finish. I can now read onnx model and tell which operator is present or not.

[shashankbhosagi]

Update: I can now generate given the model whether to include these operator or not

+`define MEGA_POOL
+`define CONCAT
+`define FC
+`define ELTWISE_SIGMOID_TANH
+`define TRANSPOSE
+`define GLOBAL_POOL
+`define GEN_LEAKY_RELU

[shashankbhosagi]

Update:
Have added more macros here is the list

+`define ELTWISE_MULT_HW
+`define RESIZE
+`define IM2COL_BOUND_GEN_WIDTH
+`define N_MOD_STAGES

[mntr31]

If the FPGA developer is targeting to run 2-3 different models with a single bitstream, can we have a feature where we load multiple models and gaticc generates a gen_hardware.vh which includes all the hardware needed for all the models?

[shashankbhosagi]

Currently focusing on running a single model at a time but in future can do this 👍🏻

[mntr31]

• Element-wise block does addition, subtraction, mult and sigmoid/tanh.
• Mult and sigmoid/tanh are inside ifdef.
• By default element-wise block gets generated, and even without mult and sigmoid/tanh, it is taking unnecessary resources.

What we want

• We will put the whole eltwise block inside ifdef (`define ELTWISE).
• If the model has QLinearSigmoid or QLinearMul, then gaticc should define the newly made macro ELTWISE, and the ELTWISE_SIGMOID_TANH or ELTWISE_MULT_HW as sigmoid/tanh and mult can't be present without the eltwise block itself.
• Apart from that, QLinearAdd only needs ELTWISE. Currently, no model has subtraction.