Run TinyCC under SGX with 3i

[Yaxuan-w]

This document outlines how to integrate 3i and run TinyCC under SGX, building upon the previous work documented in Issue #6, which covers the integration of the main branch with Enarx. This issue will focus specifically on integrating 3i and running TinyCC.

Integrate 3i

This part has already been submitted via a PR #8, which primarily involves adding the grate closure logic to the setup process.

Running TinyCC

To run TinyCC under SGX with 3i, follow these steps:

A. TinyCC Source Code

Use the TinyCC source code available on GitHub: https://github.com/Yaxuan-w/tinycc

I have made some modifications, which have been pushed to the remote main branch. These changes are mainly intended to bypass Wasmtime's restrictions on callback (indirect call) function signatures. In Wasmtime, all indirect call parameters must match exactly, whereas normal glibc functions allow some tolerance. To address this, I made minimal changes (<10 LOC), mainly adding a placeholder argument in non-direct callback functions related to qsort. I then directly call qsort_r from glibc.

B. Required lind-wasm Branch

The branch of lind-wasm to be used with Enarx is run-enarx-3i. Since we have removed the dependency on Wasmtime's CLI, this version of lind-wasm cannot be compiled independently and must be compiled via Enarx.

C. Compilation and Running TinyCC

1. Ensure glibc is compiled: First, verify that glibc has been compiled successfully.
2. Configure and Compile TinyCC:
   • Use the bootstrap-wasm.sh script to configure:

   cd tinycc
   ./bootstrap-wasm.sh

   • Then run make. The first make will fail, but you need to manually run llvm-ranlib on libtcc.a to continue the build:

   make
   /home/alice/lind-wasm/clang+llvm-16.0.4-x86_64-linux-gnu-ubuntu-22.04/bin/llvm-ranlib libtcc.a
   make

   • Compile libtcc1.c using gcc for the target platform:

   gcc -m32 -O2 -DTCC_TARGET_I386 -c lib/libtcc1.c -o libtcc1.o

   • Optimize and prepare the final Wasm binary:

   /home/alice/lind-wasm/tools/binaryen/bin/wasm-opt --epoch-injection --asyncify -O2 --debuginfo tcc -o tcc.wasm

3. File Placement:
   After the build completes, place the generated library files into the appropriate locations in the temporary file system. For this example, the root of my temporary file system is /home/alice/lind-wasm/src/RawPOSIX/tmp. Use the following command:

cp /home/alice/tinycc/libtcc1.a /home/alice/lind-wasm/src/RawPOSIX/tmp/usr/local/lib/tcc/

4. Header Files:
   TinyCC has its own search logic, and we need to ensure that all header files are compatible with the target binary runtime platform. For ease of development, I have packaged the header files for the i386 target platform into a tar.gz file. You can directly extract these files into $FS_ROOT/usr/include.

D. Enarx Setup

Ensure that Enarx is installed as outlined in Issue #2. Use the following command to run TinyCC under Enarx, with the corresponding Enarx.toml configuration file:

ENARX_WASMCFGFILE=Enarx.toml enarx run /home/alice/tinycc/tcc.wasm