Ubuntu 20.04 support

[ParthPratim]

• ns3 version : ns-3.32
• DCE version : commit 38b6242
• Operating system/version : Ubuntu 20.04.1 LTS
• Libc version : ldd (Ubuntu GLIBC 2.31-0ubuntu9) 2.31
• compiler version : gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

Description of the problem

Branch to reproduce error: ubuntu-20.04-changes
On running the dce-iperf script on Ubuntu-20.04:

./waf --run dce-iperf

It throws the following error with Glibc:

Waf: Entering directory `/home/kiit/ltest3/bake/source/ns-3-dce/build'
Building library netlink
Building library dce
Building library dce-test
Building library dce-test-netlink
[302/322] Creating build/lib/pkgconfig/libns3-dev-netlink-debug.pc
[303/322] Creating build/lib/pkgconfig/libns3-dev-dce-debug.pc
Waf: Leaving directory `/home/kiit/ltest3/bake/source/ns-3-dce/build'
Build commands will be stored in build/compile_commands.json
'build' finished successfully (1.470s)
Don't know how to configure dynamic library path for the platform 'linux'; assuming it's LD_LIBRARY_PATH.
Fatal error: glibc detected an invalid stdio handle
Command ['/home/kiit/ltest3/bake/source/ns-3-dce/build/bin/dce-iperf'] terminated with signal SIGIOT. Run it under a debugger to get more information (./waf --run <program> --command-template="gdb --args %s <args>").

output of ./waf configure

Setting top to                           : /home/kiit/ltest3/bake/source/ns-3-dce 
Setting out to                           : /home/kiit/ltest3/bake/source/ns-3-dce/build 
Checking for 'gcc' (C compiler)          : /usr/bin/gcc 
Checking for cc version                  : 9.3.0 
Checking for 'g++' (C++ compiler)        : /usr/bin/g++ 
Checking for program 'pkg-config'        : /usr/bin/pkg-config 
Checking for pkg-config version >= '0.0.0' : yes 
Checking for -Wl,--soname=foo              : yes 
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for header stdint.h               : yes 
Checking for header inttypes.h             : yes 
Checking for header sys/inttypes.h         : not found 
Checking for header sys/types.h            : yes 
Checking for header sys/stat.h             : yes 
Checking for header dirent.h               : yes 
Checking for library dl                    : yes 
Checking for glibc get_cpu_features        : yes 
Checking for glibc secure_getenv           : yes 
Checking for glibc explicit_bzero          : yes 
Checking for header valgrind/valgrind.h    : not found 
Checking for header valgrind/memcheck.h    : not found 
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Could not find lib*-core-debug
Could not find lib*-core-debug
Checking for header hook-manager.h         : not found 
Checking for program 'valgrind'            : not found 
Checking for program 'doxygen'             : /usr/bin/doxygen 
Checking for header netinet/sctp.h         : not found 
Checking for 'gcc' (C compiler)            : /usr/bin/gcc 
Checking for 'g++' (C++ compiler)          : /usr/bin/g++ 
Checking for program 'python'              : /usr/bin/python3 
Checking for python version >= 2.3         : 3.8.5 
python-config                              : /usr/bin/python3-config 
Asking python-config for pyembed '--cflags --libs --ldflags --embed' flags : yes 
Testing pyembed configuration                                              : yes 
Asking python-config for pyext '--cflags --libs --ldflags' flags           : yes 
Testing pyext configuration                                                : yes 
Checking for pybindgen location                                            : ../pybindgen (guessed) 
Checking for python module 'pybindgen'                                     : 0.21.0.post8+g3a8e313 
Checking for pybindgen version                                             : 0.21.0.post8+g3a8e313 
PyBindGen                                                                  : ok 
Checking for code snippet                                                  : yes 
Checking for types uint64_t and unsigned long equivalence                  : no 
Checking for code snippet                                                  : no 
Checking for types uint64_t and unsigned long long equivalence             : yes 
Checking for the apidefs that can be used for Python bindings              : gcc-LP64 
Checking for internal GCC cxxabi                                           : complete 
Checking for python module 'pygccxml'                                      : not found 
---- Summary of optional NS-3 features:
Static build                  : not enabled (option --enable-static not selected)
Logging                       : enabled
Assert checks                 : enabled
Code coverage                 : not enabled (option --enable-gcov not selected)
Example programs              : enabled
Test programs                 : enabled
Debug Symbols                 : enabled
ELF magic loader              : not enabled (elf-loader not found)
Aspect-based tracing          : not enabled (libaspect not found)
sctp-tools-dev                : not enabled (sctp-tools (netinet/sctp.h) not found)
Python Bindings               : enabled
Python API Scanning Support   : not enabled (Missing 'pygccxml' Python module)
'configure' finished successfully (2.696s)

Steps to reproduce

I have managed to build DCE on an Ubuntu-20.04 machine and have also mentioned the Github branch below to reproduce the same.

• Clone into ubuntu-20.04-changes branch
• bake.py build

Things I did to get the DCE code to build :

• Cloned into ns-3.32-alignment branch
• Aligned a myscripts/mt3/dce-mt3.cc to the ns-3.32 API
• I identified that the libio.h and _G_config.h header files were missing from Ubuntu-20.04 systems which implemented the _IO_getc and _IO_putc functions as required in libc-ns3.h. Bug Report
• I had to commend out the lines DCE(_IO_getc) and DCE(_IO_putc) from libc-ns3.h.
• I had to include the header file cstdarg.h to replace _G_va_list with __gnuc_va_list.
• Included the header file linux/sockios.h to enable support for SIOCGSTAMP in test/test-socket.c.
• The issue with bake.py show showing libc6-dbg requirement not satisfied was thrown because it isn't looking for ld-2.31.so and can be resolved by applying the following patch. Just download this patch and run git apply bake.patch
• After applying the patch run bake.py configure -e dce-linux-dev

After building DCE when I ran the dce-iperf example it throws the following error
Fatal error: glibc detected an invalid stdio handle

I'm trying to work on this issue and would highly appreciate support from the DCE community.

Similar Issue : #57

[mas2tg]

I am actually facing the same issue with Ubuntu 18.04. Does anyone have any updates on this?

[tomhenderson]

@ParthPratim is looking into fixing this; he will update any progress here (collaborators would be welcome).

[tomhenderson]

Possibly relevant: https://sourceware.org/bugzilla/show_bug.cgi?id=20191

It seems like disabling the vtables hardening is a possible solution (if that can be done).

Another possibility is to maintain and build a patched glibc for use with DCE.

[ParthPratim]

I tried building glibc release 2.23 on an Ubuntu-20.04 machine but it fails with the following error:

/usr/bin/ld: /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libselinux.so: undefined reference to `gettid@GLIBC_2.30'

I'm trying to find some way to get it to build.

I'll also try disabling vtables hardening, if that's possible.

[teto]

IIRC that's a documented error, as in you need to add gettid to DCE. Btw do you need selinux ? this seems like a painful dep to have with dce

[ParthPratim]

The error with selinux comes up when I tried to build the glibc release 2.23, not the DCE source.

I'm thinking of building the release/2.31/master branch of glibc and then commenting out the following line from vtables.c

__libc_fatal ("Fatal error: glibc detected an invalid stdio handle\n");

This line causes the DCE scripts to stop execution when it detects any tampering with the vtable.

I've generated the custom glibc as a proper system root against which DCE could be built.
I'm trying to figure out which scripts and build files to change so that I could tweek the gcc build arguments to initiate build against the custom glibc and not the default glibc.

[mas2tg]

Thanks for looking into this. I am just wondering, does this issue prevent ns3-dce from working at all? I tried dce-udp-simple and dce-iperf, and I was wondering if all other binaries would fail as well (I believe the answer is yes).

[ParthPratim]

All the binaries are likely to fail because the issue arises due to stack smashing used in DCE which is why Glibc throws the _libc_fatal(...)

[tomhenderson]

Can someone try to add the flag '-fvtable-verify=none' to cxxflags and see if it helps? Perhaps search the ns-3-dce/wscript and add this flag wherever there is a setting of 'cxxflags' variable?

Perhaps other options are possible from here?
https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html

[mas2tg]

Hey, I just followed your suggestion and ran into the same issue...

[tomhenderson]

After browsing the glibc source a bit today, I'm skeptical we can fix this with a linker option. Perhaps static linking would fix it, but I don't think we want to do that.

So it seems like the options are that we build a custom glibc for ourselves (downside-- a complicated build, reported to need 5.5GB disk space), or explore whether some other libc can be used instead: http://www.etalabs.net/compare_libcs.html

We could also ask glibc maintainers about a linker flag to disable this check (in future releases).

[mas2tg]

Mmm... I wonder how hard it would be to change libc to musl. Even just using a custom glibc seems hard enough for me. Thanks for your help nonetheless!

[mas2tg]

BTW, I am able to successfully run DCE with Ubuntu 16.04 (not sure if this was public knowledge).

[tomhenderson]

Yes, 16.04 is the distribution that I presently use for DCE.