View source for C-Based Toolchain Hardening

[[C-Based Toolchain Hardening]] is a treatment of project settings that will help you deliver reliable and secure code when using C, C++ and Objective C languages in a number of development environments. This article will examine Microsoft and GCC toolchains for the C, C++ and Objective C languages. It will guide you through the steps you should take to create executables with firmer defensive postures and increased integration with the available platform security. Effectively configuring the toolchain also means your project will enjoy a number of benefits during development, including enhanced warnings and static analysis, and self-debugging code.

There are four areas to be examined when hardening the toolchain: configuration, preprocessor, compiler, and linker. Nearly all areas are overlooked or neglected when setting up a project. The neglect appears to be pandemic, and it applies to nearly all projects including Auto-configured projects, Makefile-based, Eclipse-based, Visual Studio-based, and Xcode-based.

The article will also detail steps which quality assurance personnel can perform to ensure third party code meets organizational standards. Many organizations have Security Testing and Evaluation (ST&E) programs or operate in the US Federal arena where supply chain audits are necessary. If you audit a program with a lot of gaps, it could indicate the company providing the binaries does not have a mature engineering process or has gaps in its internal QA processes. For those who lack mature quality assurance or acceptance and testing criteria, then this article will also provide helpful suggestions. 

Proper use of auditing tools such as <tt>[http://www.trapkit.de/tools/checksec.html checksec]</tt> and <tt>[http://linux.die.net/man/1/readelf readelf]</tt> on Linux and <tt>[http://www.microsoft.com/en-us/download/details.aspx?id=11910 BinScope]</tt> on Windows means source code will be rarely needed for some portions of an audit. Lack of source code clears a number of legal obstacles in the acceptance testing process since NDAs or other agreements may not be required. For those who are not aware, the US's DMCA (PUBLIC LAW 105–304) has proper exceptions for reverse engineering and security testing and evaluation. The RE exemption is in Section 1205 (f) REVERSE ENGINEERING; and the ST&E exemption is in Section 1205 (i) SECURITY TESTING. If you don't need source code access, then you can decompile, re-engineer, and test without the need for consent or worry of reprisals.

This is a prescriptive article, and it will not debate semantics or speculate on behavior. As such, it will specify semantics, assign behaviors, and present a position. The semantics, behaviors, and position have worked extremely well in the past for the author (Jeffrey Walton). They have been so effective in his development strategy that code for secure channels and secure containers in the field often goes years between bug reports.

Finally, the OWASP [http://code.google.com/p/owasp-esapi-cplusplus/source ESAPI C++] project eats its own dog food. Many of the examples you will see in this article come directly from the ESAPI C++ project.

== Wisdom ==

Code '''must''' be correct. It '''should''' be secure. It '''can''' be efficient.

[http://en.wikipedia.org/wiki/Jon_Bentley Dr. Jon Bentley]: ''"If it doesn't have to be correct, I can make it as fast as you'd like it to be"''.

[http://en.wikipedia.org/wiki/Gary_McGraw Dr. Gary McGraw]: ''"Thou shalt not rely solely on security features and functions to build secure software as security is an emergent property of the entire system and thus relies on building and integrating all parts properly"''.

== Introduction ==

Authors in the toolchain work hard to help projects deliver reliable, secure, and efficient programs. Their efforts are available to you at all stages of the engineering processes - from project configuration and preprocessing to compiling and linking. For example, its non-trivial to ensure line numbers in source files match debug information due to the processing of <tt>#include</tt>, considering projects can be configured with no macros, the <tt>DEBUG</tt> macro, or the <tt>NDEBUG</tt> macro.

Compiler writers also provide a rich set of warnings from the analysis of code during compilation. Both GCC and Visual Studio have static analysis capabilities to help find mistakes early in the development process. In addition, both tool chains have options to produce a hardened executable by taking advantage of the security offered by the platform. Since users expect trouble free and safe code, it would be wise if you used all tools available in your war chests.

The built in static analysis capabilities of GCC and Visual Studio are usually sufficient to ensure proper API usage and catch a number of mistakes such as using an uninitialized variable or comparing a negative signed int and a positive unsigned int. As a concrete example, (and for those not familiar with C/C++ promotion rules), a warning will be issued if a signed integer is promoted to an unsigned integer and then compared because a side effect is <tt>-1 > 1</tt> after promotion! GCC and Visual Studio will not currently catch, for example, SQL injections and other tainted data usage. For that, you will need a tool designed to perform data flow analysis or taint analysis.

Some in the development community resist static analysis or refute its results. For example, when static analysis warned the Linux kernel's <tt>sys_prctl</tt> was comparing an unsigned value against less than zero, Jesper Juhl offered a patch to clean up the code. Linus Torvalds howled “No, you don't do this… GCC is crap” (referring to compiling with warnings). For the full discussion, see ''[http://linux.derkeiler.com/Mailing-Lists/Kernel/2006-11/msg08325.html <nowiki>[PATCH] Don't compare unsigned variable for <0 in sys_prctl()</nowiki>]'' from the Linux Kernel mailing list.

== Configuration ==

Configuration tools are popular on many Linux and Unix based systems, and they present the first opportunutiy for hardening. Configuration tools and auto tools include ''Autosetup'', ''Autoconf'', ''Automake'', ''config'', and ''Configure''. In addition to the command line tools, you also have Integrated Development Environments (IDE) such as Eclipse and Visual Studio. Both allow you to adjust settings for a project, and there are three opportunities at this stage. The first opportunity is optimizations (and debugging), the second is language and platform, and third project specific settings.

When using command line tools for auto configuration, there are files of interest worth mentioning when working with Linux and Unix command line tools. The files are part of the auto tools chain and include <tt>m4</tt> and the various <tt>*.in</tt>, <tt>*.ac</tt> (autoconf), and <tt>*.am</tt> (automake) files. A project often supply "one size fits all" settings in these files.

There are two downsides to some of the command line configuration tools in the toolchain: (1) security is often not a goal (for modern expectations of 'secure'), and (2) they cannot create configurations! The latter presents significant challenges even for switching between optimization levels (<tt>-O0</tt> or <tt>-O2</tt>) and debug levels (<tt>-g</tt> or <tt>-g3</tt>).

You will probably be disappointed to learn tools such as Autoconf and Automake miss many security related opportunities and ship insecure out of the box. There are a number of compiler switches and linker flags that improve the defensive posture of a program, but they are not 'on' by default. Tools like Autoconf - which are supposed to handle this situation - often provides setting to serve the lowest of all denominators.

A recent discussion on the Automake mailing list illuminates the issue: ''[https://lists.gnu.org/archive/html/autoconf/2012-12/msg00038.html Enabling compiler warning flags]''. Attempts to improve default configurations were met with resistance and no action was taken. The resistance is often of the form, "<nowiki><some obscure platform></nowiki> does not support <nowiki><established security feature></nowiki>" or "<nowiki><some useful warning></nowiki> also produces false positives". Its noteworthy that David Wheeler, the author of ''[http://www.dwheeler.com/secure-programs/ Secure Programming for Linux and Unix HOWTO]'', was one of the folks trying to improve the posture.

=== Optimizations ===

Optimizations and debug symbolication are controlled through two switches: <tt>-O</tt> and <tt>-g</tt>. You should use the following as part of your <tt>CFLAGS</tt> and <tt>CXXFLAGS</tt> to get the most from a debug session:

 * <tt>-O0 -g3 -ggdb</tt>

<tt>-O0</tt> turns off optimizations and <tt>-g3</tt> ensures maximum debug information is available. At <tt>-g3</tt>, symbolic constants and <tt>#defines</tt> are available under the debugger. Finally, <tt>-ggdb</tt> includes extensions to help with a debug session under GDB. For completeness, Jan Krachtovil stated <tt>-ggdb</tt> currently has no effect in a private email.

Conversely, you should use the following for release builds:

 * <tt>-O''n'' -g2</tt>

<tt>-O''n''</tt> sets optimizations for speed or size, and <tt>-g2</tt> ensure debugging information is created. Debugging information should be stripped and retained in case of symbolication for a crash report from the field. While not desired, debug information can be left in place without a performance penalty. See ''[http://gcc.gnu.org/ml/gcc-help/2005-03/msg00032.html How does the gcc -g option affect performance?]'' for details.

Another reason you might open files such as <tt>Makefile.in</tt> is to tune optimizations for an environment. Many default configurations simply use <tt>-O3</tt> as a hardcoded <tt>CFLAG</tt>. <tt>-Os</tt> is often a better choice in embedded and mobile spaces because devices do not have a page file, they are memory constrained memory, and minimizing code size helps keep the caches "hot".

=== Platforms ===

IDEs and command line tools offer opportunities to tune settings for a platform. IDEs usually provide the setting with point-and-click convenience; but you will usually need to verify and override settings from command line tools. For example, some command line configurations tools don't configure securely and don't honor command line settings, so the following might not produce expected results:

<pre>$ configure CFLAGS="-g2 -O2 -Wall -fPIE" LDFLAGS="-pie"</pre>

The ''unexpected'' loss of position independent code or layout randomizations is due to tools not detecting PIE or ALSR and ignoring <tt>CFLAGS</tt> and <tt>LDFLAGS</tt>. Because the auto tools don't enable some flags by default and ignore requested settings, you will need to open some of the template files (such as <tt>Makefile.in</tt>) and add the settings by hand to ensure the project is configured to specification.

Its truly unfortunate many projects do not honor a users' preferred settings since it so easy to do. Below is from [http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/Makefile ESAPI C++ Makefile]:

<pre># Merge ESAPI flags with user supplied flags. We perform the extra step to ensure 
# user options follow our options, which should give user option's a preference.
override CFLAGS := $(ESAPI_CFLAGS) $(CFLAGS)
override CXXFLAGS := $(ESAPI_CXXFLAGS) $(CXXFLAGS)
override LDFLAGS := $(ESAPI_LDFLAGS) $(LDFLAGS)</pre>

Finally, the problem is not limited to auto tools. Non-auto'd projects, such as [http://gcc.gnu.org/ml/gcc/2009-06/msg00697.html GCC] and [http://stackoverflow.com/questions/7827622/how-can-one-provide-custom-compiler-linker-flags-for-openssl OpenSSL], do the same.

=== Projects ===

Configuration at the project level presents opportunities to harden the application or library with domain specific knowledge. Foe example, suppose you are building OpenSSL, and you know (1) [http://www.schneier.com/paper-ssl-revised.pdf SSLv2 is insecure], (2) [http://www.yaksman.org/~lweith/ssl.pdf SSLv3 is insecure], and (3) [http://arstechnica.com/security/2012/09/crime-hijacks-https-sessions/ compression is insecure] (among others). In addition, suppose you don't use hardware and engines, and only allow static linking. Given the knowledge and specifications, you would configure the OpenSSL library as follows:

<pre>$ Configure darwin64-x86_64-cc -no-hw -no-engines -no-comp -no-shared -no-dso -no-sslv2 -no-sslv3 --openssldir=…</pre>

If you think the misconfiguration cannot be found or will be overlooked, then you should rethink you position. [[#Auditing|Auditing]] (discussed below) will reveal that compression is enabled, and the following command will bear witness:

<pre>$ nm /usr/local/ssl/iphoneos/lib/libssl.a 2>/dev/null | egrep -i "ERR_load_COMP_strings"</pre>

In fact, any symbol within the <tt>OPENSSL_NO_COMP</tt> preprocessor macro will bear witness since <tt>-no-comp</tt> is translated into a <tt>CFLAGS</tt> define.

=== Management ===

Suppose you want to support '''Debug''', '''Test''', and '''Release''' configurations. If you are working in an IDE such as Eclipse or Visual Studio, you will be in good shape and won't encounter any hardships. The development environment will manage the settings for you, and allow you to change those settings with point and click convenience.

Now, suppose you are using auto tools or a makefile. You would like to type <tt>make debug</tt>, <tt>make release</tt> or <tt>make test</tt> and build the desired configuration. You will probably be disappointed to learn [https://lists.gnu.org/archive/html/automake/2012-12/msg00019.html Automake does not support the concept of configurations]. Its not entirely Autoconf's or Automake's fault - ''Make'' is the underlying problem.

''Make'' has not evolved to support platforms undergoing evolution. For example, most modern platforms support executables and shared objects. Yet Make only supports one <tt>CFLAGS</tt> (and <tt>CXXFLAGS</tt>) and one <tt>LDFLAGS</tt>. Assuming you have a library and test suite and you want to use position independent code (or layout randomizations), it means you must make a choice: either <tt>-fPIE</tt> (compiler) and <tt>-pie</tt> (linker) for an executable; or <tt>-fPIC</tt> (compiler) and <tt>-shared</tt> for a shared object.

As another example, consider a modern engineering process that includes debug, test, and release configurations. Each would require different <tt>CFLAGS</tt>, and you would extract the relevant command and set <tt>CFLAGS</tt> like so (taken from [http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/Makefile ESAPI C++ Makefile]):

<pre># Makefile
DEBUG_GOALS = $(filter $(MAKECMDGOALS), debug)
ifneq ($(DEBUG_GOALS),)
  WANT_DEBUG := 1
  WANT_TEST := 0
  WANT_RELEASE := 0
endif
…

ifeq ($(WANT_DEBUG),1)
  ESAPI_CFLAGS += -DDEBUG=1 -UNDEBUG -g3 -ggdb -O0
  ESAPI_CXXFLAGS += -DDEBUG=1 -UNDEBUG -g3 -ggdb -O0
endif
…</pre>

Now consider what happens when you:(1) type <tt>make debug</tt>, and then type <tt>make release</tt>. Make will first build the program in a debug configuration for a session under the debugger. Next you want a release build to ensure there are no "optimized" side effects. Make will do nothing upon typing type <tt>make release</tt> because it considers everything up to date despite the fact that <tt>C{XX}FLAGS</tt> have changed. Hence, your program will actually be in a debug configuration and could <tt>SIGTERM</tt> at runtime because, for example, debug instrumentation is present (<tt>assert</tt> calls <tt>abort()</tt> when <tt>NDEBUG</tt> is '''not''' defined).

If you think its unlikely, then you should rethink your position. A number of projects have been guilty of executing instrumented code in production, including [http://web.nvd.nist.gov/view/vuln/detail?vulnId=CVE-2012-5689 critical infrastructure]. The defect is due to failures in the engineering process, including quality assurance controls.

== Preprocessor ==

The preprocessor is crucial to setting up a project for success. The C committee provided one macro - <tt>NDEBUG</tt> - and the macro can be used to derive a number of configurations and drive engineering processes. Unfortunately, the committee also left many related items to chance, which has resulted in programmers abusing builtin facilities. This section will help you set up you projects to integrate well with other projects and ensure reliability and security.

There are three areas to discuss when hardening the preprocessor. The first is well defined configurations which produce well defined behaviors, the second is useful behavior from assert, and the third is proper use of macros when integrating vendor code and third party libraries.

=== Configurations ===

To remove ambiguity, you should recognize two configurations: Release and Debug. Release is for production code on live servers, and its behavior is requested via the C/C++ <tt>NDEBUG</tt> macro. Its also the only macro observed by the C and C++ Committees and Posix. Diametrically opposed to release is Debug. While there is a compelling argument for <tt>!defined(NDEBUG)</tt>, you should have an explicit macro for the configuration and that macro should be <tt>DEBUG</tt>. This is because vendors and outside libraries use the same (or similar) macro for their configuration.

In addition to <tt>NDEBUG</tt> (Release) and <tt>DEBUG</tt> (Debug), you have two additional cross products: both are defined or neither are defined. Defining both should be an error, and defining neither should default to a release configuration. Below is from [http://code.google.com/p/owasp-esapi-cplusplus/source/browse/trunk/esapi/EsapiCommon.h ESAPI C++ EsapiCommon.h], which is the configuration file used by all source files:

<pre>// Only one or the other, but not both
#if (defined(DEBUG) || defined(_DEBUG)) && (defined(NDEBUG) || defined(_NDEBUG))
# error Both DEBUG and NDEBUG are defined.
#endif

// The only time we switch to debug is when asked. NDEBUG or {nothing} results
// in release build (fewer surprises at runtime).
#if defined(DEBUG) || defined(_DEBUG)
# define ESAPI_BUILD_DEBUG 1
#else
# define ESAPI_BUILD_RELEASE 1
#endif</pre>

When <tt>DEBUG</tt> is in effect, your code should receive full debug instrumentation, and redefine the behavior of <tt>assert</tt>. Debug instrumentation and asserts go hand-in-hand, and help ensure your code is self-debugging. Self-debugging code will not only validate parameters and return values, it will also assert those values. The assert will alert to a problem in the code (for example, an unexpected parameter), but their their true beauty is they will fire at any time there is a problem without the need to set a breakpoint. Its an incredible return on investment: write once, enjoy forever.

There is one problem with using asserts - [http://pubs.opengroup.org/onlinepubs/009604499/functions/assert.html Posix states an assert should call <tt>abort()</tt>] if <tt>NDEBUG</tt> is not defined. When debugging, <tt>NDEBUG</tt> will never be defined since you want the "program diagnostics" (quote from the Posix description). That makes <tt>assert</tt> and its accompanying <tt>abort()</tt> almost completely useless behavior. Asserts are discussed next.

Live servers running production code should always define <tt>NDEBUG</tt> (i.e., release configuration), which means they do not auto-abort (by indirectly calling <tt>abort()</tt> due to <tt>assert</tt>) or shit all over themselves because of "program diagnostics". Auto-abortion is not acceptable behavior, and anyone who asks for the behavior has (1) lowered his or her standards to the point of disgrace; or (2) is completely abusing the functionality of "program diagnostics".

=== ASSERT ===

Aborting during a debug session while trying to troubleshoot a problem is completely and utterly useless to the point its almost insulting. The result of "program diagnostics" calling <tt>abort()</tt> due to standard C/C++ behavior is disuse - developers simply don't use them. Its incredibly bad for the development community because self-debugging programs can nearly eradicate so many stability problems. Because self-debugging programs are so powerful, you will have to have to supply your own assert with improved behavior. Your assert will exchange auto-aborting behavior for auto-debugging behavior. The auto-debugging facility will ensure the debugger snaps when a problem is detected.

ESAPI C++ supplies its own assert with the behavior described above. In the code below, <tt>ASSERT</tt> raises <tt>SIGTRAP</tt> when in effect or it evaluates to <tt>void</tt> in other cases.

<pre>// A debug assert which should be sprinkled liberally. This assert fires and then continues rather
// than calling abort(). Useful when examining negative test cases from the command line.
#if (defined(ESAPI_BUILD_DEBUG) && defined(ESAPI_OS_STARNIX))
#  define ESAPI_ASSERT1(exp) {                                    \
    if(!(exp)) {                                                  \
      std::ostringstream oss;                                     \
      oss << "Assertion failed: " << (char*)(__FILE__) << "("     \
          << (int)__LINE__ << "): " << (char*)(__func__)          \
          << std::endl;                                           \
      std::cerr << oss.str();                                     \
      raise(SIGTRAP);                                             \
    }                                                             \
  }
#  define ESAPI_ASSERT2(exp, msg) {                               \
    if(!(exp)) {                                                  \
      std::ostringstream oss;                                     \
      oss << "Assertion failed: " << (char*)(__FILE__) << "("     \
          << (int)__LINE__ << "): " << (char*)(__func__)          \
          << ": \"" << (msg) << "\"" << std::endl;                \
      std::cerr << oss.str();                                     \
      raise(SIGTRAP);                                             \
    }                                                             \
  }
#elif (defined(ESAPI_BUILD_DEBUG) && defined(ESAPI_OS_WINDOWS))
#  define ASSERT(exp)             assert(exp)
#  define ESAPI_ASSERT1(exp)      assert(exp)
#  define ESAPI_ASSERT2(exp, msg) assert(exp)
#else
#  define ASSERT(exp)             ((void)(exp))
#  define ESAPI_ASSERT1(exp)      ((void)(exp))
#  define ESAPI_ASSERT2(exp, msg) ((void)(exp))
#endif</pre>

A program startup, a <tt>SIGTRAP</tt> handler is installed if one is not provided (for example, by an IDE or command line debugger):

<pre>struct DebugTrapHandler
{
  DebugTrapHandler()
  {
    struct sigaction new_handler, old_handler;

    do
      {
        int ret = 0;

        ret = sigaction (SIGTRAP, NULL, &old_handler);
        if (ret != 0) break; // Failed

        // Don't step on another's handler
        if (old_handler.sa_handler != NULL) break;

        new_handler.sa_handler = &DebugTrapHandler::NullHandler;
        new_handler.sa_flags = 0;

        ret = sigemptyset (&new_handler.sa_mask);
        if (ret != 0) break; // Failed

        ret = sigaction (SIGTRAP, &new_handler, NULL);
        if (ret != 0) break; // Failed

      } while(0);
  }

  static void NullHandler(int /*unused*/) { }

};

// We specify a relatively low priority, to make sure we run before other CTORs
// http://gcc.gnu.org/onlinedocs/gcc/C_002b_002b-Attributes.html#C_002b_002b-Attributes
static const DebugTrapHandler g_dummyHandler __attribute__ ((init_priority (110)));</pre>

=== Additional Macros ===

== Compiler ==

== Linker ==

== Runtime ==

After all the discussions on hardening in the development stage, there are still opportunities to improve the engineering process.

== Auditing ==

Auditing a binary for compliance against an SDLC focuses on the compilation phase and the linking phase.