mcpp is a C preprocessor developed by kmatsui (Kiyoshi Matsui) based on the DECUS cpp written by Martin Minow, and then rewritten entirely. mcpp means Matsui cpp. This software is supplied as source codes, and to use mcpp in any systems, a small amount of compiler-system-specific modifications are required before it can be compiled into an executable. *1
This document explains how to port the source to different compiler systems. Please refer to the separate manual called "mcpp-manual.html" for the operating instructions of the generated executable.
All these sources and related documents are provided as an open-source-software.
Before going into detail, some of the mcpp features are introduced here.
Note:
*1 mcpp V.2.6.3 onward provides some binary packages too, at the following site. This document, however, does not explain them. As for the binary packages, see the web page.
mcpp is a portable preprocessor, supporting various operating systems, including Linux, FreeBSD and Windows. Its source has a wide portability, and can be compiled by any compilers which support Standard C (ANSI/ISO C). The library functions used are only the classic ones.
To port mcpp to each compiler system, in many cases, one only needs to change some macro definitions in the header files and simply compile it. In the worst case, adding several dozen of lines into a source file would be enough.
To process multi-byte characters (Kanji), it supports Japanese EUC-JP, shift-JIS and ISO2022-JP, Chinese GB-2312, Taiwanese Big-5 and Korean KSC-5601 (KSX 1001), as well as UTF-8. For shift-JIS, ISO2022-JP or Big-5, mcpp can complement the compiler-proper if it does not recognize them.
mcpp has various behavioral modes. Other than Standard-conforming mode, there are K&R 1st mode, "Reiser" cpp mode and what I call post-Standard mode. mcpp has also an execution option for C++ preprocessor.
Different from many existing preprocessors, Standard mode of mcpp has the highest conformance to Standards: all of C90, C99 and C++98. It has been developed aiming to become the reference model of the Standard C preprocessor. Those versions of the Standard can be specified by an execution option. *1
In addition, it provides several useful enhancements: #pragma MCPP debug, which traces the process of macro expansion or #if expression evaluation, and the header file "pre-preprocessing" facility.
mcpp also provides several useful execution options, such as warning level or include directory specification options.
Even if there are any mistakes in source, mcpp deals suitably with accurate plain diagnostic messages without running out of control or displaying misguiding error messages. It also displays warnings for portability problems. The detailed documents are also attached.
In spite of its high quality, mcpp code size and memory usage is relatively small.
A disadvantage of mcpp, if any, is slower processing speed. It takes two or three times time of GCC 3.*, 4.* / cc1, but seeing that its processing speed is almost the same as that of Borland C 5.5/cpp32 and that it runs a little bit faster when the header file pre-preprocessing facility is used, it cannot be described as particularly slow. mcpp puts an emphasis on standard conformance, source portability and operability in a small memory space, making this level of processing speed inevitable.
Validation Suite for Standard C Preprocessing, which is used to test the extent to which a preprocessor conforms to Standard C, its documentation cpp-test.html, which contains results of applying Validation Suite to various preprocessors, are also released with mcpp. When looking through this file, you will notice that so-called Standard-conforming preprocessors have so many conformance-related problems.
Note:
*1 ISO/IEC 9899:1990 (JIS X 3010-1993) had been used as C Standard, but in 1999, ISO/IEC 9899:1999 was adopted as a new Standard. This document calls the former C90 and latter C99. The former is generally called ANSI C or C89 because it migrated from ANSI X3.159-1989. ISO/IEC 9899:1990 plus its Amendment 1995 is sometimes called C95. C++ Standards are ISO/IEC 14882:1998 and its corrigendum version ISO/IEC 14882:2003. This document calls both of them C++98.
Though this document was text-file in the older versions, it has changed to html-file at V.2.6.2.
This document uses the following typographical conventions:
Note:
*1 The outline of the "Exploratory Software Project" can be seen at the following site (Japanese pages only).
mcpp from V.2.3 through V.2.5 had been located at:
In April 2006, mcpp project moved to:
The older version of mcpp, cpp V.2.2 and Validation Suite V.1.2 are located in the following Vector's web site. They are in the directory called dos/prog/c, but they are not for MS-DOS exclusively. Sources are for UNIX, WIN32, MS-DOS. The documents are Japanese only.
http://www.vector.co.jp/soft/dos/prog/se081188.html
http://www.vector.co.jp/soft/dos/prog/se081189.html
http://www.vector.co.jp/soft/dos/prog/se081186.html
The text files in these archive files available at Vector use [CR]+[LF] as a <newline> and encode Kanji in shift-JIS for DOS/Windows. On the other hand, those from V.2.3 through V.2.5 available at SourceForge use [LF] as a <newline> and encode Kanji in EUC-JP for UNIX. From V.2.6 on two types of archive, .tar.gz file with [LF]/EUC-JP and .zip file with [CR]+[LF]/shift-JIS, are provided.
The source of mcpp consists of five header files and seven *.c files. The parts which are dependent on OS or compiler system are included in the four source files configed.H, noconfig.H, system.H and system.c. Either of configed.H or noconfig.H is used depending the compiling method, they are never used simultaneously. There are also a few library function sources in system.c. When mcpp is compiled by any compiler system, these source files need to be modified to match that compiler system.
There are several types of MCPP executable corresponding to its building methods. The building methods of MCPP have following two axis:
The following sections from 3.1 through 3.9 explain compiler-specific-builds. "mcpp for GCC", "implemented for Visual C" or such in this document mean GCC-specific-build, Visual C-specific-build, respectively.
There are two ways to compile mcpp. The first is to automatically generate a header file named config.h and a Makefile by executing the 'configure' script. After generating them, just run 'make; make install'. The header file named configed.H will be used in this way. However, the configure script can only be used in UNIX-like systems and CygWIN or MinGW.
Another way is to 'make' using a makefile for each compiler system, with the modified/edited (if required) header file by difference files. noconfig.H will be used in this case. Difference files and makefiles are in the 'noconfig' directory. Even for systems which can use the configure script, editing header files and makefiles directly allows you to control compilation in detail. However, difference files are only available for supported compiler systems.
In this chapter, I explain how to compile mcpp using the difference files. Please refer to INSTALL for the configure script.
Note:
*1 While V.2.6 and V.2.6.1 called this as 'stand-alone-build', V.2.6.2 changed the name according to the creation of subroutine-build.
*2 mcpp V.2.6.3 and later provides some binary packages at the SourceForge site. They are all stand-alone and compiler-independent-builds.
The C/C++ compiler systems I could use are the following, and mcpp has been ported to all of these. Therefore, it has been verified that this source code can be compiled, and that generated preprocessors run correctly on each compiler system. In any case the CPU used is the x86 type. The systems are all 32 bit version, except Ubuntu which is 64 bit version.
FreeBSD 6.3 GCC V.3.4.6 Vine Linux 4.2 GCC V.2.95.3, V.3.2, V.3.3.6, V.3.4.3, V.4.1.1 Fedora Linux 9 GCC V.4.3.0 Debian LInux 4.0 GCC V.4.1.2 Ubuntu Linux 8.04 / x86_64 GCC V.4.2.3 Mac OS 10.5 GCC V.4.0.1 CygWIN 1.3.10 GCC V.2.95.3 CygWIN 1.5.18 GCC V.3.4.4 MinGW (MSYS 1.0.11) GCC V.3.4.5 WIN32 Visual C++ 2003, 2005, 2008 WIN32 Borland C++ V.5.5J WIN32 LCC-Win32 2003-08, 2006-03
In addition, there are informations from some users on Visual C++ V.6.0, Visual C++ 2002 and C++Builder 2007 (aka BCC V.5.9), and you can compile mcpp on them, too.
Settings are quite easy for creating mcpp executables by these compiler systems. One only needs to change some macro definitions in noconfig.H.
*.dif files in noconfig directory are difference files for modifying noconfig.H, which is by default for FreeBSD 6.* / GCC 3.4, to use with each compiler system.
For Visual C++ 2005, as an example, in the src directory, doing the following command modifies these files.
patch -c < ..\noconfig\vc2005.dif
Patch is a standard UNIX command, and has been ported to Windows or other. Of course, you can directly edit the source file referring the difference file without using patch.
Modifications to match your own systems, such as specifying include directory have to be done by yourself, apart from the modifications made by difference file.
Makefiles for each compiler system which are to compile these modified sources, are also attached. (See sec. 3.7)
Copy the makefile into the src directory as:
copy ..\noconfig\visualc.mak Makefile
All the following operations should be done in the src directory. These are all modifications of noconfig.H unless it is otherwise mentioned.
For any of the following compiler systems, in order to make the compiler-specific-build, change the macro INDEPENDENT of the line:
#define COMPILER INDEPENDENT
to the macro for the compiler system, for example:
#define COMPILER MSC
Next, change the line appropriately:
#define VERSION_MSG "GCC 3.4"
as:
#define VERSION_MSG "Visual C 2005"
You can also overwrite the definition of COMPILER by make option as:
nmake COMPILER=MSC nmake COMPILER=MSC install
If you modify noconfig.H applying the difference file, the compiler-specific setting will be also modified for the compiler system, so you need not rewrite the definition of COMPILER in the file. Then, if you do 'make' with option defining COMPILER, compiler-specific-build will be made, otherwise compiler-independent-build will be made.
In case of the default include directories are different from the ones in this file, the macros C_INCLUDE_DIR1 and C_INCLUDE_DIR2 should be rewritten. If C++ has its own include directories different from the ones in C, these should be written in CPLUS_INCLUDE_DIR1, CPLUS_INCLUDE_DIR2 and CPLUS_INCLUDE_DIR3. (These directories can be specified also by environment variables or the -I option at the time of execution.) All of these directories are of compiler-system-specific ones.
Include directories are also set in system.c. In UNIX terms, those set by system.c are OS-specific (usually /usr/include) and site specific (usually /usr/local/include). As for Windows, nothing is set for include directories in system.c nor in noconfig.H by default, they are to be specified by environment variables INCLUDE and CPLUS_INCLUDE.
If required, you should also change built-in macro names defined by the macros such as COMPILER_STD1 or COMPILER_STD2.
The default setting of multi-byte character encodings is set to EUC-JP on UNIX, shift-JIS on Windows. If required, modify the macro called MBCHAR to change the encoding. (The change of multi-byte character encoding can be done also by the environment variables, execution options and #pragma.)
On certain compiler systems, because they do not support encodings such as shift-JIS or Big5, the tokenization gets errors when there is the same value byte of 0x5c as '\\' within multi-byte characters. For these systems, mcpp needs special setting to compensate for an inability of the compiler. Please refer to sec 4.1.1.5 for this setting.
With regard to the attached makefiles, you need to rewrite BINDIR, which is the directory where the executables of the compiler system are located.
In GCC V.3, V.4, the preprocessor is absorbed into the compiler (ccl, cclplus). So, to use mcpp, you must replace the call of gcc, g++ with shell-script and set to execute first mcpp, then cc1 or cc1plus. The attached makefiles set this automatically by doing:
make COMPILER=GNUC make COMPILER=GNUC install
For the details, please see mcpp-manual.html#3.9.7.
When the user does not have write permission into the BINDIR, you must do 'sudo make COMPILER=GNUC install' on UNIX-like systems. On Windows, you must modify the permission of the directory by an administrator account, prior to installation.
The source is to be compiled by GCC (GNU C) V.3.4 on FreeBSD 6.* and to make mcpp of compiler-independent-build. In order to make the compiler-specific-build for FreeBSD 6.* / GCC V.3.4.*, first change the line:
#define COMPILER INDEPENDENT
to:
#define COMPILER GNUC
Then, just complete it by compiling. You can also overwrite COMPILER by 'make COMPILER=GNUC' command.
For the other version of GCC, modify the version number of the VERSION_MSG, and
#define COMPILER_EXT_VAL "3" #define COMPILER_EXT2_VAL "4" #define COMPILER_CPLUS_VAL "3" #define GCC_MAJOR_VERSION 3
For the first, write major version number of GCC, and for the second, write minor version number, both by string-literal. The third is value of the macro __GNUG__, which is the same with the first. And for the fourth, write the same number with the first by a digit.
If the version of FreeBSD is not 6.*, then change the following values.
#define SYSTEM_EXT_VAL "6" /* V.5.*: 5, V.6.*:6 */
Furthermore, in case of include directories are different from the default ones of FreeBSD 6.*, you need to change the following definition.
#define CPLUS_INCLUDE_DIR1 "/usr/include/c++/3.4" #define CPLUS_INCLUDE_DIR2 "/usr/include/c++/3.4/backward"
In some cases you may need to set also C_PLUS_INCLUDE_DIR3 and C_INCLUDE_DIR1.
If the version of GCC is 2.7-2.95, then change the following macro to 199409L.
#define STDC_VERSION 0L
Even for other UNIX-like OSes, if the compiler system is GCC, I suspect one only needs to change things like these version numbers, the setting of include directories or OS specific built-in macros. (See sec 4.1.1)
To change the setup for GCC on FreeBSD to GCC on Linux, you should change the line:
#define SYSTEM SYS_FREEBSD
to:
#define SYSTEM SYS_LINUX
Then modify the macros, as on FreeBSD, COMPILER, VERSION_MSG, COMPILER_EXT_VAL, COMPILER_EXT2_VAL, COMPILER_CPLUS_VAL, GCC_MAJOR_VERSION, CPLUS_INCLUDE_DIR1, CPLUS_INCLUDE_DIR2, C_INCLUDE_DIR1.
For GCC 2.* modify the value of STDC_VERSION. and change:
#define COMPILER_SP3_VAL "int"
to:
#define COMPILER_SP3_VAL "long int"
You should make sure the include directories by these commands:
gcc -xc -E -v /dev/null g++ -xc++ -E -v /dev/null
The difference files in 'noconfig' directory named linux_gcc2953.dif, linux_gcc32.dif, linux_gcc336.dif, linux_gcc343.dif and linux_gcc411.dif are for VineLinux 4.0 / GCC V.2.95.3, V.3.2, V.3.3.6, V.3.4.3 and V.4.1.1, respectively. For the compiler-specific-build, change COMPILER too. The include directories may vary between distributions of Linux. Also, if another version is installed in addition to the system standard version of GCC, it should create another include directory for the specific version. Specify the particular directory using the above macros.
The specification of getopt() of glibc is different from the standard ones such as POSIX, please use the mcpp_getopt() in system.c instead.
On Mac OS X, you can install GCC by installing the Xcode package.
After that installation, you will find many gcc, cc, g++, c++ or such in /usr/bin. In Mac OS X 10.5 Leopard on Intel-Mac, i686-apple-darwin9-gcc-4.0.1 and i686-apple-darwin9-g++-4.0.1 are the native compilers for the machine. The cross compilers are installed, too. On Intel-Mac, powerpc-apple-darwin9-gcc-4.0.1 and powerpc-apple-darwin9-g++-4.0.1 are the cross compilers to generate a binary for PowerPc. The names just gcc and g++ are symbolic links to gcc-4.0 and g++-4.0, which behave as native compilers by default, but when invoked with '-arch ppc' option, call the compiler-propers cc1 or cc1plus for powerpc. Note that these compilers are also installed into /Developer/usr/bin.
These are GCCs with many extensions specific to Mac OS X made by Apple. The compiler system of Mac OS X differs from the GCCs on other systems in some important aspects. First, it handles the special directories called "framework" as its system header directories. Second, it can generate both of the binaries for Intel-Mac and PowerPc-Mac on either machine. Moreover, it has a mechanism to make a "universal binary", which is a bundle of both binaries and is able to run on either machine. In fact, gcc-4.0, i686-apple-darwin9-gcc-4.0.1, powerpc-apple-darwin9-gcc-4.0.1 and their corresponding g++s and cc1, cc1plus and other compiler-propers in /usr/libexec/gcc/SYSTEM/4.0.1 are all universal binaries for i386 and ppc. (SYSTEM is i686-apple-darwin9 and powerpc-apple-darwin9.) If we copy these universal binaries from Intel-Mac to PowerPc-Mac, they will run reversing native and cross positions, I suppose. In addition, Intel-Mac even executes most of ppc binaries automatically translating to x86 codes.
To sum up: there are many gccs and g++s in /usr/bin and /Developer/usr/bin: also there are links to them; there are libexec directories for x86 and ppc: an executable contains two binaries bundled in it: a binary for ppc runs on x86. Such being the situation, we easily lose which is which. Be careful. Here, I take examples of Mac OS X 10.5 (Leopard) on Intel-Mac. On PowerPc-Mac, read these sections swapping i686 and powerpc (ppc). On Mac OS X 10.4 (Tiger), read darwin9 as darwin8.
It is quite simple to install mcpp by the native compiler. To make settings for Mac OS X 10.5 / GCC 4.0.1 on Intel-Mac, apply mac_gcc401_i686.dif to noconfig.H. Use mac_osx.mak as a Makefile. The command sequence 'make; sudo make install' will generate a compiler-independent-build, and 'make COMPILER=GNUC; sudo make COMPILER=GNUC install' will install a GCC-specific-build. The compilers in /usr/bin will be used for these commands, since usually /Devepoler/usr/bin is not set to $PATH.
To install mcpp on Intel-Mac with or for the cross compiler for PowerPc, apply mac_gcc401_powerpc.dif to noconfig.H. Then edit the Makefile (mac_osx.mak).
For a compiler-independent-build, change the definition of variables NAME, CC and CXX to those containing "powerpc" as noted by the comments in mac_osx.mak. Then, do 'make; sudo make install'. The binary is the one compiled "with the cross compiler", so should run on ppc-Mac.
For a GCC-specific-build, change the definition of variables NAME, INCDIR, BINDIR, target_cc and arch to those containing "powerpc" (do not change CC and CXX), and do 'make COMPILER=GNUC; sudo make COMPILER=GNUC install'. This is a binary "for the cross compiler" running on Intel-Mac, hence runs on Intel-Mac.
On PowerPc-Mac, mac_gcc401_powerpc.dif makes settings for the native compiler, and mac_gcc401_i686.dif do for the cross compiler, in reverse of Intel-Mac. To compile mcpp with or for the cross compiler for Intel-Mac on PowerPc-Mac, change the definition of the variables above to those containing "i686".
To make a universal binary, just enable the variable UFLAGS in mac_osx.mak removing the '#' which comments out the line. All the other settings are the same with the section above.
For CygWIN V.1.3.10 / GCC V.2.95.3, add the changes in cyg1310.dif to noconfig.H.
For CygWIN V.1.5.18 / GCC V.3.4.4, apply cyg1518.dif.
Then, rewrite the macro CYGWIN_ROOT_DIRECTORY to define CygWIN's root directory on Windows as:
#define CYGWIN_ROOT_DIRECTORY "C:/pub/compilers/cygwin"
The letters in the path-list are case-insensitive.
For other versions, it should be able to be ported by modifying macros such as VERSION_MSG, C_INCLUDE_DIR?, CPLUS_INCLUDE_DIR? and CYGWIN_ROOT_DIRECTORY.
Although CygWIN is a system on Windows, it simulates UNIX file system. Therefore, mcpp treats CygWIN/GCC in almost the same way with UNIX/GCC, and presets include directories as mcpp on UNIX.
For MinGW / GCC V.3.4.5, add the changes in mingw345.dif to noconfig.H. Then, rewrite the macro MSYS_ROOT_DIRECTORY and MINGW_DIRECTORY to define MSYS's / and /mingw directory on Windows as:
#define MSYS_ROOT_DIRECTORY "C:/Program Files/MSYS/1.0" #define MINGW_DIRECTORY "C:/Program Files/MinGW"
The letters in the path-list are case-insensitive.
For other versions, it should be able to be ported by modifying macros such as VERSION_MSG, C_INCLUDE_DIR?, CPLUS_INCLUDE_DIR?, MSYS_ROOT_DIRECTORY and MINGW_DIRECTORY. The path-list for the include directories may be either of absolute path as "c:/dir/mingw/include" or MinGW's own path as "/mingw/include".
Since MinGW does not support symbolic link, GCC-specific-build of mcpp cannot be invoked from gcc through symbolic link. Moreover, MinGW / gcc rejects to invoke a shell-script even if it is named cc1. Therefore, the compiling of mcpp generates an executable named cc1.exe instead of shell-script. In execution, gcc invokes this cc1.exe from which mcpp.exe or GCC's cc1.exe/cc1plus.exe are invoked.
Although the include directories are preset on GCC-specific-build, they are not set on compiler-independent-build, hence you should specify them by the environment variables INCLUDE and CPLUS_INCLUDE.
In LCC-WIN32 2003-08 or 2006-03, it needs to be changed as per lcc0308.dif, lcc0603.dif respectively. In other versions, the VERSION_MSG macro needs to be modified.
In Visual C++ 6.0, 2002, 2003, 2005, 2008, it needs modifications as vc6.dif, vc2002.dif, vc2003.dif, vc2005.dif, vc2008.dif respectively. For the compiler-specific-build, modify COMPILER or overwrite it by the nmake option, of course.
For other versions of Visual C, besides modifying VERSION_MSG macro, the values of predefined macros, _MSC_VER and _MSC_FULL_VER, should be changed by modifying the definition of COMPILER_EXT_VAL and COMPILER_EXT2_VAL respectively.
In Borland C V.5.5, V.5.9 (C++Builder 2007) / bcc32, it needs to be changed with bc55.dif, bc59.dif respectively.
In other versions of Borland C++, besides the VERSION_MSG macro, the values of predefined macros, __TURBOC__, __BORLANDC__ and __BCPLUSPLUS__ should be changed by modifying macros COMPILER_STD2_VAL, COMPILER_EXT_VAL and COMPILER_CPLUS_VAL, in noconfig.H. (Refer Sec 4.1.1.1.) If the version can handle digraphs, the definition of HAVE_DIGRAPHS needs to be changed. If the version has __STDC_VERSION__ macro, change the definition of STDC_VERSION.
The DECUS cpp seems to had supported RT-11/DECUS C and RSX/DECUS C on PDP-11, VMS/VAX-11C, PDP-11/UNIX and VAX/ULTRIX - some kind of C on VAX. It also seemed to have supported a quite old version of Microsoft C and Lattice C on MS-DOS. I removed these, as I suppose it is no longer required and I cannot maintain them.
system.H includes configed.H when the macro HAVE_CONFIG_H is defined to non-0, otherwise it includes noconfig.H. PART 1 and PART 2 of the mcpp setting are in configed.H and noconfig.H, and PART 3 is in system.H.
In these files, some macros which are required to port to each compiler system are defined. When porting to compiler systems which have not been ported to yet, one needs to add from a few lines to a dozen lines in Part 1.
Part 1 is the definition dependent on OS and target compilers, Part 2 is the definition dependent on host systems, and Part 3 is the definition of the mcpp behavior specification.
In configed.H and noconfig.H, the target compiler system is assumed to be the same as the host, so PART 2 needs to be modified when it is different.
When you do porting with different configurations from the default, please make sure to look through these files.
system.c absorbs the discrepancies of OS or compiler which cannot be absorbed solely by configed.H (noconfig.H) or system.H macros. To port to a new compiler system, adding tens of lines of source into this file may be required.
This file includes functions such as options for mcpp invocation, usage message, include directory, the handling of OS unique directory paths when opening header files or source files, processing of #pragma, and processing of compiler system unique extension directives. Most of them are setup for the target OS and target compiler systems.
Of library functions, C source code for getopt() and stpcpy(), which are not in Standard, are written in system.c. Though mcpp uses also getcwd(), stat() and in UNIXes readlink(), they are not included here, because they are functions dependent on OS, so cannot be written portably. They are only three low-level functions used in mcpp. Though they are not Standard C function, they are required by POSIX. Every compiler system seems to provide it. *1, *2
Usage of library functions in mcpp does not depend on the specification difference on different compiler systems, so those functions of any compiler systems will not cause a problem unless there is a bug.
Note:
*1 On MinGW, spawnv() is used too.
*2 mcpp up to V.2.6.4 had a separate source file lib.c. But, it was absorbed into system.c on V.2.7, since the functions written in it decreased to only two (getopt() and stpcpy()). At the same time, getopt() was renamed to mcpp_getopt() in order to prevent linking troubles.
In the source code of mcpp, stdio.h, string.h, stdlib.h, ctype.h, errno.h, limits.h, time.h, sys/types.h, sys/stat.h are included unconditionally. For UNIX-like systems, unistd.h is also included. There should not be a compiler system which does not have these.
*.mak are the makefiles for each compiler system, and a detailed setup is possible. 'make' command itself is assumed to the one which is attached to each compiler system or the standard for the system. For Visual C, 'nmake' should be used instead of 'make'.
Except for FreeBSD/GCC, modify the noconfig.H as follows: (Assume the system is xyz)
patch -c < ../noconfig/xyz.dif
Then edit macros COMPILER and VERSION_MSG, and edit the macros such as C_INCLUDE_DIR? in noconfig.H to suit your own system. After copying the corresponding noconfig/xyz.mak to Makefile, and setting up the target directory to match your system, run as
make make install make clean
For other compiler systems, please write the necessary makefile referring to these files. The dependencies of the source files are simple:
system.H needs to be included before internal.H.
internal.H further includes mcpp_lib.h.
To recompile mcpp using mcpp itself, place the executable into the location where the preprocessor of the compiler system exists. For instance, in the cases of GCC 2.95, rename the resident cpp0 to something like cpp0_gnuc and link cpp0 to whichever cpp you use at the time. Therefore, if mcpp is the preprocessor you are going to use, you need to do
ln -sf mcpp cpp0
For Windows, you need to copy the one you are going to use, to cpp32.exe or such. *1
You can name the executable of mcpp as:
make NAME=mcpp
(The same thing needs to be done in BC make requires make -DNAME=mcpp. For UCB make, -D can be either added or not. For GNU make, -D should not be added.)
Using the attached makefiles, 'make install' does not do any detailed work. Except for GCCs (i.e.: freebsd.mak, linux.mak, mac_osx.mak, cygwin.mak and mingw.mak), please do rest of the work manually. Please copy the resident preprocessor into the other name beforehand, so as to prevent being deleted by 'make install'.
When you recompile mcpp using the one-pass compiler such as Visual C or Borland C, you should supply the output file of mcpp as the source file to the compiler. (For instance, output the preprocessed result of source file main.c as main.i, and compile that with cl or bcc32.)
When recompiling using mcpp, if the "pre-preprocess" functionality for the header file is used, the preprocess time will be reduced dramatically. When you use the attached makefile, for UCB make, GNU make or MS nmake, you run
make PREPROCESSED=1
for BC make, you run
make -DPREPROCESSED=1
which automatically pre-preprocesses the header files, next preprocess, then compiles. For LCC-Win32's 'make', 'if' statement cannot be used, so you need to edit the makefile and recompile. The details of the modification are in the makefile itself as comments.
In BSD make, GNU make or MS nmake, if you run make with the option MALLOC=KMMALLOC, this links the malloc() which I wrote. About this, please refer to 4.extra. For BC make, the same thing can be done by the option -DKMMALLOC. To link my malloc() with the make of LCC-Win32, you need to edit the makefile.
Note:
*1 In FreeBSD, the standard directory in which preprocessor is located is /usr/libexec. See mcpp-manual.html#2.1. In Linux, it is located in the really deep directory as /usr/lib/gcc-lib/i686-redhat-linux/3.3.2. In Linux/GCC, according to the distribution or the version, this directory setting in the makefile needs to be modified. There are various different include directories, for which you need to check.
Also, in Linux or FreeBSD, there is /usr/bin/cpp which calls cpp0 or cc1, and gcc also calls cpp0 or cc1.
For further information, see mcpp-manual.html#3.9.5 and mcpp-manual.html#3.9.7. In GCC V.3 or V.4, the preprocessor is absorbed in the compiler (ccl, cclplus), so the call of gcc, g++ needs to be replaced with shell-scripts if you want to use mcpp.
Though some configuration is required to port to each compiler system, compiling mcpp's source code can be done by any compiler system which satisfies C90 specifications. *1, *2
The char type can be either signed or unsigned.
Floating point operation is not necessary.
This source code is written so as not be affected by the minor discrepancies of the compiler systems. Of course, it is necessary to avoid the compiler system's own bugs in order to actually compile with the compiler system. This cannot be found out until it has to be done. When I was porting to some compiler systems, there were a few cases which took me a long time to trace the bug and to find out a work around.
The compiler systems which mcpp does not support are those with special character sets or special CPU, as well as pre-C90 compilers.
EBCDIC is not supported.
The CPUs for which integer operation is not two's complement are also not supported. If it is not two's complement, it may run incorrectly when an overflow has occurred at a #if expression.
Note:
*1 Up to V.2.5, mcpp source was compilable even by K&R 1st compiler. From V.2.6, it presupposes C90 compiler, because K&R spec is no longer required by current compiler systems. I tidied up the source and this document accordingly.
*2 Up to V.2.6.2, mcpp source was compilable by C++, too. V.2.6.3 and later, however, is compilable only by C.
There is no need for the compiler system which compiles the mcpp source code (host) and the compiler system which will use the generated mcpp executable (target) to be the same. If these are different, select the target by SYSTEM and COMPILER and the host by HOST_SYSTEM and HOST_COMPILER within noconfig.H (configed.H). Also, the definitions in PART 1 are the settings for the target, and the ones in PART 2 are for the host. system.c is mainly for the target.
However, there are the following limitations.
By the way, the host and the target stated here are nothing to do with the ones in the cross-compiler. Cross-compiling is the job of the compiler itself, and in principle the preprocessor is not concerned about that. When mcpp is ported "to a cross-compiler", this cross-compiler is the target compiler system in here. As for the host compiler, you need to use the one which is not the cross-compiler. When mcpp is compiled "by a cross-compiler", the cross-compiler is the host compiler system, and the target of the cross-compiler becomes the target compiler system.
This section describes about the compiler systems which the older versions of mcpp had supported and later stopped support.
Although the following systems were supported in the older versions, mcpp V.2.4 removed the settings for them.
MS-DOS Turbo C V.2.0 OS-9/6x09 Microware C
The documents on the following compiler systems were removed in V.2.5.
DJGPP V.1.12 GCC V.2.7.1 MS-DOS LSI C-86 V.3.3 Trial Version
In V.2.6, the codes for the above two were removed, and codes and documents on the following compiler systems were removed.
MS-DOS Borland C 4.0 Plan 9 pcc
V.2.6 also removed all the settings for MS-DOS and other small memory systems, and removed the settings for pre-C90 compiler systems.
V.2.7.2 removed a setting for the following system.
Win32 Borland C 4.0
The above compiler systems are old ones, and users of these systems probably have gotten small now.
However, if you would like to compile mcpp with these compilers, you will easily succeed in making compiler-independent-build by the following procedures, as long as the compiler has most of C90 specifications, though compiler-specific-build is not easy because it requires various settings.
As for DJGPP, define SYSTEM, HOST_SYSTEM to SYS_WIN32 and HAVE_INTMAX_T, HAVE_INTTYPES_H to FALSE in noconfig.H, define NBUFF as about 1/4 of the default value in system.H. *1
As for the compiler systems on MS-DOS, define SYSTEM, HOST_SYSTEM to SYS_WIN32 and HAVE_LONG_LONG to FALSE in noconfig.H, define NBUFF as about 1/16 of the default, IDMAX as about 1/4 of the default in system.H, define SBSIZE as about 1/8 of the default in directive.c. Then compile with large memory model.
However, since memory is scarce on MS-DOS, you may get an "out of memory" error on preprocessing a source which has many long macro definitions, even if you have compiled mcpp with these settings.
Note:
*1 It was reported that DJGPP / GCC 4.1.0 successfully compiled mcpp V.2.6.1 with this setting.
mcpp can be built as an independent preprocessor which behaves on its own not depending on any compiler systems. Making a compiler-independent-build is quite easy, because the almost only requirement is that the compiler system can compile mcpp's source successfully. The invocation options and other specifications of the compiler-independent-build are the same across the compilers with which mcpp is compiled. The include directories are not preset except /usr/include and /usr/local/include in UNIX-like systems, hence you have to specify the rest of them by environment variables or by -I option. *1, *2
To make a compiler-independent-build with GCC, simply do in the mcpp's root directory:
./configure; make; make install
In this case, the header file configed.H is used. For further details of configuring, see the document INSTALL.
On a system where configure is not applicable, you can patch noconfig.H using the certain difference file in noconfig directory, if mcpp has already been ported to the compiler system. No other modification of source is needed. As a makefile, you can copy the corresponding *.mak file in noconfig directory, and edit the variable BINDIR to specify the installation directory. Then, in src directory, do 'make' and 'make install'.
In case of the version of the compiler differs a little from the already ported version, first apply the patch for the nearest version, then edit noconfig.H.
For the compiler systems to which mcpp is not yet ported, edit noconfig.H and modify or add several macros. First, define HOST_COMPILER appropriately. Next, define COMPILER as INDEPENDENT, and define VERSION_MSG appropriately. There is no target compiler for the compiler-independent-build, so nothing is required in PART 1.
PART 2 depends on the extent to which the host compiler implements the Standard's specifications, and also depends on whether the necessary functions are provided. The most often encountered discrepancy among the compilers is implementation of 'long long' or its corresponding data type. In Visual C 2002, 2003, 2005 and Borland C 5.5, the type is '__int64'. It's length modifier for printf() is 'I64', not 'j' nor 'll', except Visual C 2005. Hence, define the macro LL_FORM as "I64" for these compilers. On MinGW, the specifier is also "I64", though it has long long. In Visual C 2008, type name 'long long' is available.
If the compiler's library has not the function stpcpy(), define HOST_HAVE_STPCPY to FALSE.
Write makefile yourself referring the *.mak files. (Refer 3.7 too.)
Mac OS X provides a cross compiler as well as a native compiler so that we can make binaries for both of x86 and powerpc on either machine. Moreover, it provides a mechanism to make a "universal binary", which is a bundle of binaries for x86 and powerpc. Therefore, the settings necessary to compile mcpp on Mac OS X is a bit complex. I explain compiler-independent-build and compiler-specific-build together at 3.1.4.
By the way, at the SourceForge site, mcpp V.2.6.3 and later supplies some binary packages of compiler-independent-build. These are the ones packaged after each packaging method, and their packaging specifications are found in the setting files contained in each corresponding source package. All the packages on FreeBSD, Linux and Mac OS X use configure script to compile.
Note:
*1 In mcpp V.2.4 and V.2.5, the specification of the compiler-independent-build was a compromise with the compiler's specification. From V.2.6 on, the specification is its own and independent from the compiler.
*2 Even if you could compile mcpp on MS-DOS, you may run into shortage of memory on execution, since the memory is very scarce on MS-DOS. You must largely cut down translation limits before compiling. Refer 3.10.
mcpp can be compiled as a subroutine to be called from some other main program. Like a stand-alone preprocessor, this subroutine accepts execution options from the caller, preprocesses the specified input file, writes its result into the output file, then returns to the caller. It can be called repeatedly on several input files, if required. It does not, however, pass on the output token by token to its caller.
The subroutine-build also can write output into on-memory buffer.
mcpp is compiled as a subroutine when a macro MCPP_LIB is defined to non-0. The entry to this subroutine is mcpp_lib_main() which is a rename of main() in stand-alone-build, and is declared as:
int mcpp_lib_main( int argc, char ** argv);
If you use GCC, do not define the macro COMPILER to GNUC, leave it defined as INDEPENDENT, because GCC-specific setting generates an executable to be installed into GCC libexec directory and to be called from gcc command, which is not the desired one.
On the other hand, if you use Visual C, Borland C or LCC-Win32, you can define COMPILER to either of INDEPENDENT or one of MSC, BORLANDC, LCC, because these compilers have no independent preprocessor and they never call mcpp even if it is compiled with compiler-specific settings. Each compiler-specific setting defines the compiler-specific predefined macros, compiler-specific options, and some peculiar language specifications. You can choose either of compiler-specific or compiler-independent setting for your convenience.
There are two ways of compiling subroutine (library) build: using configure script and using noconfig/*.mak makefiles.
You can use 'configure' script, if you compile with GCC.
./configure --enable-mcpplib make sudo make install
On Linux and FreeBSD, these commands will generate libmcpp.a and libmcpp.so.$(SHLIB_VER) of compiler-independent-build, and will install them into /usr/local/lib by default. Then libmcpp.so and libmcpp.so.0 will be created as links to libmcpp.so.$(SHLIB_VER). The *.a is a static library, and the *.so is a shared library. The file named libmcpp.la will also be created, which is for the tool 'libtool'. $(SHLIB_VER) is 0.0.0 on mcpp V.2.6.3, 0.0.1 on V.2.6.4, 0.1.0 on V.2.7, 0.2.0 on V.2.7.1 and 0.3.0 on V.2.7.2. Also header files mcpp_lib.h and mcpp_out.h are installed into /usr/local/include, which are necessary for user program to use libmcpp.
On Mac OS X, the name of the shared library is *.dylib, not *.so. And, if you specify some CPUs by -arch option, such as "make CFLAGS+='-arch i386 -arch ppc'", a universal binary for those CPUs will be generated. In addition, you can widen the range of Mac OS X version on which the binary will run by -isysroot and -mmacosx-version-min= options. This is an example to generate a shared library on Leopard as a universal binary for i386 and ppc usable on Tiger.
make CFLAGS+='-isysroot /Developer/SDKs/MacOSX10.4u.sdk -mmacosx-version-min=10.4 -arch i386 -arch ppc'
On CygWIN and MinGW, DLL named *.dll will be generated instead of *.so. On CygWIN, libmcpp.a, libmcpp.dll.a, libmcpp.la will be generated and installed into /usr/local/lib, and cygmcpp-0.dll will be generated and installed into /usr/local/bin. On MinGW, those are the same with CygWIN, except cygmcpp-0.dll changes to libmcpp-0.dll. To use the DLL, link your main program against libmcpp.dll.a, which is an 'import library' for the DLL.
Also main_libmcpp.c is compiled and installed into /usr/local/bin under the name of mcpp which links the generated libmcpp.so. As you see in the source, you should include mcpp_lib.h to use libmcpp. *1
Minimal documents are installed too.
You can compile testmain.c as well and link it against one of the libraries. The configure does not do this. See 3.12.2.
Note:
*1 The compiler-independent-build of stand-alone mcpp, which does not use libmcpp, is installed into the same directory under the same name. So, they overwrite each other. As for documents, the same ones are installed into the same directory, so overwrite each other.
If you use 'noconfig.H' and the makefiles in 'noconfig' directory, (after applying an appropriate patch to noconfig.H according to the compiler and its version, and adjusting the directory settings in the makefile), do:
make MCPP_LIB=1 mcpplib make MCPP_LIB=1 mcpplib_install
While the above commands makes a compiler-independent-build, you can also make a compiler-specific-build by adding an option 'COMPILER=MSC' or the like, unless the compiler is GCC.
For Visual C, use 'nmake' instead of 'make'.
Since the 'make' command attached to LCC-Win32 cannot handle 'if' directive, you must edit the makefile whenever you do different 'make's.
On Linux, FreeBSD, Max OS X, CygWIN and MinGW, the result is the same with that of the configure above, except libmcpp.la is not created and the documents are not installed.
On Mac OS X, if you remove the '#' which comments out the definition of variable UFLAGS in mac_osx.mak, universal binary will be generated for each library.
On Windows, the name of the libraries are different from those on Linux as shown in the table below.
| FreeBSD / GCC | Linux / GCC | Mac OS X / GCC | CygWIN / GCC | MinGW / GCC | Visual C, Borland C, LCC-Win32 | |
|---|---|---|---|---|---|---|
| static library | libmcpp.a | libmcpp.a | libmcpp.a | libmcpp.a | libmcpp.a | mcpp.lib |
| shared library or DLL | libmcpp.so.$(SHL_VER) | libmcpp.so.$(SHLIB_VER) | libmcpp.$(SHLIB_VER).dylib | cygmcpp-$(DLL_VER).dll | libmcpp-$(DLL_VER).dll | mcpp$(DLL_VER).dll |
| import library of DLL | libmcpp-$(DLL_VER).dll.a | libmcpp-$(DLL_VER).dll.a | mcpp$(DLL_VER).lib |
$(SHL_VER) is 0, 1, 2, 3 on mcpp V.2.6.3-V.2.6.4, V.2.7, V.2.7.1, V.2.7.2, respectively. $(SHLIB_VER) is 0.0.0, 0.0.1, 0.1.0, 0.2.0, 0.3.0 on mcpp V.2.6.3, V.2.6.4, V.2.7, V.2.7.1, V.2.7.2, respectively. $(DLL_VER) is 0 on any of mcpp V.2.6.3, V.2.6.4, V.2.7, V.2.7.1 and V.2.7.2. If the first digit of $(SHLIB_VER) or $(DLL_VER) is the same, the versions with the higher second or third digit of $(SHLIB_VER) are upper-compatible to the one with lower second or third digit.
On Windows, so-called 'import library' is generated too, which is to use the DLL. In order to use the DLL, you must link your main program against this import library. The static library and the import library are installed into the $(LIBDIR) specified in the makefile, and the DLL itself is installed into the $(BINDIR). Note that any DLL must be located at some of the execution directories specified by the environment variable PATH on Windows.
mcpp_lib.h and mcpp_out.h are installed into $(INCDIR).
main_mcpplib.c is compiled and installed into $(BINDIR) under the name of mcpp, which links libmcpp. On Windows, an option 'DLL_IMPORT=1' specifies to link against DLL.
You can test libmcpp further by compiling testmain.c as a main program and linking it against one of the libraries as follows.
make testmain make testmain_install
testmain.c also has a sample to pass output via memory buffer. To enable the memory buffer, include the header file 'mcpp_lib.h' in your main program, and use the functions declared in it. The option 'OUT2MEM=1' will enable this in testmain.c. Note that the macro OUT2MEM is only for testmain.c, not for mcpp.
When you use the library-build of mcpp, write your makefile referring to noconfig/*.mak, and write your main program referring to testmain.c.
There are two kinds of library: static and shared. On Windows the latter is called DLL (dynamically-linked library). A static library is a collection of *.o (*.obj) files, which is linked (i.e. copied) to an executable at compile time. All the global names of functions and data in the static library are visible to the executable, and face to danger of name collisions. This is a problem for mcpp, since it had been developed without considering library use until V.2.6.1. On the other hand, a shared library (or a DLL) is linked at run time, and shared by several executables, if any, at the same time.
On Windows, the global names in the DLL are not visible from outside, and only the names explicitly exported can be imported. From the DLL of mcpp, for example, only the names in mcpp_lib.h can be imported.
On UNIXes, the global names in the shared library are all visible from outside by default, hence you have to be sensitive to name collisions. GCC 4.0 and later, however, can cope with this problem. From the shared library of mcpp V.2.7 and later, compiled by GCC V.4.0 or later, only the names in mcpp_lib.h are visible. *1, *2
To sum up, it is recommended to use DLL on Windows and shared library compiled by GCC 4.0 or later on UNIXes, because you need not to worry about name collision on these libraries.
Note:
*1 mcpp V.2.7 and later uses '#pragma GCC visibility *' directive which has been implemented since GCC V.4.0.
*2 Though GCC V.4.1 and later have '-fstack-protector' option, the option does not seem to coexist with '#pragma GCC visibility hidden' directive. So, the option can't be used to compile libmcpp.
I think you should be able to understand most of what is written in these header files if you read them. I have written lots of comments as well. In case, I write the following note.
noconfig.H (configed.H) contains PART 1 and PART 2 of the settings, and PART 3 is in system.H.
First, select the target system (the system for which mcpp is to be built) and the host system (the system which compiles mcpp.).
#define HOST_SYSTEM SYSTEM #define HOST_COMPILER COMPILER
Though there is a certain naming convention for SYSTEM and COMPILER, it is easier to see the source code. Though this is overstating it a bit, SYSTEM is only used for the type of path list of include files or to know the standard include directory of the OS, so one does not need to be concerned with it too much.
There are some other macros predefined in system.c according to run-time options, such as CPU-dependent ones. Besides, GCC V.3.3 or later predefines many macros, hence mcpp installation auto-generates specific 4 header files named mcpp_g*.h for those macros.
All the macros predefined by above settings become disabled by the -N option at execution time.
#line 123 "fname"The format of the above Standard C source code is set as default. Write an alternative sequence into the string literal to replace this "#line " for compilers which use other formats.
#123 "fname"If the above is the format, define as "# ". If it is a peculiar format, which is not any of the above, define the format to match. (In some cases, these may need to be added to sharp() or other functions in main.c)
The macro called MBCHAR is used to specify the type of encoding for multi-byte characters. In mcpp, all the following encodings are implemented at the same time. MBCHAR only specifies the default encoding, that can be changed by environment variables/options/#pragma at execution time (Refer mcpp-manual.#2.3, mcpp-manual.html#2.8, mcpp-manual.html#3.4 for how to use).
The first five are all encodings with a character occupying 2-bytes and without shift-states. Though wchar_t is a 4-byte type in some compiler systems, despite the encoding of multi-byte characters and wide characters being 2-byte, the preprocessor is not concerned with the type of wchar_t. As multi-byte or wide characters occupy 2-bytes on source code, it processes accordingly.
EUC_JP Japanese extended UNIX code (UJIS) SJIS Japanese shift-JIS (MS-Kanji) GB2312 Chinese EUC-like GB2312 (simplified-Chinese) BIGFIVE Taiwanese Big Five (traditional-Chinese) KSC5601 Korean EUC-like KSC-5601 (KSX 1001) ISO2022_JP International standard ISO-2022-JP1 Japanese UTF8 A type of encoding of Unicode, UTF-8
By the way, the behavior of the compiler as regards multi-byte characters may vary depending on the environment at execution time. Set these macros to match your environment. Regarding this, please refer to mcpp-manual.#2.8.
The next two are written in PART 2 for convenience. Set these TRUE when both target and host systems have the nominated type, otherwise set to FALSE.
In noconfig.H and configed.H, the target system is assumed to be the same as the host system. If not, PART 2 needs to be rewritten.
Also in PART 1, there are a few parts which assume the target is the same as the host. Modify it, if required. For example, the line using the predefined macro of the host compiler as:
#if _MSC_VER >= 1200
In system.H, there are macro definitions to specify the behavioral specification of mcpp.
There is a variable named 'mcpp_mode' in mcpp source, and the value of this variable determines the frame of mcpp behavior, such as macro expansion method, available preprocessing directives and predefined macros. There are following 4 modes (4 values of variable 'mcpp_mode') in mcpp. The mode of mcpp is specified by the run-time options. Therefore, in compiling mcpp, nothing is required to be set concerning these 4 macros. Nevertheless, you must understand the difference of these behavioral modes in order to set the other macros correctly.
There is another mode called COMPAT which is a variation of STD.
In this document, I group OLD_PREP and KR into pre-Standard mode, and group STD and POST_STD into Standard mode. For the details of the specification of these modes, refer to mcpp-manual.html#2.1.
Note:
*1 UCN is a C++98, C99 specification, notation of Unicode character value by hexadecimal escape sequence beginning with \u or \U. (See mcpp-manual.html#3.7, cpp-test.html#2.8, cpp-test.html#4.6).
The specification becomes better with bigger sizes for each, but the bigger the size of NWORK, NBUFF, NMACWORK or SBSIZE thus uses more memory. Other than the buffer, the actual memory consumption increases with the number of macro definitions. (Specifically, this is not the actual number of macro definitions themselves, but the total of each macro definition length, which is a problem. The internal format of macro definitions are written as 'struct defbuf' in internal.H)
NMACWORK, NEXP and RESCAN_LIMIT consumes stack.
Other settings do not need much memory, but it may be meaningless in real processing if the values are set to over the default ones within system.H.
The minimal limitations of translation limits required by C90 or C99 are written towards to the end of system.H. The translation limits of the C++98 are also written, but this is not the required specification, unlike the C Standards.
Some settings, mainly for the target compiler systems, are written here.
Source code for getopt() and stpcpy() among non-Standard library functions are written here. getopt() is renamed to mcpp_getopt() in order to prevent linking troubles. The stpcpy() is used when HOST_HAVE_STPCPY == FALSE.
"kmmalloc -- malloc() with debugging functions" is a portable source of malloc(), free(), realloc() and calloc() which I wrote. I wrote this to improve the memory efficiency and debugging convenience. I also attach the debug routine. Unexpected bugs can be caught if this is linked. *1, *2
The reason why I provide -DKMMALLOC -D_MEM_DEBUG -DXMALLOC options in noconfig/*.mak, is to link my malloc() which has debug routines. If the mcpp, linked with this, exits with error number EFREEP, EFREEBLK, EALLOCBLK, EFREEWRT or ETRAILWRT, it indicates a mcpp bug.
If you define any of BSD_MALLOC, DB_MALLOC or MALLOC_DBG to 1 and compile mcpp, with each debugging malloc() will be used, not my malloc(). In any case, to use the malloc() other than system library, you have to make the library before you compile. About this, please see the document of kmmalloc. (This document is written in Japanese only, sorry.)
Note:
*1 kmmalloc is at the following location.
*2 In CygWIN, my malloc() cannot be used as other malloc()s are not allowed to be used by the library structure.
Also, Visual C 2005 and 2008 have the same kind of problem.
"The Validation Suite for the Standard C conformance of preprocessing" is also made public with mcpp. I tried to make this be able to verify all the specifications of Standard C preprocessing. Of course, mcpp is checked by this suite. It was also compiled by all the above mentioned compiler systems and verified. Therefore, I don't think there are much bugs or wrong specifications, but there may have been some left. When porting to new compiler systems never ported before, it may be that there are some bugs of the compiler systems.
If you find unusual behavior, please contact me. Please check the following points.
If the diagnostic message of "Bug: ..." is displayed, that is definitely a bug of the mcpp or compiler systems (more like mcpp). Even if the mcpp goes out of control by processing jumbled "source", that is also a bug.
Of course, mcpp of modes other than STD behave "incorrectly" in the Validation Suite, as that is the specification. (Even that should not run uncontrollably). Please see 4.1.3 for details of the specifications.
There is a library called kmmalloc which I wrote, with functions such as malloc(). (Refer 4.extra
If mcpp is linked to my malloc() and exits out with the error number 120-124 (or 2120-2124 for some compilers), that is definitely the mcpp or compiler bug. (Possibly the library function's.) Also, if you write,
#pragma MCPP debug memory
somewhere in the sample source used in the test, the information for the heap memory will be output at that location and at the end. If the error message "Heap error: ..." is shown there, then that is also the mcpp or compiler system's bug.
If any bugs are found, please repeat the test by enclosing each part of the sample source by #if 0 and #endif, and mark out where the bug is.
Please attach the following data for the bug report.
I tried to write mcpp to be able to be ported relatively easily to any compiler system. However, I only have a small number of the compiler systems. Porting to other compiler systems will require adding some source code. I am looking forward to hearing about the porting reports to those compiler systems. I would like to feedback the reports into source.
Please include the following data in the porting report.
For the compiler-specific-build, to check whether it has been ported correctly, it may be easiest replacing the preprocessor first and then re-compiling mcpp itself by using the pre-preprocess functionality.
Furthermore, use the Validation Suite for STD mode. However, this requires lots of effort when repeating the debug since there are so many files. During the debug, at first, compile 'n_std.c' to see if this compiles and executes correctly. Some compiler drivers attached to the system may not have the option to pass to the mcpp, please refer to mcpp-manual.html#2.2 for that. Alternatively, you can first preprocess with mcpp, then pass the output to the compiler.
If it failed, check manually where the problem is by using the sample n_std.t. If this is a success, check e_std.t, m_*.t, unspcs.t, warns.t and misc.t. In "post-Standard" mode, n_post.t and e_post.t should be used.
Process these with mcpp -QCz23 option (except -3 for post-Standard). If mcpp is compiled with STDC == 0, add -S1 -V199409L option as well. As the comments will also be outputted by the -C option, you should be able to see that the process result is the expected one or not.
As the diagnostic messages are output to the file called mcpp.err by the -Q option, read it using a pager or similar. -z option omits the output of the header files.
Digraph and trigraph becomes valid by -2 or -3. -S1 and -V199409L sets __STDC__ to 1 and __STDC_VERSION__ to 199409L.
To test C99 compatibility, check n_std99.t, e_std99.t with -V199901L option.
If you use the program cpp_test.c in the Validation Suite, you can run the sample test of n_*.c, i_*.c automatically. (This is just to check yes and no, and this doesn't tell the details. Also, other tests such as e_*.?, u_*.?, unspcs.?, warns.? are not included. To test mcpp itself, it is quicker to compile n_std.c.)
Validation Suite has testcases for GCC / testsuite. Therefore, when mcpp is ported to one of the versions of GCC, mcpp's automatic test can be done by replacing the preprocessor of GCC to mcpp, if GCC / testsuite is installed. About this, please see cpp-test.html#3.2.3 and mcpp-manual.html#3.9.7.
mcpp provides the configure script available in UNIX-like systems. However, I do not have any idea for other compiler systems besides GCC in UNIX systems, so some options need to be specified to configure compiler-specific-build.
Someone who is using these compiler systems should know or be able to check about details of specifying these options. If you know, please let me know. I would like to do further work with the configure script.
Please refer INSTALL for the configure script.
When you can't port successfully, please let me know what is happening. If you attach the following data, I may be able to return the ported source.
In environments where the configure can be used, you can find out lots of data through its use.
By the way, from mcpp V.2.6 onward, pre-C90 compiler system is excluded from supporting.
/* t_line.c */
#include <stdio.h>
#line 1000
error line;
main(void)
{
return 0;
}
If the host compiler and the target compiler are different, I need all the above data for both systems.
To look at it like this, there are so many things to check, but practically, most of the compiler systems should have common characteristics with the ones already successfully ported, so it should not have too many problems to port for just running. The implementations of the execution options, #pragma and the non-standard specification will be the relatively time consuming ones. These can be done gradually after porting just to be able to run. The only annoying aspects are when one gets caught by compiler bugs.
The Validation Suite results of preprocessors for the compiler systems I have are summarized in cpp-test.#6.
Please let me know the result of testing with other compiler systems. It may be a bit of effort, as there are so many items.
The test by cpp_test.c does not take long, please send me this. In case of GCC, the automatic test can be done by the Validation Suite.
Besides reporting bugs, please send me feedback for anything, such as the handiness of mcpp, diagnostic messages, source code, Validation Suite, my interpretation of Standard C or the document writing method.
This preprocessor was created as a hobby, but it was the result of having devoted six and a half years, with lots of ideas even up until V.2.0. I want to make this the best, as much as I can, after such a work. About the C preprocessor, I think I have done almost everything meaningful, except testing and porting to the compiler systems I don't have. I would like to improve it if there are any problems that exist.
The code of Martin Minow was very clear, viceless and easy to understand, and I learned a lot by just reading this source code.
The people who are interested in this field may be very limited, but I am looking forward the feedback and the information.
Please send the information and the feedback to "Open Discussion Forum" at:
or by e-mail.
When I started messing about with DECUS cpp in Jan 1992, I had never even dreamed to take this long a stretch. I just thought I would change it a bit in the new years break.
Once I started, I realized I had to read the source properly and it took me about two months to read through. I did it because the source was worth reading as well. Then I revised some of the specification to adapt to C90. It was as planned till this point.
Then, I realized I did not really know the preprocessor specification of C90 precisely. When I read P. J. Plauger & Jim Brodie "Standard C" (1989), the function-like macro expansion methods turned my prejudice around completely. (A Japanese translation version of this part was miss-translated.) So I bought a copy of Standard C and I repeatedly read those difficult sentences related to preprocessing. As a result, I found the preprocessing of C90 is different in many points from the traditional one. The addition of #, ## operators are only a small part of them.
Significantly, I puzzled my brain about the function-like macro expansion routine. I thought it over for 2-3 weeks consulting the cpp source of E. Ream, and then I wrote the new macro expansion routine for C90. I have never used my brain so hard as for thinking the algorithm of the program. That was April, 1992.
Well, I thought I was over the hump and that the cpp playing was finished, but it took almost a further six years since then. There were not many problems that made me suffer during the rest. Nevertheless, it took so long. That was partly because I got bored thinking and couldn't concentrate on messing around with cpp. But that wasn't all. I did the following things.
In this list, the documentation took a long time. Especially the last four years, the time changing the source was only a little bit while most of the time was dominated by writing the documentation. Due to that, the documentation became such a volume, but the time taken was not only because of the volume. When I was writing the documents, the uncertain parts of the specification kept coming up. Each time I re-read the Standards, I revised the source code. The length of time revising the source was not a lot, but the number of times revising the source was a lot. Not only the preprocess specifications, but I also read well whole of the Standard including the Rationale of ANSI C. It's like I learned C90 by creating the preprocessor. Also, I could understand the problems of the C90 standard through this.
At first, I wrote a few test programs as samples. However, I found unexpected bugs each time I wrote samples and tested on mcpp. Then I decided to write the Validation Suite which would test every specification of the C90 preprocess. The problems of C90 became obvious by writing this Validation Suite. To comply to the irregular parts of C90 was such trouble and a bit meaningless for myself, but I am sure there were more meaningful things.
What I learned through this work are the following things.
This thinking is a sort of perfectionism. Things in the world mostly cannot be achieved by perfectionism, and software is not an exception. Nevertheless, there are some areas for which perfectionism has a very important role. The language processing systems may be one of them.
I can say that I could spend so many years, through and through, because this was my hobby. But six and half years was too long. I kept thinking about who would be going to use this after I spent so many years to create a perfect program. I think this must be the limit of the size for making a program as a hobby.
Anyway, as I have already done mcpp, I will keep maintaining it. Therefore, could everyone please send me feedback, bug reports or porting reports.
After releasing mcpp V.2.0, I updated to V.2.1, V.2.2 and then V.2.3. These updates were adapting to C99 or officially approved ISO / C++, increasing the supported systems or fixing bugs.
I could update quite easily until V.2.2. It only took three months from V.2.0 to V.2.2. In contrast, it took nearly four years from V.2.2 to V.2.3. The main reason was that I became busy and didn't have enough time to spend. I cut down my jobs to 4 days a week after turning to 60 years of age in July 2000, then I went back to playing with cpp again.
V.2.3 not only took time but took quite a lot of work as well. When I ported to GCC V.2.9x, I found out that I had to modify a lot to keep the compatibility with GCC/cpp. I added many options and implemented the extended specifications. Also I eased restrictions of the Standard by downgrading some errors to warnings or removing the highly frequent warnings from the default warning class.
Lots of those modifications are backward ones and were not enjoyable. Especially, maintaining both the C99 specification and the part of the "traditional" specification earlier than C90 was very much against my will. Unfortunately, this is a reality of the open source world, I had to meet to certain expectations.
By relaxing the restrictions of the standard, I think mcpp became easier to use also for the other compiler systems, in replacing the system resident preprocessor.
During the update to V.2.3, mcpp and Validation Suite was selected to 2002 "Exploratory Software Project" of Information-technology Promotion Agency, Japan (IPA). I found out about this project by chance and I entered. Then, the project manager Yutaka Niibe selected me. That is how the development went from July 2002 to Feb 2003 by IPA's funding and based on PM Niibe's advice. The translation of the documents is also taken by HighWell.
Though this was relatively small software, it became a kind of my life work after spending so much time. I had confidence with the quality, but I was disappointed without having an opportunity to be known to the world. Finally, the opportunity was given. To accomplish this project, I cut down my job to three days a week.
These were the things that I had intended to do in this project.
Then, Project Manager Niibe proposed the following points:
As I wanted to do these things too, I gratefully added these points to the project.
Actually, however, my project had delay after delay for various reasons. First, I was hit by a disc crash. Whenever I did new things, it took a long time as I had to use new software never used before. It was also the first time to compile GCC from the source, but also I had got a few problems. The updating of massive volumes of documents and the review and the correction of English version also took a considerable time. Furthermore, my mother was admitted to hospital. As a result, a part of the project, such as the support of the commercial compiler systems, had to be given up at the end.
As I had always done the way which is like digging a hole deeper and deeper, it took a long time when I had to try to widen the hole. When an amateur-programmer digs deeper into the matter, this is the only way to do it. Nevertheless, to make the result to go out into the world, the hole had to be widened to some extent.
During the process of widening up the hole, I managed to learn some new software and to be in the frontline of development while receiving the advice and the encouragements from Project Manager Niibe. Also, I was delighted to see my documents coming back in a flowing English. Though being pressed for time was a painful thing, each experience was fresh and fun.
This "Exploratory Software Project" did not finish there. Project Manager Ichiji also selected mcpp as a continual project for year 2003. This is how I started to do some unfinished tasks from the previous year, and also some areas which I did not have experience of before.
This time, my six year old PC experienced some troubles, and there were also further troubles during the upgrade of the hardware and OS. It also took time to learn the new software, and of course, the development was getting behind schedule. The condition of my mother, who had been out of hospital and in relatively good condition, became worse along with getting closer to the end of the project. This was also a source of my anxiety. (My mother died in February, 2004.) However, thanks to Project Manager Ichiji setting the due date to a reasonable time frame, I could work the tasks through thoroughly without rushing.
I accomplished tasks such as the porting to Visual C++, the creation of the configure script and supporting the various multi-byte character encodings. I also managed to do the clean-up of the source code which, though inconspicuous, can not be ignored by myself as the author. The time consuming work of updating the Japanese and English documents was accomplished with the co-operation of HighWell.
With these achievements, I was evaluated as one of the "super-creator"s of software by PM Ichiji! Though it may be overestimation for my ability, I think it is mcpp development over years that was recognized, and I am very glad.
I think mcpp has become the world's best quality C/C++ preprocessor, thanks to the "Exploratory Software Project" which took nearly two years. As a middle-aged amateur-programmer, I am satisfied with myself having done my best.
I am keeping on updates of mcpp even after the project. Many tasks are still to be done. To achieve the remaining tasks and to make mcpp widely known, I will continue to proceed steadily.